Automatic Chinese Meme Generation Using Deep Neural Networks

Wang Lin, Zhang Qimeng, Youngbin Kim, Ruizheng Wu, Hongyu Jin, Haoke Deng, Pengchu Luo, Chang Hun Kim

Research output: Contribution to journalArticlepeer-review

Abstract

Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder-decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones.

Original languageEnglish
Pages (from-to)152657-152667
Number of pages11
JournalIEEE Access
Volume9
DOIs
Publication statusPublished - 2021

Keywords

  • computer vision
  • Deep learning
  • image captioning
  • internet meme
  • meme generation

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint

Dive into the research topics of 'Automatic Chinese Meme Generation Using Deep Neural Networks'. Together they form a unique fingerprint.

Cite this