Megan: Mixture of experts of generative adversarial networks for multimodal image generation

David Keetae Park, Seungjoo Yoo, Hyojin Bahng, Jaegul Choo, Noseong Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.

Original languageEnglish
Title of host publicationProceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018
EditorsJerome Lang
PublisherInternational Joint Conferences on Artificial Intelligence
Pages878-884
Number of pages7
ISBN (Electronic)9780999241127
Publication statusPublished - 2018 Jan 1
Event27th International Joint Conference on Artificial Intelligence, IJCAI 2018 - Stockholm, Sweden
Duration: 2018 Jul 132018 Jul 19

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
Volume2018-July
ISSN (Print)1045-0823

Other

Other27th International Joint Conference on Artificial Intelligence, IJCAI 2018
CountrySweden
CityStockholm
Period18/7/1318/7/19

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Park, D. K., Yoo, S., Bahng, H., Choo, J., & Park, N. (2018). Megan: Mixture of experts of generative adversarial networks for multimodal image generation. In J. Lang (Ed.), Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018 (pp. 878-884). (IJCAI International Joint Conference on Artificial Intelligence; Vol. 2018-July). International Joint Conferences on Artificial Intelligence.

Megan : Mixture of experts of generative adversarial networks for multimodal image generation. / Park, David Keetae; Yoo, Seungjoo; Bahng, Hyojin; Choo, Jaegul; Park, Noseong.

Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018. ed. / Jerome Lang. International Joint Conferences on Artificial Intelligence, 2018. p. 878-884 (IJCAI International Joint Conference on Artificial Intelligence; Vol. 2018-July).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Park, DK, Yoo, S, Bahng, H, Choo, J & Park, N 2018, Megan: Mixture of experts of generative adversarial networks for multimodal image generation. in J Lang (ed.), Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018. IJCAI International Joint Conference on Artificial Intelligence, vol. 2018-July, International Joint Conferences on Artificial Intelligence, pp. 878-884, 27th International Joint Conference on Artificial Intelligence, IJCAI 2018, Stockholm, Sweden, 18/7/13.
Park DK, Yoo S, Bahng H, Choo J, Park N. Megan: Mixture of experts of generative adversarial networks for multimodal image generation. In Lang J, editor, Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018. International Joint Conferences on Artificial Intelligence. 2018. p. 878-884. (IJCAI International Joint Conference on Artificial Intelligence).
Park, David Keetae ; Yoo, Seungjoo ; Bahng, Hyojin ; Choo, Jaegul ; Park, Noseong. / Megan : Mixture of experts of generative adversarial networks for multimodal image generation. Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018. editor / Jerome Lang. International Joint Conferences on Artificial Intelligence, 2018. pp. 878-884 (IJCAI International Joint Conference on Artificial Intelligence).
@inproceedings{80724cf983fa4d2581e5c9bff60fcd3b,
title = "Megan: Mixture of experts of generative adversarial networks for multimodal image generation",
abstract = "Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.",
author = "Park, {David Keetae} and Seungjoo Yoo and Hyojin Bahng and Jaegul Choo and Noseong Park",
year = "2018",
month = "1",
day = "1",
language = "English",
series = "IJCAI International Joint Conference on Artificial Intelligence",
publisher = "International Joint Conferences on Artificial Intelligence",
pages = "878--884",
editor = "Jerome Lang",
booktitle = "Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018",

}

TY - GEN

T1 - Megan

T2 - Mixture of experts of generative adversarial networks for multimodal image generation

AU - Park, David Keetae

AU - Yoo, Seungjoo

AU - Bahng, Hyojin

AU - Choo, Jaegul

AU - Park, Noseong

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.

AB - Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.

UR - http://www.scopus.com/inward/record.url?scp=85055718770&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055718770&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85055718770

T3 - IJCAI International Joint Conference on Artificial Intelligence

SP - 878

EP - 884

BT - Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018

A2 - Lang, Jerome

PB - International Joint Conferences on Artificial Intelligence

ER -