Medical image synthesis with context-aware generative adversarial networks

Dong Nie, Roger Trullo, Jun Lian, Caroline Petitjean, Su Ruan, Qian Wang, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

141 Citations (Scopus)

Abstract

Computed tomography (CT) is critical for various clinical applications, e.g., radiation treatment planning and also PET attenuation correction in MRI/PET scanner. However, CT exposes radiation during acquisition, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve radiations. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiation planning. In this paper, we propose a data-driven approach to address this challenging problem. Specifically, we train a fully convolutional network (FCN) to generate CT given the MR image. To better model the nonlinear mapping from MRI to CT and produce more realistic images, we propose to use the adversarial training strategy to train the FCN. Moreover, we propose an image-gradient-difference based loss function to alleviate the blurriness of the generated CT. We further apply Auto-Context Model (ACM) to implement a context-aware generative adversarial network. Experimental results show that our method is accurate and robust for predicting CT images from MR images, and also outperforms three state-of-the-art methods under comparison.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings
PublisherSpringer Verlag
Pages417-425
Number of pages9
Volume10435 LNCS
ISBN (Print)9783319661780
DOIs
Publication statusPublished - 2017
Externally publishedYes
Event20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017 - Quebec City, Canada
Duration: 2017 Sep 112017 Sep 13

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10435 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017
CountryCanada
CityQuebec City
Period17/9/1117/9/13

Keywords

  • Auto-context
  • Deep learning
  • GAN
  • Generative models
  • Image synthesis

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Medical image synthesis with context-aware generative adversarial networks'. Together they form a unique fingerprint.

  • Cite this

    Nie, D., Trullo, R., Lian, J., Petitjean, C., Ruan, S., Wang, Q., & Shen, D. (2017). Medical image synthesis with context-aware generative adversarial networks. In Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings (Vol. 10435 LNCS, pp. 417-425). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10435 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-66179-7_48