Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis

Yongsheng Pan, Mingxia Liu, Chunfeng Lian, Tao Zhou, Yong Xia, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

Multi-modal neuroimages (e.g., MRI and PET) have been widely used for diagnosis of brain diseases such as Alzheimer’s disease (AD) by providing complementary information. However, in practice, it is unavoidable to have missing data, i.e., missing PET data for many subjects in the ADNI dataset. A straightforward strategy to tackle this challenge is to simply discard subjects with missing PET, but this will significantly reduce the number of training subjects for learning reliable diagnostic models. On the other hand, since different modalities (i.e., MRI and PET) were acquired from the same subject, there often exist underlying relevance between different modalities. Accordingly, we propose a two-stage deep learning framework for AD diagnosis using both MRI and PET data. Specifically, in the first stage, we impute missing PET data based on their corresponding MRI data by using 3D Cycle-consistent Generative Adversarial Networks (3D-cGAN) to capture their underlying relationship. In the second stage, with the complete MRI and PET (i.e., after imputation for the case of missing PET), we develop a deep multi-instance neural network for AD diagnosis and also mild cognitive impairment (MCI) conversion prediction. Experimental results on subjects from ADNI demonstrate that our synthesized PET images with 3D-cGAN are reasonable, and also our two-stage deep learning method outperforms the state-of-the-art methods in AD diagnosis.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
EditorsAlejandro F. Frangi, Christos Davatzikos, Gabor Fichtinger, Carlos Alberola-López, Julia A. Schnabel
PublisherSpringer Verlag
Pages455-463
Number of pages9
ISBN (Print)9783030009304
DOIs
Publication statusPublished - 2018 Jan 1
Externally publishedYes
Event21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: 2018 Sep 162018 Sep 20

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11072 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018
CountrySpain
CityGranada
Period18/9/1618/9/20

Fingerprint

Alzheimer's Disease
Magnetic resonance imaging
Cycle
Modality
Model Diagnostics
Imputation
Missing Data
Neural Networks
Brain
Prediction
Experimental Results
Neural networks
Demonstrate
Learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., & Shen, D. (2018). Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. In A. F. Frangi, C. Davatzikos, G. Fichtinger, C. Alberola-López, & J. A. Schnabel (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings (pp. 455-463). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11072 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-00931-1_52

Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. / Pan, Yongsheng; Liu, Mingxia; Lian, Chunfeng; Zhou, Tao; Xia, Yong; Shen, Dinggang.

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. ed. / Alejandro F. Frangi; Christos Davatzikos; Gabor Fichtinger; Carlos Alberola-López; Julia A. Schnabel. Springer Verlag, 2018. p. 455-463 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11072 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Pan, Y, Liu, M, Lian, C, Zhou, T, Xia, Y & Shen, D 2018, Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. in AF Frangi, C Davatzikos, G Fichtinger, C Alberola-López & JA Schnabel (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11072 LNCS, Springer Verlag, pp. 455-463, 21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018, Granada, Spain, 18/9/16. https://doi.org/10.1007/978-3-030-00931-1_52
Pan Y, Liu M, Lian C, Zhou T, Xia Y, Shen D. Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. In Frangi AF, Davatzikos C, Fichtinger G, Alberola-López C, Schnabel JA, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. Springer Verlag. 2018. p. 455-463. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-00931-1_52
Pan, Yongsheng ; Liu, Mingxia ; Lian, Chunfeng ; Zhou, Tao ; Xia, Yong ; Shen, Dinggang. / Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. editor / Alejandro F. Frangi ; Christos Davatzikos ; Gabor Fichtinger ; Carlos Alberola-López ; Julia A. Schnabel. Springer Verlag, 2018. pp. 455-463 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{443bbc3d38ec49c4a1db98473cf87bba,
title = "Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis",
abstract = "Multi-modal neuroimages (e.g., MRI and PET) have been widely used for diagnosis of brain diseases such as Alzheimer’s disease (AD) by providing complementary information. However, in practice, it is unavoidable to have missing data, i.e., missing PET data for many subjects in the ADNI dataset. A straightforward strategy to tackle this challenge is to simply discard subjects with missing PET, but this will significantly reduce the number of training subjects for learning reliable diagnostic models. On the other hand, since different modalities (i.e., MRI and PET) were acquired from the same subject, there often exist underlying relevance between different modalities. Accordingly, we propose a two-stage deep learning framework for AD diagnosis using both MRI and PET data. Specifically, in the first stage, we impute missing PET data based on their corresponding MRI data by using 3D Cycle-consistent Generative Adversarial Networks (3D-cGAN) to capture their underlying relationship. In the second stage, with the complete MRI and PET (i.e., after imputation for the case of missing PET), we develop a deep multi-instance neural network for AD diagnosis and also mild cognitive impairment (MCI) conversion prediction. Experimental results on subjects from ADNI demonstrate that our synthesized PET images with 3D-cGAN are reasonable, and also our two-stage deep learning method outperforms the state-of-the-art methods in AD diagnosis.",
author = "Yongsheng Pan and Mingxia Liu and Chunfeng Lian and Tao Zhou and Yong Xia and Dinggang Shen",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-00931-1_52",
language = "English",
isbn = "9783030009304",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "455--463",
editor = "Frangi, {Alejandro F.} and Christos Davatzikos and Gabor Fichtinger and Carlos Alberola-L{\'o}pez and Schnabel, {Julia A.}",
booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings",

}

TY - GEN

T1 - Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis

AU - Pan, Yongsheng

AU - Liu, Mingxia

AU - Lian, Chunfeng

AU - Zhou, Tao

AU - Xia, Yong

AU - Shen, Dinggang

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Multi-modal neuroimages (e.g., MRI and PET) have been widely used for diagnosis of brain diseases such as Alzheimer’s disease (AD) by providing complementary information. However, in practice, it is unavoidable to have missing data, i.e., missing PET data for many subjects in the ADNI dataset. A straightforward strategy to tackle this challenge is to simply discard subjects with missing PET, but this will significantly reduce the number of training subjects for learning reliable diagnostic models. On the other hand, since different modalities (i.e., MRI and PET) were acquired from the same subject, there often exist underlying relevance between different modalities. Accordingly, we propose a two-stage deep learning framework for AD diagnosis using both MRI and PET data. Specifically, in the first stage, we impute missing PET data based on their corresponding MRI data by using 3D Cycle-consistent Generative Adversarial Networks (3D-cGAN) to capture their underlying relationship. In the second stage, with the complete MRI and PET (i.e., after imputation for the case of missing PET), we develop a deep multi-instance neural network for AD diagnosis and also mild cognitive impairment (MCI) conversion prediction. Experimental results on subjects from ADNI demonstrate that our synthesized PET images with 3D-cGAN are reasonable, and also our two-stage deep learning method outperforms the state-of-the-art methods in AD diagnosis.

AB - Multi-modal neuroimages (e.g., MRI and PET) have been widely used for diagnosis of brain diseases such as Alzheimer’s disease (AD) by providing complementary information. However, in practice, it is unavoidable to have missing data, i.e., missing PET data for many subjects in the ADNI dataset. A straightforward strategy to tackle this challenge is to simply discard subjects with missing PET, but this will significantly reduce the number of training subjects for learning reliable diagnostic models. On the other hand, since different modalities (i.e., MRI and PET) were acquired from the same subject, there often exist underlying relevance between different modalities. Accordingly, we propose a two-stage deep learning framework for AD diagnosis using both MRI and PET data. Specifically, in the first stage, we impute missing PET data based on their corresponding MRI data by using 3D Cycle-consistent Generative Adversarial Networks (3D-cGAN) to capture their underlying relationship. In the second stage, with the complete MRI and PET (i.e., after imputation for the case of missing PET), we develop a deep multi-instance neural network for AD diagnosis and also mild cognitive impairment (MCI) conversion prediction. Experimental results on subjects from ADNI demonstrate that our synthesized PET images with 3D-cGAN are reasonable, and also our two-stage deep learning method outperforms the state-of-the-art methods in AD diagnosis.

UR - http://www.scopus.com/inward/record.url?scp=85053889491&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053889491&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-00931-1_52

DO - 10.1007/978-3-030-00931-1_52

M3 - Conference contribution

AN - SCOPUS:85053889491

SN - 9783030009304

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 455

EP - 463

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings

A2 - Frangi, Alejandro F.

A2 - Davatzikos, Christos

A2 - Fichtinger, Gabor

A2 - Alberola-López, Carlos

A2 - Schnabel, Julia A.

PB - Springer Verlag

ER -