TY - JOUR
T1 - Spatially-Constrained Fisher Representation for Brain Disease Identification with Incomplete Multi-Modal Neuroimages
AU - Pan, Yongsheng
AU - Liu, Mingxia
AU - Lian, Chunfeng
AU - Xia, Yong
AU - Shen, Dinggang
N1 - Funding Information:
Manuscript received February 8, 2020; accepted March 17, 2020. Date of publication March 24, 2020; date of current version August 31, 2020. The work of Yongsheng Pan and Yong Xia was supported in part by the National Natural Science Foundation of China under Grant 61771397, in part by the Science and Technology Innovation Committee of Shen-zhen Municipality, China, under Grant JCYJ20180306171334997, and in part by the Innovation Foundation for Doctor Dissertation of North-western Polytechnical University under Grant CX201835. The work of Mingxia Liu, Chunfeng Lian, and Dinggang Shen was supported in part by the National Institutes of Health (NIH) under Grant EB008374 and Grant AG041721. (Corresponding authors: Mingxia Liu; Yong Xia; Dinggang Shen.) Yongsheng Pan and Yong Xia are with the National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, China (e-mail: yspan@mail.nwpu.edu.cn; yxia@nwpu.edu.cn).
Publisher Copyright:
© 1982-2012 IEEE.
PY - 2020/9
Y1 - 2020/9
N2 - Multi-modal neuroimages, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), can provide complementary structural and functional information of the brain, thus facilitating automated brain disease identification. Incomplete data problem is unavoidable in multi-modal neuroimage studies due to patient dropouts and/or poor data quality. Conventional methods usually discard data-missing subjects, thus significantly reducing the number of training samples. Even though several deep learning methods have been proposed, they usually rely on pre-defined regions-of-interest in neuroimages, requiring disease-specific expert knowledge. To this end, we propose a spatially-constrained Fisher representation framework for brain disease diagnosis with incomplete multi-modal neuroimages. We first impute missing PET images based on their corresponding MRI scans using a hybrid generative adversarial network. With the complete (after imputation) MRI and PET data, we then develop a spatially-constrained Fisher representation network to extract statistical descriptors of neuroimages for disease diagnosis, assuming that these descriptors follow a Gaussian mixture model with a strong spatial constraint (i.e., images from different subjects have similar anatomical structures). Experimental results on three databases suggest that our method can synthesize reasonable neuroimages and achieve promising results in brain disease identification, compared with several state-of-the-art methods.
AB - Multi-modal neuroimages, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), can provide complementary structural and functional information of the brain, thus facilitating automated brain disease identification. Incomplete data problem is unavoidable in multi-modal neuroimage studies due to patient dropouts and/or poor data quality. Conventional methods usually discard data-missing subjects, thus significantly reducing the number of training samples. Even though several deep learning methods have been proposed, they usually rely on pre-defined regions-of-interest in neuroimages, requiring disease-specific expert knowledge. To this end, we propose a spatially-constrained Fisher representation framework for brain disease diagnosis with incomplete multi-modal neuroimages. We first impute missing PET images based on their corresponding MRI scans using a hybrid generative adversarial network. With the complete (after imputation) MRI and PET data, we then develop a spatially-constrained Fisher representation network to extract statistical descriptors of neuroimages for disease diagnosis, assuming that these descriptors follow a Gaussian mixture model with a strong spatial constraint (i.e., images from different subjects have similar anatomical structures). Experimental results on three databases suggest that our method can synthesize reasonable neuroimages and achieve promising results in brain disease identification, compared with several state-of-the-art methods.
KW - MRI
KW - Multi-modal neuroimage
KW - PET
KW - brain disease diagnosis
KW - fisher vector
KW - generative adversarial network
KW - incomplete data
UR - http://www.scopus.com/inward/record.url?scp=85090170507&partnerID=8YFLogxK
U2 - 10.1109/TMI.2020.2983085
DO - 10.1109/TMI.2020.2983085
M3 - Article
C2 - 32217472
AN - SCOPUS:85090170507
VL - 39
SP - 2965
EP - 2975
JO - IEEE Transactions on Medical Imaging
JF - IEEE Transactions on Medical Imaging
SN - 0278-0062
IS - 9
M1 - 9046025
ER -