Estimating CT image from MRI data using 3D fully convolutional networks

Dong Nie, Xiaohuan Cao, Yaozong Gao, Li Wang, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

39 Citations (Scopus)

Abstract

Computed tomography (CT) is critical for various clinical applications, e.g., radiotherapy treatment planning and also PET attenuation correction. However, CT exposes radiation during CT imaging, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve any radiation. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiotherapy planning. In this paper, we propose a 3D deep learning based method to address this challenging problem. Specifically, a 3D fully convolutional neural network (FCN) is adopted to learn an endto-end nonlinear mapping from MR image to CT image. Compared to the conventional convolutional neural network (CNN), FCN generates structured output and can better preserve the neighborhood information in the predicted CT image. We have validated our method in a real pelvic CT/MRI dataset. Experimental results show that our method is accurate and robust for predicting CT image from MRI image, and also outperforms three state-of-the-art methods under comparison. In addition, the parameters, such as network depth and activation function, are extensively studied to give an insight for deep learning based regression tasks in our application.

Original languageEnglish
Title of host publicationDeep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings
PublisherSpringer Verlag
Pages170-178
Number of pages9
Volume10008 LNCS
ISBN (Print)9783319469751
DOIs
Publication statusPublished - 2016
Externally publishedYes
Event1st International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2016 and 2nd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2016 - Athens, Greece
Duration: 2016 Oct 212016 Oct 21

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10008 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other1st International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2016 and 2nd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2016
CountryGreece
CityAthens
Period16/10/2116/10/21

Fingerprint

Magnetic Resonance Imaging
Computed Tomography
Magnetic resonance
Tomography
Imaging techniques
Radiotherapy
Neural Networks
Neural networks
Radiation
Planning
Nonlinear Mapping
Activation Function
Attenuation
Regression
Chemical activation
Imaging
Output
Experimental Results

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Nie, D., Cao, X., Gao, Y., Wang, L., & Shen, D. (2016). Estimating CT image from MRI data using 3D fully convolutional networks. In Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings (Vol. 10008 LNCS, pp. 170-178). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10008 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46976-8_18

Estimating CT image from MRI data using 3D fully convolutional networks. / Nie, Dong; Cao, Xiaohuan; Gao, Yaozong; Wang, Li; Shen, Dinggang.

Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings. Vol. 10008 LNCS Springer Verlag, 2016. p. 170-178 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10008 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nie, D, Cao, X, Gao, Y, Wang, L & Shen, D 2016, Estimating CT image from MRI data using 3D fully convolutional networks. in Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings. vol. 10008 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10008 LNCS, Springer Verlag, pp. 170-178, 1st International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2016 and 2nd International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2016, Athens, Greece, 16/10/21. https://doi.org/10.1007/978-3-319-46976-8_18
Nie D, Cao X, Gao Y, Wang L, Shen D. Estimating CT image from MRI data using 3D fully convolutional networks. In Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings. Vol. 10008 LNCS. Springer Verlag. 2016. p. 170-178. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46976-8_18
Nie, Dong ; Cao, Xiaohuan ; Gao, Yaozong ; Wang, Li ; Shen, Dinggang. / Estimating CT image from MRI data using 3D fully convolutional networks. Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings. Vol. 10008 LNCS Springer Verlag, 2016. pp. 170-178 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{d5584cf12812445986b55d46350753d7,
title = "Estimating CT image from MRI data using 3D fully convolutional networks",
abstract = "Computed tomography (CT) is critical for various clinical applications, e.g., radiotherapy treatment planning and also PET attenuation correction. However, CT exposes radiation during CT imaging, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve any radiation. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiotherapy planning. In this paper, we propose a 3D deep learning based method to address this challenging problem. Specifically, a 3D fully convolutional neural network (FCN) is adopted to learn an endto-end nonlinear mapping from MR image to CT image. Compared to the conventional convolutional neural network (CNN), FCN generates structured output and can better preserve the neighborhood information in the predicted CT image. We have validated our method in a real pelvic CT/MRI dataset. Experimental results show that our method is accurate and robust for predicting CT image from MRI image, and also outperforms three state-of-the-art methods under comparison. In addition, the parameters, such as network depth and activation function, are extensively studied to give an insight for deep learning based regression tasks in our application.",
author = "Dong Nie and Xiaohuan Cao and Yaozong Gao and Li Wang and Dinggang Shen",
year = "2016",
doi = "10.1007/978-3-319-46976-8_18",
language = "English",
isbn = "9783319469751",
volume = "10008 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "170--178",
booktitle = "Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings",

}

TY - GEN

T1 - Estimating CT image from MRI data using 3D fully convolutional networks

AU - Nie, Dong

AU - Cao, Xiaohuan

AU - Gao, Yaozong

AU - Wang, Li

AU - Shen, Dinggang

PY - 2016

Y1 - 2016

N2 - Computed tomography (CT) is critical for various clinical applications, e.g., radiotherapy treatment planning and also PET attenuation correction. However, CT exposes radiation during CT imaging, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve any radiation. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiotherapy planning. In this paper, we propose a 3D deep learning based method to address this challenging problem. Specifically, a 3D fully convolutional neural network (FCN) is adopted to learn an endto-end nonlinear mapping from MR image to CT image. Compared to the conventional convolutional neural network (CNN), FCN generates structured output and can better preserve the neighborhood information in the predicted CT image. We have validated our method in a real pelvic CT/MRI dataset. Experimental results show that our method is accurate and robust for predicting CT image from MRI image, and also outperforms three state-of-the-art methods under comparison. In addition, the parameters, such as network depth and activation function, are extensively studied to give an insight for deep learning based regression tasks in our application.

AB - Computed tomography (CT) is critical for various clinical applications, e.g., radiotherapy treatment planning and also PET attenuation correction. However, CT exposes radiation during CT imaging, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve any radiation. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiotherapy planning. In this paper, we propose a 3D deep learning based method to address this challenging problem. Specifically, a 3D fully convolutional neural network (FCN) is adopted to learn an endto-end nonlinear mapping from MR image to CT image. Compared to the conventional convolutional neural network (CNN), FCN generates structured output and can better preserve the neighborhood information in the predicted CT image. We have validated our method in a real pelvic CT/MRI dataset. Experimental results show that our method is accurate and robust for predicting CT image from MRI image, and also outperforms three state-of-the-art methods under comparison. In addition, the parameters, such as network depth and activation function, are extensively studied to give an insight for deep learning based regression tasks in our application.

UR - http://www.scopus.com/inward/record.url?scp=84992513308&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84992513308&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46976-8_18

DO - 10.1007/978-3-319-46976-8_18

M3 - Conference contribution

AN - SCOPUS:84992513308

SN - 9783319469751

VL - 10008 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 170

EP - 178

BT - Deep Learning and Data Labeling for Medical Applications - 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016 Held in Conjunction with MICCAI 2016, Proceedings

PB - Springer Verlag

ER -