Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image

Lei Xiang, Qian Wang, Dong Nie, Lichi Zhang, Xiyao Jin, Yu Qiao, Dinggang Shen

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

Recently, more and more attention is drawn to the field of medical image synthesis across modalities. Among them, the synthesis of computed tomography (CT) image from T1-weighted magnetic resonance (MR) image is of great importance, although the mapping between them is highly complex due to large gaps of appearances of the two modalities. In this work, we aim to tackle this MR-to-CT synthesis task by a novel deep embedding convolutional neural network (DECNN). Specifically, we generate the feature maps from MR images, and then transform these feature maps forward through convolutional layers in the network. We can further compute a tentative CT synthesis from the midway of the flow of feature maps, and then embed this tentative CT synthesis result back to the feature maps. This embedding operation results in better feature maps, which are further transformed forward in DECNN. After repeating this embedding procedure for several times in the network, we can eventually synthesize a final CT image in the end of the DECNN. We have validated our proposed method on both brain and prostate imaging datasets, by also comparing with the state-of-the-art methods. Experimental results suggest that our DECNN (with repeated embedding operations) demonstrates its superior performances, in terms of both the perceptive quality of the synthesized CT image and the run-time cost for synthesizing a CT image.

Original languageEnglish
Pages (from-to)31-44
Number of pages14
JournalMedical Image Analysis
Volume47
DOIs
Publication statusPublished - 2018 Jul 1

Fingerprint

Magnetic resonance
Tomography
Magnetic Resonance Spectroscopy
Neural networks
Neuroimaging
Prostate
Brain
Imaging techniques
Costs and Cost Analysis
Costs

Keywords

  • Deep convolutional neural network
  • Embedding block
  • Image synthesis

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Computer Graphics and Computer-Aided Design

Cite this

Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image. / Xiang, Lei; Wang, Qian; Nie, Dong; Zhang, Lichi; Jin, Xiyao; Qiao, Yu; Shen, Dinggang.

In: Medical Image Analysis, Vol. 47, 01.07.2018, p. 31-44.

Research output: Contribution to journalArticle

Xiang, Lei ; Wang, Qian ; Nie, Dong ; Zhang, Lichi ; Jin, Xiyao ; Qiao, Yu ; Shen, Dinggang. / Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image. In: Medical Image Analysis. 2018 ; Vol. 47. pp. 31-44.
@article{dac12b6328234b96ba86426e87805180,
title = "Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image",
abstract = "Recently, more and more attention is drawn to the field of medical image synthesis across modalities. Among them, the synthesis of computed tomography (CT) image from T1-weighted magnetic resonance (MR) image is of great importance, although the mapping between them is highly complex due to large gaps of appearances of the two modalities. In this work, we aim to tackle this MR-to-CT synthesis task by a novel deep embedding convolutional neural network (DECNN). Specifically, we generate the feature maps from MR images, and then transform these feature maps forward through convolutional layers in the network. We can further compute a tentative CT synthesis from the midway of the flow of feature maps, and then embed this tentative CT synthesis result back to the feature maps. This embedding operation results in better feature maps, which are further transformed forward in DECNN. After repeating this embedding procedure for several times in the network, we can eventually synthesize a final CT image in the end of the DECNN. We have validated our proposed method on both brain and prostate imaging datasets, by also comparing with the state-of-the-art methods. Experimental results suggest that our DECNN (with repeated embedding operations) demonstrates its superior performances, in terms of both the perceptive quality of the synthesized CT image and the run-time cost for synthesizing a CT image.",
keywords = "Deep convolutional neural network, Embedding block, Image synthesis",
author = "Lei Xiang and Qian Wang and Dong Nie and Lichi Zhang and Xiyao Jin and Yu Qiao and Dinggang Shen",
year = "2018",
month = "7",
day = "1",
doi = "10.1016/j.media.2018.03.011",
language = "English",
volume = "47",
pages = "31--44",
journal = "Medical Image Analysis",
issn = "1361-8415",
publisher = "Elsevier",

}

TY - JOUR

T1 - Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image

AU - Xiang, Lei

AU - Wang, Qian

AU - Nie, Dong

AU - Zhang, Lichi

AU - Jin, Xiyao

AU - Qiao, Yu

AU - Shen, Dinggang

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Recently, more and more attention is drawn to the field of medical image synthesis across modalities. Among them, the synthesis of computed tomography (CT) image from T1-weighted magnetic resonance (MR) image is of great importance, although the mapping between them is highly complex due to large gaps of appearances of the two modalities. In this work, we aim to tackle this MR-to-CT synthesis task by a novel deep embedding convolutional neural network (DECNN). Specifically, we generate the feature maps from MR images, and then transform these feature maps forward through convolutional layers in the network. We can further compute a tentative CT synthesis from the midway of the flow of feature maps, and then embed this tentative CT synthesis result back to the feature maps. This embedding operation results in better feature maps, which are further transformed forward in DECNN. After repeating this embedding procedure for several times in the network, we can eventually synthesize a final CT image in the end of the DECNN. We have validated our proposed method on both brain and prostate imaging datasets, by also comparing with the state-of-the-art methods. Experimental results suggest that our DECNN (with repeated embedding operations) demonstrates its superior performances, in terms of both the perceptive quality of the synthesized CT image and the run-time cost for synthesizing a CT image.

AB - Recently, more and more attention is drawn to the field of medical image synthesis across modalities. Among them, the synthesis of computed tomography (CT) image from T1-weighted magnetic resonance (MR) image is of great importance, although the mapping between them is highly complex due to large gaps of appearances of the two modalities. In this work, we aim to tackle this MR-to-CT synthesis task by a novel deep embedding convolutional neural network (DECNN). Specifically, we generate the feature maps from MR images, and then transform these feature maps forward through convolutional layers in the network. We can further compute a tentative CT synthesis from the midway of the flow of feature maps, and then embed this tentative CT synthesis result back to the feature maps. This embedding operation results in better feature maps, which are further transformed forward in DECNN. After repeating this embedding procedure for several times in the network, we can eventually synthesize a final CT image in the end of the DECNN. We have validated our proposed method on both brain and prostate imaging datasets, by also comparing with the state-of-the-art methods. Experimental results suggest that our DECNN (with repeated embedding operations) demonstrates its superior performances, in terms of both the perceptive quality of the synthesized CT image and the run-time cost for synthesizing a CT image.

KW - Deep convolutional neural network

KW - Embedding block

KW - Image synthesis

UR - http://www.scopus.com/inward/record.url?scp=85045465902&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045465902&partnerID=8YFLogxK

U2 - 10.1016/j.media.2018.03.011

DO - 10.1016/j.media.2018.03.011

M3 - Article

C2 - 29674235

AN - SCOPUS:85045465902

VL - 47

SP - 31

EP - 44

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

ER -