3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients

Dong Nie, Han Zhang, Ehsan Adeli, Luyan Liu, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

56 Citations (Scopus)

Abstract

High-grade glioma is the most aggressive and severe brain tumor that leads to death of almost 50% patients in 1–2 years. Thus,accurate prognosis for glioma patients would provide essential guidelines for their treatment planning. Conventional survival prediction generally utilizes clinical information and limited handcrafted features from magnetic resonance images (MRI),which is often time consuming,laborious and subjective. In this paper,we propose using deep learning frameworks to automatically extract features from multi-modal preoperative brain images (i.e.,T1 MRI,fMRI and DTI) of high-grade glioma patients. Specifically,we adopt 3D convolutional neural networks (CNNs) and also propose a new network architecture for using multi-channel data and learning supervised features. Along with the pivotal clinical features,we finally train a support vector machine to predict if the patient has a long or short overall survival (OS) time. Experimental results demonstrate that our methods can achieve an accuracy as high as 89.9% We also find that the learned features from fMRI and DTI play more important roles in accurately predicting the OS time,which provides valuable insights into functional neuro-oncological applications.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings
PublisherSpringer Verlag
Pages212-220
Number of pages9
Volume9901 LNCS
ISBN (Print)9783319467221
DOIs
Publication statusPublished - 2016
Externally publishedYes

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9901 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Fingerprint

Brain Tumor
Magnetic Resonance Image
Functional Magnetic Resonance Imaging
Survival Time
Magnetic resonance
Tumors
Brain
Imaging
Imaging techniques
Prediction
Prognosis
Supervised learning
Supervised Learning
Network Architecture
Network architecture
Support vector machines
Support Vector Machine
Planning
Neural Networks
Neural networks

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Nie, D., Zhang, H., Adeli, E., Liu, L., & Shen, D. (2016). 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings (Vol. 9901 LNCS, pp. 212-220). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9901 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46723-8_25

3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. / Nie, Dong; Zhang, Han; Adeli, Ehsan; Liu, Luyan; Shen, Dinggang.

Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. Vol. 9901 LNCS Springer Verlag, 2016. p. 212-220 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9901 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nie, D, Zhang, H, Adeli, E, Liu, L & Shen, D 2016, 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. vol. 9901 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9901 LNCS, Springer Verlag, pp. 212-220. https://doi.org/10.1007/978-3-319-46723-8_25
Nie D, Zhang H, Adeli E, Liu L, Shen D. 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. Vol. 9901 LNCS. Springer Verlag. 2016. p. 212-220. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46723-8_25
Nie, Dong ; Zhang, Han ; Adeli, Ehsan ; Liu, Luyan ; Shen, Dinggang. / 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. Vol. 9901 LNCS Springer Verlag, 2016. pp. 212-220 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{76d3296bb53d435cb0f684cc99ac4477,
title = "3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients",
abstract = "High-grade glioma is the most aggressive and severe brain tumor that leads to death of almost 50{\%} patients in 1–2 years. Thus,accurate prognosis for glioma patients would provide essential guidelines for their treatment planning. Conventional survival prediction generally utilizes clinical information and limited handcrafted features from magnetic resonance images (MRI),which is often time consuming,laborious and subjective. In this paper,we propose using deep learning frameworks to automatically extract features from multi-modal preoperative brain images (i.e.,T1 MRI,fMRI and DTI) of high-grade glioma patients. Specifically,we adopt 3D convolutional neural networks (CNNs) and also propose a new network architecture for using multi-channel data and learning supervised features. Along with the pivotal clinical features,we finally train a support vector machine to predict if the patient has a long or short overall survival (OS) time. Experimental results demonstrate that our methods can achieve an accuracy as high as 89.9{\%} We also find that the learned features from fMRI and DTI play more important roles in accurately predicting the OS time,which provides valuable insights into functional neuro-oncological applications.",
author = "Dong Nie and Han Zhang and Ehsan Adeli and Luyan Liu and Dinggang Shen",
year = "2016",
doi = "10.1007/978-3-319-46723-8_25",
language = "English",
isbn = "9783319467221",
volume = "9901 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "212--220",
booktitle = "Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings",

}

TY - GEN

T1 - 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients

AU - Nie, Dong

AU - Zhang, Han

AU - Adeli, Ehsan

AU - Liu, Luyan

AU - Shen, Dinggang

PY - 2016

Y1 - 2016

N2 - High-grade glioma is the most aggressive and severe brain tumor that leads to death of almost 50% patients in 1–2 years. Thus,accurate prognosis for glioma patients would provide essential guidelines for their treatment planning. Conventional survival prediction generally utilizes clinical information and limited handcrafted features from magnetic resonance images (MRI),which is often time consuming,laborious and subjective. In this paper,we propose using deep learning frameworks to automatically extract features from multi-modal preoperative brain images (i.e.,T1 MRI,fMRI and DTI) of high-grade glioma patients. Specifically,we adopt 3D convolutional neural networks (CNNs) and also propose a new network architecture for using multi-channel data and learning supervised features. Along with the pivotal clinical features,we finally train a support vector machine to predict if the patient has a long or short overall survival (OS) time. Experimental results demonstrate that our methods can achieve an accuracy as high as 89.9% We also find that the learned features from fMRI and DTI play more important roles in accurately predicting the OS time,which provides valuable insights into functional neuro-oncological applications.

AB - High-grade glioma is the most aggressive and severe brain tumor that leads to death of almost 50% patients in 1–2 years. Thus,accurate prognosis for glioma patients would provide essential guidelines for their treatment planning. Conventional survival prediction generally utilizes clinical information and limited handcrafted features from magnetic resonance images (MRI),which is often time consuming,laborious and subjective. In this paper,we propose using deep learning frameworks to automatically extract features from multi-modal preoperative brain images (i.e.,T1 MRI,fMRI and DTI) of high-grade glioma patients. Specifically,we adopt 3D convolutional neural networks (CNNs) and also propose a new network architecture for using multi-channel data and learning supervised features. Along with the pivotal clinical features,we finally train a support vector machine to predict if the patient has a long or short overall survival (OS) time. Experimental results demonstrate that our methods can achieve an accuracy as high as 89.9% We also find that the learned features from fMRI and DTI play more important roles in accurately predicting the OS time,which provides valuable insights into functional neuro-oncological applications.

UR - http://www.scopus.com/inward/record.url?scp=84996490301&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84996490301&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46723-8_25

DO - 10.1007/978-3-319-46723-8_25

M3 - Conference contribution

SN - 9783319467221

VL - 9901 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 212

EP - 220

BT - Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings

PB - Springer Verlag

ER -