Detecting Anatomical Landmarks from Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks

Jun Zhang, Mingxia Liu, Dinggang Shen

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

One of the major challenges in anatomical landmark detection, based on deep neural networks, is the limited availability of medical imaging data for network learning. To address this problem, we present a two-stage task-oriented deep learning method to detect large-scale anatomical landmarks simultaneously in real time, using limited training data. Specifically, our method consists of two deep convolutional neural networks (CNN), with each focusing on one specific task. Specifically, to alleviate the problem of limited training data, in the first stage, we propose a CNN based regression model using millions of image patches as input, aiming to learn inherent associations between local image patches and target anatomical landmarks. To further model the correlations among image patches, in the second stage, we develop another CNN model, which includes a) a fully convolutional network that shares the same architecture and network weights as the CNN used in the first stage and also b) several extra layers to jointly predict coordinates of multiple anatomical landmarks. Importantly, our method can jointly detect large-scale (e.g., thousands of) landmarks in real time. We have conducted various experiments for detecting 1200 brain landmarks from the 3D T1-weighted magnetic resonance images of 700 subjects, and also 7 prostate landmarks from the 3D computed tomography images of 73 subjects. The experimental results show the effectiveness of our method regarding both accuracy and efficiency in the anatomical landmark detection.

Original languageEnglish
Article number7961205
Pages (from-to)4753-4764
Number of pages12
JournalIEEE Transactions on Image Processing
Volume26
Issue number10
DOIs
Publication statusPublished - 2017 Oct 1

Fingerprint

Medical imaging
Diagnostic Imaging
Neural networks
Learning
Neural Networks (Computer)
Magnetic resonance
Tomography
Prostate
Brain
Magnetic Resonance Spectroscopy
Availability
Efficiency
Weights and Measures
Deep neural networks
Experiments

Keywords

  • Anatomical landmark detection
  • deep convolutional neural networks
  • limited medical imaging data
  • real-time
  • task-oriented

ASJC Scopus subject areas

  • Software
  • Medicine(all)
  • Computer Graphics and Computer-Aided Design

Cite this

Detecting Anatomical Landmarks from Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks. / Zhang, Jun; Liu, Mingxia; Shen, Dinggang.

In: IEEE Transactions on Image Processing, Vol. 26, No. 10, 7961205, 01.10.2017, p. 4753-4764.

Research output: Contribution to journalArticle

@article{5ae1520229e948a88e79c5befd76fe58,
title = "Detecting Anatomical Landmarks from Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks",
abstract = "One of the major challenges in anatomical landmark detection, based on deep neural networks, is the limited availability of medical imaging data for network learning. To address this problem, we present a two-stage task-oriented deep learning method to detect large-scale anatomical landmarks simultaneously in real time, using limited training data. Specifically, our method consists of two deep convolutional neural networks (CNN), with each focusing on one specific task. Specifically, to alleviate the problem of limited training data, in the first stage, we propose a CNN based regression model using millions of image patches as input, aiming to learn inherent associations between local image patches and target anatomical landmarks. To further model the correlations among image patches, in the second stage, we develop another CNN model, which includes a) a fully convolutional network that shares the same architecture and network weights as the CNN used in the first stage and also b) several extra layers to jointly predict coordinates of multiple anatomical landmarks. Importantly, our method can jointly detect large-scale (e.g., thousands of) landmarks in real time. We have conducted various experiments for detecting 1200 brain landmarks from the 3D T1-weighted magnetic resonance images of 700 subjects, and also 7 prostate landmarks from the 3D computed tomography images of 73 subjects. The experimental results show the effectiveness of our method regarding both accuracy and efficiency in the anatomical landmark detection.",
keywords = "Anatomical landmark detection, deep convolutional neural networks, limited medical imaging data, real-time, task-oriented",
author = "Jun Zhang and Mingxia Liu and Dinggang Shen",
year = "2017",
month = "10",
day = "1",
doi = "10.1109/TIP.2017.2721106",
language = "English",
volume = "26",
pages = "4753--4764",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "10",

}

TY - JOUR

T1 - Detecting Anatomical Landmarks from Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks

AU - Zhang, Jun

AU - Liu, Mingxia

AU - Shen, Dinggang

PY - 2017/10/1

Y1 - 2017/10/1

N2 - One of the major challenges in anatomical landmark detection, based on deep neural networks, is the limited availability of medical imaging data for network learning. To address this problem, we present a two-stage task-oriented deep learning method to detect large-scale anatomical landmarks simultaneously in real time, using limited training data. Specifically, our method consists of two deep convolutional neural networks (CNN), with each focusing on one specific task. Specifically, to alleviate the problem of limited training data, in the first stage, we propose a CNN based regression model using millions of image patches as input, aiming to learn inherent associations between local image patches and target anatomical landmarks. To further model the correlations among image patches, in the second stage, we develop another CNN model, which includes a) a fully convolutional network that shares the same architecture and network weights as the CNN used in the first stage and also b) several extra layers to jointly predict coordinates of multiple anatomical landmarks. Importantly, our method can jointly detect large-scale (e.g., thousands of) landmarks in real time. We have conducted various experiments for detecting 1200 brain landmarks from the 3D T1-weighted magnetic resonance images of 700 subjects, and also 7 prostate landmarks from the 3D computed tomography images of 73 subjects. The experimental results show the effectiveness of our method regarding both accuracy and efficiency in the anatomical landmark detection.

AB - One of the major challenges in anatomical landmark detection, based on deep neural networks, is the limited availability of medical imaging data for network learning. To address this problem, we present a two-stage task-oriented deep learning method to detect large-scale anatomical landmarks simultaneously in real time, using limited training data. Specifically, our method consists of two deep convolutional neural networks (CNN), with each focusing on one specific task. Specifically, to alleviate the problem of limited training data, in the first stage, we propose a CNN based regression model using millions of image patches as input, aiming to learn inherent associations between local image patches and target anatomical landmarks. To further model the correlations among image patches, in the second stage, we develop another CNN model, which includes a) a fully convolutional network that shares the same architecture and network weights as the CNN used in the first stage and also b) several extra layers to jointly predict coordinates of multiple anatomical landmarks. Importantly, our method can jointly detect large-scale (e.g., thousands of) landmarks in real time. We have conducted various experiments for detecting 1200 brain landmarks from the 3D T1-weighted magnetic resonance images of 700 subjects, and also 7 prostate landmarks from the 3D computed tomography images of 73 subjects. The experimental results show the effectiveness of our method regarding both accuracy and efficiency in the anatomical landmark detection.

KW - Anatomical landmark detection

KW - deep convolutional neural networks

KW - limited medical imaging data

KW - real-time

KW - task-oriented

UR - http://www.scopus.com/inward/record.url?scp=85021876379&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021876379&partnerID=8YFLogxK

U2 - 10.1109/TIP.2017.2721106

DO - 10.1109/TIP.2017.2721106

M3 - Article

C2 - 28678706

AN - SCOPUS:85021876379

VL - 26

SP - 4753

EP - 4764

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 10

M1 - 7961205

ER -