STRAINet: Spatially Varying sTochastic Residual AdversarIal Networks for MRI Pelvic Organ Segmentation

Dong Nie, Li Wang, Yaozong Gao, Jun Lian, Dinggang Shen

Research output: Contribution to journalArticle

Abstract

Accurate segmentation of pelvic organs is important for prostate radiation therapy. Modern radiation therapy starts to use a magnetic resonance image (MRI) as an alternative to computed tomography image because of its superior soft tissue contrast and also free of risk from radiation exposure. However, segmentation of pelvic organs from MRI is a challenging problem due to inconsistent organ appearance across patients and also large intrapatient anatomical variations across treatment days. To address such challenges, we propose a novel deep network architecture, called ``Spatially varying sTochastic Residual AdversarIal Network'' (STRAINet), to delineate pelvic organs from MRI in an end-to-end fashion. Compared to the traditional fully convolutional networks (FCN), the proposed architecture has two main contributions: 1) inspired by the recent success of residual learning, we propose an evolutionary version of the residual unit, i.e., stochastic residual unit, and use it to the plain convolutional layers in the FCN. We further propose long-range stochastic residual connections to pass features from shallow layers to deep layers; and 2) we propose to integrate three previously proposed network strategies to form a new network for better medical image segmentation: a) we apply dilated convolution in the smallest resolution feature maps, so that we can gain a larger receptive field without overly losing spatial information; b) we propose a spatially varying convolutional layer that adapts convolutional filters to different regions of interest; and c) an adversarial network is proposed to further correct the segmented organ structures. Finally, STRAINet is used to iteratively refine the segmentation probability maps in an autocontext manner. Experimental results show that our STRAINet achieved the state-of-the-art segmentation accuracy. Further analysis also indicates that our proposed network components contribute most to the performance.

Original languageEnglish
JournalIEEE Transactions on Neural Networks and Learning Systems
DOIs
Publication statusAccepted/In press - 2018 Jan 1
Externally publishedYes

Fingerprint

Magnetic resonance
Magnetic Resonance Spectroscopy
Radiotherapy
Network components
Network architecture
Convolution
Image segmentation
Tomography
Prostate
Learning
Tissue
Radiation
Therapeutics

Keywords

  • Adversarial learning
  • Biological systems
  • Biomedical imaging
  • dilation
  • Feature extraction
  • Image segmentation
  • Magnetic resonance imaging
  • Manuals
  • pelvic organ segmentation
  • stochastic residual learning.
  • Task analysis

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

STRAINet : Spatially Varying sTochastic Residual AdversarIal Networks for MRI Pelvic Organ Segmentation. / Nie, Dong; Wang, Li; Gao, Yaozong; Lian, Jun; Shen, Dinggang.

In: IEEE Transactions on Neural Networks and Learning Systems, 01.01.2018.

Research output: Contribution to journalArticle

@article{7fe82c83313c4978b153d093f13b3e74,
title = "STRAINet: Spatially Varying sTochastic Residual AdversarIal Networks for MRI Pelvic Organ Segmentation",
abstract = "Accurate segmentation of pelvic organs is important for prostate radiation therapy. Modern radiation therapy starts to use a magnetic resonance image (MRI) as an alternative to computed tomography image because of its superior soft tissue contrast and also free of risk from radiation exposure. However, segmentation of pelvic organs from MRI is a challenging problem due to inconsistent organ appearance across patients and also large intrapatient anatomical variations across treatment days. To address such challenges, we propose a novel deep network architecture, called ``Spatially varying sTochastic Residual AdversarIal Network'' (STRAINet), to delineate pelvic organs from MRI in an end-to-end fashion. Compared to the traditional fully convolutional networks (FCN), the proposed architecture has two main contributions: 1) inspired by the recent success of residual learning, we propose an evolutionary version of the residual unit, i.e., stochastic residual unit, and use it to the plain convolutional layers in the FCN. We further propose long-range stochastic residual connections to pass features from shallow layers to deep layers; and 2) we propose to integrate three previously proposed network strategies to form a new network for better medical image segmentation: a) we apply dilated convolution in the smallest resolution feature maps, so that we can gain a larger receptive field without overly losing spatial information; b) we propose a spatially varying convolutional layer that adapts convolutional filters to different regions of interest; and c) an adversarial network is proposed to further correct the segmented organ structures. Finally, STRAINet is used to iteratively refine the segmentation probability maps in an autocontext manner. Experimental results show that our STRAINet achieved the state-of-the-art segmentation accuracy. Further analysis also indicates that our proposed network components contribute most to the performance.",
keywords = "Adversarial learning, Biological systems, Biomedical imaging, dilation, Feature extraction, Image segmentation, Magnetic resonance imaging, Manuals, pelvic organ segmentation, stochastic residual learning., Task analysis",
author = "Dong Nie and Li Wang and Yaozong Gao and Jun Lian and Dinggang Shen",
year = "2018",
month = "1",
day = "1",
doi = "10.1109/TNNLS.2018.2870182",
language = "English",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",

}

TY - JOUR

T1 - STRAINet

T2 - Spatially Varying sTochastic Residual AdversarIal Networks for MRI Pelvic Organ Segmentation

AU - Nie, Dong

AU - Wang, Li

AU - Gao, Yaozong

AU - Lian, Jun

AU - Shen, Dinggang

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Accurate segmentation of pelvic organs is important for prostate radiation therapy. Modern radiation therapy starts to use a magnetic resonance image (MRI) as an alternative to computed tomography image because of its superior soft tissue contrast and also free of risk from radiation exposure. However, segmentation of pelvic organs from MRI is a challenging problem due to inconsistent organ appearance across patients and also large intrapatient anatomical variations across treatment days. To address such challenges, we propose a novel deep network architecture, called ``Spatially varying sTochastic Residual AdversarIal Network'' (STRAINet), to delineate pelvic organs from MRI in an end-to-end fashion. Compared to the traditional fully convolutional networks (FCN), the proposed architecture has two main contributions: 1) inspired by the recent success of residual learning, we propose an evolutionary version of the residual unit, i.e., stochastic residual unit, and use it to the plain convolutional layers in the FCN. We further propose long-range stochastic residual connections to pass features from shallow layers to deep layers; and 2) we propose to integrate three previously proposed network strategies to form a new network for better medical image segmentation: a) we apply dilated convolution in the smallest resolution feature maps, so that we can gain a larger receptive field without overly losing spatial information; b) we propose a spatially varying convolutional layer that adapts convolutional filters to different regions of interest; and c) an adversarial network is proposed to further correct the segmented organ structures. Finally, STRAINet is used to iteratively refine the segmentation probability maps in an autocontext manner. Experimental results show that our STRAINet achieved the state-of-the-art segmentation accuracy. Further analysis also indicates that our proposed network components contribute most to the performance.

AB - Accurate segmentation of pelvic organs is important for prostate radiation therapy. Modern radiation therapy starts to use a magnetic resonance image (MRI) as an alternative to computed tomography image because of its superior soft tissue contrast and also free of risk from radiation exposure. However, segmentation of pelvic organs from MRI is a challenging problem due to inconsistent organ appearance across patients and also large intrapatient anatomical variations across treatment days. To address such challenges, we propose a novel deep network architecture, called ``Spatially varying sTochastic Residual AdversarIal Network'' (STRAINet), to delineate pelvic organs from MRI in an end-to-end fashion. Compared to the traditional fully convolutional networks (FCN), the proposed architecture has two main contributions: 1) inspired by the recent success of residual learning, we propose an evolutionary version of the residual unit, i.e., stochastic residual unit, and use it to the plain convolutional layers in the FCN. We further propose long-range stochastic residual connections to pass features from shallow layers to deep layers; and 2) we propose to integrate three previously proposed network strategies to form a new network for better medical image segmentation: a) we apply dilated convolution in the smallest resolution feature maps, so that we can gain a larger receptive field without overly losing spatial information; b) we propose a spatially varying convolutional layer that adapts convolutional filters to different regions of interest; and c) an adversarial network is proposed to further correct the segmented organ structures. Finally, STRAINet is used to iteratively refine the segmentation probability maps in an autocontext manner. Experimental results show that our STRAINet achieved the state-of-the-art segmentation accuracy. Further analysis also indicates that our proposed network components contribute most to the performance.

KW - Adversarial learning

KW - Biological systems

KW - Biomedical imaging

KW - dilation

KW - Feature extraction

KW - Image segmentation

KW - Magnetic resonance imaging

KW - Manuals

KW - pelvic organ segmentation

KW - stochastic residual learning.

KW - Task analysis

UR - http://www.scopus.com/inward/record.url?scp=85054636735&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054636735&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2018.2870182

DO - 10.1109/TNNLS.2018.2870182

M3 - Article

C2 - 30307879

AN - SCOPUS:85054636735

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

ER -