Locally-constrained boundary regression for segmentation of prostate and rectum in the planning CT images

Yeqin Shao, Yaozong Gao, Qian Wang, Xin Yang, Dinggang Shen

Research output: Contribution to journalArticlepeer-review

32 Citations (Scopus)


Automatic and accurate segmentation of the prostate and rectum in planning CT images is a challenging task due to low image contrast, unpredictable organ (relative) position, and uncertain existence of bowel gas across different patients. Recently, regression forest was adopted for organ deformable segmentation on 2D medical images by training one landmark detector for each point on the shape model. However, it seems impractical for regression forest to guide 3D deformable segmentation as a landmark detector, due to large number of vertices in the 3D shape model as well as the difficulty in building accurate 3D vertex correspondence for each landmark detector. In this paper, we propose a novel boundary detection method by exploiting the power of regression forest for prostate and rectum segmentation. The contributions of this paper are as follows: (1) we introduce regression forest as a local boundary regressor to vote the entire boundary of a target organ, which avoids training a large number of landmark detectors and building an accurate 3D vertex correspondence for each landmark detector; (2) an auto-context model is integrated with regression forest to improve the accuracy of the boundary regression; (3) we further combine a deformable segmentation method with the proposed local boundary regressor for the final organ segmentation by integrating organ shape priors. Our method is evaluated on a planning CT image dataset with 70 images from 70 different patients. The experimental results show that our proposed boundary regression method outperforms the conventional boundary classification method in guiding the deformable model for prostate and rectum segmentations. Compared with other state-of-the-art methods, our method also shows a competitive performance.

Original languageEnglish
Pages (from-to)345-356
Number of pages12
JournalMedical Image Analysis
Issue number1
Publication statusPublished - 2015 Dec 1


  • Deformable segmentation
  • Local boundary regression
  • Prostate segmentation
  • Rectum segmentation
  • Regression forest

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Locally-constrained boundary regression for segmentation of prostate and rectum in the planning CT images'. Together they form a unique fingerprint.

Cite this