TY - JOUR
T1 - Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation
AU - Ren, Xuhua
AU - Ahmad, Sahar
AU - Zhang, Lichi
AU - Xiang, Lei
AU - Nie, Dong
AU - Yang, Fan
AU - Wang, Qian
AU - Shen, DInggang
N1 - Funding Information:
Manuscript received June 14, 2019; revised March 21, 2020 and May 17, 2020; accepted June 15, 2020. Date of publication June 25, 2020; date of current version July 13, 2020. The work of Qian Wang was supported in part by the National Key Research and Development Program of China under Grant 2018YFC0116400, in part by the Science and Technology Commission of Shanghai Municipality (STCSM) under Grant 19QC1400600, Grant 17411953300, and Grant 18JC1420305, in part by the Shanghai Pujiang Program under Grant 19PJ1406800, and in part by the Interdisciplinary Program of Shanghai Jiao Tong University. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Denis Kouame. (Corresponding authors: Qian Wang; Dinggang Shen.) Xuhua Ren, Lichi Zhang, and Qian Wang are with the School of Biomedical Engineering, Institute for Medical Imaging Technology, Shanghai Jiao Tong University, Shanghai 200030, China (e-mail: wang.qian@sjtu.edu.cn).
Publisher Copyright:
© 1992-2012 IEEE.
PY - 2020
Y1 - 2020
N2 - Semantic segmentation is essentially important to biomedical image analysis. Many recent works mainly focus on integrating the Fully Convolutional Network (FCN) architecture with sophisticated convolution implementation and deep supervision. Such complex networks need large training datasets, a requirement which is challenging for medical image analysis. In this paper, we propose to decompose the single segmentation task into three subsequent sub-tasks, including (1) pixel-wise image semantic segmentation, (2) prediction of the instance class labels of the objects within the image, and (3) classification of the scene the image belonging to. While these three sub-tasks are trained to optimize their individual loss functions at different perceptual levels, we propose to allow their interaction within the task-task context ensemble. Moreover, we propose a novel sync-regularization to penalize the deviation between the outputs of the pixel-wise semantic segmentation and the instance class prediction tasks. These effective regularizations help FCN utilize context information comprehensively and attain accurate segmentation, even though the number of images for training may be limited in many biomedical applications. We have successfully applied our framework to three diverse 2D/3D medical image datasets, including Robotic Scene Segmentation Challenge 18 (ROBOT18), Brain Tumor Segmentation Challenge 18 (BRATS18), and Retinal Fundus Glaucoma Challenge (REFUGE18). We have achieved outperformed or comparable performance in all the three challenges. Our code, typical data and trained models are available at https://github.com/xuhuaren/TDSNet.
AB - Semantic segmentation is essentially important to biomedical image analysis. Many recent works mainly focus on integrating the Fully Convolutional Network (FCN) architecture with sophisticated convolution implementation and deep supervision. Such complex networks need large training datasets, a requirement which is challenging for medical image analysis. In this paper, we propose to decompose the single segmentation task into three subsequent sub-tasks, including (1) pixel-wise image semantic segmentation, (2) prediction of the instance class labels of the objects within the image, and (3) classification of the scene the image belonging to. While these three sub-tasks are trained to optimize their individual loss functions at different perceptual levels, we propose to allow their interaction within the task-task context ensemble. Moreover, we propose a novel sync-regularization to penalize the deviation between the outputs of the pixel-wise semantic segmentation and the instance class prediction tasks. These effective regularizations help FCN utilize context information comprehensively and attain accurate segmentation, even though the number of images for training may be limited in many biomedical applications. We have successfully applied our framework to three diverse 2D/3D medical image datasets, including Robotic Scene Segmentation Challenge 18 (ROBOT18), Brain Tumor Segmentation Challenge 18 (BRATS18), and Retinal Fundus Glaucoma Challenge (REFUGE18). We have achieved outperformed or comparable performance in all the three challenges. Our code, typical data and trained models are available at https://github.com/xuhuaren/TDSNet.
KW - Semantic segmentation
KW - deep learning
KW - fully convolutional network
KW - sync-regularization
KW - task decomposition
UR - http://www.scopus.com/inward/record.url?scp=85088301397&partnerID=8YFLogxK
U2 - 10.1109/TIP.2020.3003735
DO - 10.1109/TIP.2020.3003735
M3 - Article
AN - SCOPUS:85088301397
VL - 29
SP - 7497
EP - 7510
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
SN - 1057-7149
M1 - 9126262
ER -