TY - JOUR
T1 - Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images
AU - He, Kelei
AU - Zhao, Wei
AU - Xie, Xingzhi
AU - Ji, Wen
AU - Liu, Mingxia
AU - Tang, Zhenyu
AU - Shi, Yinghuan
AU - Shi, Feng
AU - Gao, Yang
AU - Liu, Jun
AU - Zhang, Junfeng
AU - Shen, Dinggang
N1 - Funding Information:
This work is supported in part by Key Emergency Project of Pneumonia Epidemic of novel coronavirus infection under grant 2020sk3006, Emergency Project of Prevention and Control for COVID-19 of Central South University under grant 160260005, Foundation of Changsha Scientific and Technical Bureau under grant kq2001001, and National Key Research and Development Program of China under grant 2018YFC0116400.
PY - 2021/5
Y1 - 2021/5
N2 - Understanding chest CT imaging of the coronavirus disease 2019 (COVID-19) will help detect infections early and assess the disease progression. Especially, automated severity assessment of COVID-19 in CT images plays an essential role in identifying cases that are in great need of intensive clinical care. However, it is often challenging to accurately assess the severity of this disease in CT images, due to variable infection regions in the lungs, similar imaging biomarkers, and large inter-case variations. To this end, we propose a synergistic learning framework for automated severity assessment of COVID-19 in 3D CT images, by jointly performing lung lobe segmentation and multi-instance classification. Considering that only a few infection regions in a CT image are related to the severity assessment, we first represent each input image by a bag that contains a set of 2D image patches (with each cropped from a specific slice). A multi-task multi-instance deep network (called M2UNet) is then developed to assess the severity of COVID-19 patients and also segment the lung lobe simultaneously. Our M2UNet consists of a patch-level encoder, a segmentation sub-network for lung lobe segmentation, and a classification sub-network for severity assessment (with a unique hierarchical multi-instance learning strategy). Here, the context information provided by segmentation can be implicitly employed to improve the performance of severity assessment. Extensive experiments were performed on a real COVID-19 CT image dataset consisting of 666 chest CT images, with results suggesting the effectiveness of our proposed method compared to several state-of-the-art methods.
AB - Understanding chest CT imaging of the coronavirus disease 2019 (COVID-19) will help detect infections early and assess the disease progression. Especially, automated severity assessment of COVID-19 in CT images plays an essential role in identifying cases that are in great need of intensive clinical care. However, it is often challenging to accurately assess the severity of this disease in CT images, due to variable infection regions in the lungs, similar imaging biomarkers, and large inter-case variations. To this end, we propose a synergistic learning framework for automated severity assessment of COVID-19 in 3D CT images, by jointly performing lung lobe segmentation and multi-instance classification. Considering that only a few infection regions in a CT image are related to the severity assessment, we first represent each input image by a bag that contains a set of 2D image patches (with each cropped from a specific slice). A multi-task multi-instance deep network (called M2UNet) is then developed to assess the severity of COVID-19 patients and also segment the lung lobe simultaneously. Our M2UNet consists of a patch-level encoder, a segmentation sub-network for lung lobe segmentation, and a classification sub-network for severity assessment (with a unique hierarchical multi-instance learning strategy). Here, the context information provided by segmentation can be implicitly employed to improve the performance of severity assessment. Extensive experiments were performed on a real COVID-19 CT image dataset consisting of 666 chest CT images, with results suggesting the effectiveness of our proposed method compared to several state-of-the-art methods.
KW - COVID-19
KW - CT
KW - Lung lobe segmentation
KW - Multi-instance learning
KW - Severity assessment
UR - http://www.scopus.com/inward/record.url?scp=85099498502&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85099498502&partnerID=8YFLogxK
U2 - 10.1016/j.patcog.2021.107828
DO - 10.1016/j.patcog.2021.107828
M3 - Article
AN - SCOPUS:85099498502
VL - 113
JO - Pattern Recognition
JF - Pattern Recognition
SN - 0031-3203
M1 - 107828
ER -