Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization

Jun Zhang, Mingxia Liu, Li Wang, Si Chen, Peng Yuan, Jianfu Li, Steve Guo Fang Shen, Zhen Tang, Ken Chung Chen, James J. Xia, Dinggang Shen

Research output: Contribution to journalArticle

Abstract

Cone-beam computed tomography (CBCT) scans are commonly used in diagnosing and planning surgical or orthodontic treatment to correct craniomaxillofacial (CMF) deformities. Based on CBCT images, it is clinically essential to generate an accurate 3D model of CMF structures (e.g., midface, and mandible) and digitize anatomical landmarks. This process often involves two tasks, i.e., bone segmentation and anatomical landmark digitization. Because landmarks usually lie on the boundaries of segmented bone regions, the tasks of bone segmentation and landmark digitization could be highly associated. Also, the spatial context information (e.g., displacements from voxels to landmarks) in CBCT images is intuitively important for accurately indicating the spatial association between voxels and landmarks. However, most of the existing studies simply treat bone segmentation and landmark digitization as two standalone tasks without considering their inherent relationship, and rarely take advantage of the spatial context information contained in CBCT images. To address these issues, we propose a Joint bone Segmentation and landmark Digitization (JSD) framework via context-guided fully convolutional networks (FCNs). Specifically, we first utilize displacement maps to model the spatial context information in CBCT images, where each element in the displacement map denotes the displacement from a voxel to a particular landmark. An FCN is learned to construct the mapping from the input image to its corresponding displacement maps. Using the learned displacement maps as guidance, we further develop a multi-task FCN model to perform bone segmentation and landmark digitization jointly. We validate the proposed JSD method on 107 subjects, and the experimental results demonstrate that our method is superior to the state-of-the-art approaches in both tasks of bone segmentation and landmark digitization.

Original languageEnglish
Article number101621
JournalMedical Image Analysis
Volume60
DOIs
Publication statusPublished - 2020 Feb

Fingerprint

Analog to digital conversion
Cone-Beam Computed Tomography
Bone
Joints
Bone and Bones
Tomography
Cones
Orthodontics
Mandible
Planning

Keywords

  • Bone segmentation
  • Cone-beam computed tomography
  • Fully convolutional networks
  • Landmark digitization

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Computer Graphics and Computer-Aided Design

Cite this

Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization. / Zhang, Jun; Liu, Mingxia; Wang, Li; Chen, Si; Yuan, Peng; Li, Jianfu; Shen, Steve Guo Fang; Tang, Zhen; Chen, Ken Chung; Xia, James J.; Shen, Dinggang.

In: Medical Image Analysis, Vol. 60, 101621, 02.2020.

Research output: Contribution to journalArticle

Zhang, Jun ; Liu, Mingxia ; Wang, Li ; Chen, Si ; Yuan, Peng ; Li, Jianfu ; Shen, Steve Guo Fang ; Tang, Zhen ; Chen, Ken Chung ; Xia, James J. ; Shen, Dinggang. / Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization. In: Medical Image Analysis. 2020 ; Vol. 60.
@article{bddf8210e2a54f499c1da900eaf763cd,
title = "Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization",
abstract = "Cone-beam computed tomography (CBCT) scans are commonly used in diagnosing and planning surgical or orthodontic treatment to correct craniomaxillofacial (CMF) deformities. Based on CBCT images, it is clinically essential to generate an accurate 3D model of CMF structures (e.g., midface, and mandible) and digitize anatomical landmarks. This process often involves two tasks, i.e., bone segmentation and anatomical landmark digitization. Because landmarks usually lie on the boundaries of segmented bone regions, the tasks of bone segmentation and landmark digitization could be highly associated. Also, the spatial context information (e.g., displacements from voxels to landmarks) in CBCT images is intuitively important for accurately indicating the spatial association between voxels and landmarks. However, most of the existing studies simply treat bone segmentation and landmark digitization as two standalone tasks without considering their inherent relationship, and rarely take advantage of the spatial context information contained in CBCT images. To address these issues, we propose a Joint bone Segmentation and landmark Digitization (JSD) framework via context-guided fully convolutional networks (FCNs). Specifically, we first utilize displacement maps to model the spatial context information in CBCT images, where each element in the displacement map denotes the displacement from a voxel to a particular landmark. An FCN is learned to construct the mapping from the input image to its corresponding displacement maps. Using the learned displacement maps as guidance, we further develop a multi-task FCN model to perform bone segmentation and landmark digitization jointly. We validate the proposed JSD method on 107 subjects, and the experimental results demonstrate that our method is superior to the state-of-the-art approaches in both tasks of bone segmentation and landmark digitization.",
keywords = "Bone segmentation, Cone-beam computed tomography, Fully convolutional networks, Landmark digitization",
author = "Jun Zhang and Mingxia Liu and Li Wang and Si Chen and Peng Yuan and Jianfu Li and Shen, {Steve Guo Fang} and Zhen Tang and Chen, {Ken Chung} and Xia, {James J.} and Dinggang Shen",
year = "2020",
month = "2",
doi = "10.1016/j.media.2019.101621",
language = "English",
volume = "60",
journal = "Medical Image Analysis",
issn = "1361-8415",
publisher = "Elsevier",

}

TY - JOUR

T1 - Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization

AU - Zhang, Jun

AU - Liu, Mingxia

AU - Wang, Li

AU - Chen, Si

AU - Yuan, Peng

AU - Li, Jianfu

AU - Shen, Steve Guo Fang

AU - Tang, Zhen

AU - Chen, Ken Chung

AU - Xia, James J.

AU - Shen, Dinggang

PY - 2020/2

Y1 - 2020/2

N2 - Cone-beam computed tomography (CBCT) scans are commonly used in diagnosing and planning surgical or orthodontic treatment to correct craniomaxillofacial (CMF) deformities. Based on CBCT images, it is clinically essential to generate an accurate 3D model of CMF structures (e.g., midface, and mandible) and digitize anatomical landmarks. This process often involves two tasks, i.e., bone segmentation and anatomical landmark digitization. Because landmarks usually lie on the boundaries of segmented bone regions, the tasks of bone segmentation and landmark digitization could be highly associated. Also, the spatial context information (e.g., displacements from voxels to landmarks) in CBCT images is intuitively important for accurately indicating the spatial association between voxels and landmarks. However, most of the existing studies simply treat bone segmentation and landmark digitization as two standalone tasks without considering their inherent relationship, and rarely take advantage of the spatial context information contained in CBCT images. To address these issues, we propose a Joint bone Segmentation and landmark Digitization (JSD) framework via context-guided fully convolutional networks (FCNs). Specifically, we first utilize displacement maps to model the spatial context information in CBCT images, where each element in the displacement map denotes the displacement from a voxel to a particular landmark. An FCN is learned to construct the mapping from the input image to its corresponding displacement maps. Using the learned displacement maps as guidance, we further develop a multi-task FCN model to perform bone segmentation and landmark digitization jointly. We validate the proposed JSD method on 107 subjects, and the experimental results demonstrate that our method is superior to the state-of-the-art approaches in both tasks of bone segmentation and landmark digitization.

AB - Cone-beam computed tomography (CBCT) scans are commonly used in diagnosing and planning surgical or orthodontic treatment to correct craniomaxillofacial (CMF) deformities. Based on CBCT images, it is clinically essential to generate an accurate 3D model of CMF structures (e.g., midface, and mandible) and digitize anatomical landmarks. This process often involves two tasks, i.e., bone segmentation and anatomical landmark digitization. Because landmarks usually lie on the boundaries of segmented bone regions, the tasks of bone segmentation and landmark digitization could be highly associated. Also, the spatial context information (e.g., displacements from voxels to landmarks) in CBCT images is intuitively important for accurately indicating the spatial association between voxels and landmarks. However, most of the existing studies simply treat bone segmentation and landmark digitization as two standalone tasks without considering their inherent relationship, and rarely take advantage of the spatial context information contained in CBCT images. To address these issues, we propose a Joint bone Segmentation and landmark Digitization (JSD) framework via context-guided fully convolutional networks (FCNs). Specifically, we first utilize displacement maps to model the spatial context information in CBCT images, where each element in the displacement map denotes the displacement from a voxel to a particular landmark. An FCN is learned to construct the mapping from the input image to its corresponding displacement maps. Using the learned displacement maps as guidance, we further develop a multi-task FCN model to perform bone segmentation and landmark digitization jointly. We validate the proposed JSD method on 107 subjects, and the experimental results demonstrate that our method is superior to the state-of-the-art approaches in both tasks of bone segmentation and landmark digitization.

KW - Bone segmentation

KW - Cone-beam computed tomography

KW - Fully convolutional networks

KW - Landmark digitization

UR - http://www.scopus.com/inward/record.url?scp=85075993052&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075993052&partnerID=8YFLogxK

U2 - 10.1016/j.media.2019.101621

DO - 10.1016/j.media.2019.101621

M3 - Article

C2 - 31816592

AN - SCOPUS:85075993052

VL - 60

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

M1 - 101621

ER -