Automatic brain labeling via multi-atlas guided fully convolutional networks

Longwei Fang, Lichi Zhang, Dong Nie, Xiaohuan Cao, Islem Rekik, Seong Whan Lee, Huiguang He, Dinggang Shen

Research output: Contribution to journalArticle

Abstract

Multi-atlas-based methods are commonly used for MR brain image labeling, which alleviates the burdening and time-consuming task of manual labeling in neuroimaging analysis studies. Traditionally, multi-atlas-based methods first register multiple atlases to the target image, and then propagate the labels from the labeled atlases to the unlabeled target image. However, the registration step involves non-rigid alignment, which is often time-consuming and might lack high accuracy. Alternatively, patch-based methods have shown promise in relaxing the demand for accurate registration, but they often require the use of hand-crafted features. Recently, deep learning techniques have demonstrated their effectiveness in image labeling, by automatically learning comprehensive appearance features from training images. In this paper, we propose a multi-atlas guided fully convolutional network (MA-FCN) for automatic image labeling, which aims at further improving the labeling performance with the aid of prior knowledge from the training atlases. Specifically, we train our MA-FCN model in a patch-based manner, where the input data consists of not only a training image patch but also a set of its neighboring (i.e., most similar) affine-aligned atlas patches. The guidance information from neighboring atlas patches can help boost the discriminative ability of the learned FCN. Experimental results on different datasets demonstrate the effectiveness of our proposed method, by significantly outperforming the conventional FCN and several state-of-the-art MR brain labeling methods.

LanguageEnglish
Pages157-168
Number of pages12
JournalMedical Image Analysis
Volume51
DOIs
Publication statusPublished - 2019 Jan 1

Fingerprint

Atlases
Labeling
Brain
Neuroimaging
Learning
Labels
Aptitude
Hand

Keywords

  • Brain image labeling
  • Fully convolutional network
  • Multi-atlas-based method
  • Patch-based labeling

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Computer Graphics and Computer-Aided Design

Cite this

Automatic brain labeling via multi-atlas guided fully convolutional networks. / Fang, Longwei; Zhang, Lichi; Nie, Dong; Cao, Xiaohuan; Rekik, Islem; Lee, Seong Whan; He, Huiguang; Shen, Dinggang.

In: Medical Image Analysis, Vol. 51, 01.01.2019, p. 157-168.

Research output: Contribution to journalArticle

Fang, Longwei ; Zhang, Lichi ; Nie, Dong ; Cao, Xiaohuan ; Rekik, Islem ; Lee, Seong Whan ; He, Huiguang ; Shen, Dinggang. / Automatic brain labeling via multi-atlas guided fully convolutional networks. In: Medical Image Analysis. 2019 ; Vol. 51. pp. 157-168.
@article{f16fe319a26140a18d7c28219cf96192,
title = "Automatic brain labeling via multi-atlas guided fully convolutional networks",
abstract = "Multi-atlas-based methods are commonly used for MR brain image labeling, which alleviates the burdening and time-consuming task of manual labeling in neuroimaging analysis studies. Traditionally, multi-atlas-based methods first register multiple atlases to the target image, and then propagate the labels from the labeled atlases to the unlabeled target image. However, the registration step involves non-rigid alignment, which is often time-consuming and might lack high accuracy. Alternatively, patch-based methods have shown promise in relaxing the demand for accurate registration, but they often require the use of hand-crafted features. Recently, deep learning techniques have demonstrated their effectiveness in image labeling, by automatically learning comprehensive appearance features from training images. In this paper, we propose a multi-atlas guided fully convolutional network (MA-FCN) for automatic image labeling, which aims at further improving the labeling performance with the aid of prior knowledge from the training atlases. Specifically, we train our MA-FCN model in a patch-based manner, where the input data consists of not only a training image patch but also a set of its neighboring (i.e., most similar) affine-aligned atlas patches. The guidance information from neighboring atlas patches can help boost the discriminative ability of the learned FCN. Experimental results on different datasets demonstrate the effectiveness of our proposed method, by significantly outperforming the conventional FCN and several state-of-the-art MR brain labeling methods.",
keywords = "Brain image labeling, Fully convolutional network, Multi-atlas-based method, Patch-based labeling",
author = "Longwei Fang and Lichi Zhang and Dong Nie and Xiaohuan Cao and Islem Rekik and Lee, {Seong Whan} and Huiguang He and Dinggang Shen",
year = "2019",
month = "1",
day = "1",
doi = "10.1016/j.media.2018.10.012",
language = "English",
volume = "51",
pages = "157--168",
journal = "Medical Image Analysis",
issn = "1361-8415",
publisher = "Elsevier",

}

TY - JOUR

T1 - Automatic brain labeling via multi-atlas guided fully convolutional networks

AU - Fang, Longwei

AU - Zhang, Lichi

AU - Nie, Dong

AU - Cao, Xiaohuan

AU - Rekik, Islem

AU - Lee, Seong Whan

AU - He, Huiguang

AU - Shen, Dinggang

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Multi-atlas-based methods are commonly used for MR brain image labeling, which alleviates the burdening and time-consuming task of manual labeling in neuroimaging analysis studies. Traditionally, multi-atlas-based methods first register multiple atlases to the target image, and then propagate the labels from the labeled atlases to the unlabeled target image. However, the registration step involves non-rigid alignment, which is often time-consuming and might lack high accuracy. Alternatively, patch-based methods have shown promise in relaxing the demand for accurate registration, but they often require the use of hand-crafted features. Recently, deep learning techniques have demonstrated their effectiveness in image labeling, by automatically learning comprehensive appearance features from training images. In this paper, we propose a multi-atlas guided fully convolutional network (MA-FCN) for automatic image labeling, which aims at further improving the labeling performance with the aid of prior knowledge from the training atlases. Specifically, we train our MA-FCN model in a patch-based manner, where the input data consists of not only a training image patch but also a set of its neighboring (i.e., most similar) affine-aligned atlas patches. The guidance information from neighboring atlas patches can help boost the discriminative ability of the learned FCN. Experimental results on different datasets demonstrate the effectiveness of our proposed method, by significantly outperforming the conventional FCN and several state-of-the-art MR brain labeling methods.

AB - Multi-atlas-based methods are commonly used for MR brain image labeling, which alleviates the burdening and time-consuming task of manual labeling in neuroimaging analysis studies. Traditionally, multi-atlas-based methods first register multiple atlases to the target image, and then propagate the labels from the labeled atlases to the unlabeled target image. However, the registration step involves non-rigid alignment, which is often time-consuming and might lack high accuracy. Alternatively, patch-based methods have shown promise in relaxing the demand for accurate registration, but they often require the use of hand-crafted features. Recently, deep learning techniques have demonstrated their effectiveness in image labeling, by automatically learning comprehensive appearance features from training images. In this paper, we propose a multi-atlas guided fully convolutional network (MA-FCN) for automatic image labeling, which aims at further improving the labeling performance with the aid of prior knowledge from the training atlases. Specifically, we train our MA-FCN model in a patch-based manner, where the input data consists of not only a training image patch but also a set of its neighboring (i.e., most similar) affine-aligned atlas patches. The guidance information from neighboring atlas patches can help boost the discriminative ability of the learned FCN. Experimental results on different datasets demonstrate the effectiveness of our proposed method, by significantly outperforming the conventional FCN and several state-of-the-art MR brain labeling methods.

KW - Brain image labeling

KW - Fully convolutional network

KW - Multi-atlas-based method

KW - Patch-based labeling

UR - http://www.scopus.com/inward/record.url?scp=85056564930&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056564930&partnerID=8YFLogxK

U2 - 10.1016/j.media.2018.10.012

DO - 10.1016/j.media.2018.10.012

M3 - Article

VL - 51

SP - 157

EP - 168

JO - Medical Image Analysis

T2 - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

ER -