Adversarial learning for mono- or multi-modal registration

Jingfan Fan, Xiaohuan Cao, Qian Wang, Pew Thian Yap, Dinggang Shen

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration methods, our approach can train a deformable registration network without the need of ground-truth deformations and specific similarity metrics. We connect a registration network and a discrimination network with a deformable transformation layer. The registration network is trained with the feedback from the discrimination network, which is designed to judge whether a pair of registered images are sufficiently similar. Using adversarial training, the registration network is trained to predict deformations that are accurate enough to fool the discrimination network. The proposed method is thus a general registration framework, which can be applied for both mono-modal and multi-modal image registration. Experiments on four brain MRI datasets and a multi-modal pelvic image dataset indicate that our method yields promising registration performance in accuracy, efficiency and generalizability compared with state-of-the-art registration methods, including those based on deep learning.

Original languageEnglish
Article number101545
JournalMedical Image Analysis
Volume58
DOIs
Publication statusPublished - 2019 Dec

Fingerprint

Image registration
Learning
Magnetic resonance imaging
Brain
Feedback
Experiments
Efficiency
Deep learning
Discrimination (Psychology)
Datasets

Keywords

  • Deformable image registration
  • Fully convolutional neural network
  • Generative adversarial network

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Computer Graphics and Computer-Aided Design

Cite this

Adversarial learning for mono- or multi-modal registration. / Fan, Jingfan; Cao, Xiaohuan; Wang, Qian; Yap, Pew Thian; Shen, Dinggang.

In: Medical Image Analysis, Vol. 58, 101545, 12.2019.

Research output: Contribution to journalArticle

Fan, Jingfan ; Cao, Xiaohuan ; Wang, Qian ; Yap, Pew Thian ; Shen, Dinggang. / Adversarial learning for mono- or multi-modal registration. In: Medical Image Analysis. 2019 ; Vol. 58.
@article{925c3771579d40ac9e4e0b95733fdb7d,
title = "Adversarial learning for mono- or multi-modal registration",
abstract = "This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration methods, our approach can train a deformable registration network without the need of ground-truth deformations and specific similarity metrics. We connect a registration network and a discrimination network with a deformable transformation layer. The registration network is trained with the feedback from the discrimination network, which is designed to judge whether a pair of registered images are sufficiently similar. Using adversarial training, the registration network is trained to predict deformations that are accurate enough to fool the discrimination network. The proposed method is thus a general registration framework, which can be applied for both mono-modal and multi-modal image registration. Experiments on four brain MRI datasets and a multi-modal pelvic image dataset indicate that our method yields promising registration performance in accuracy, efficiency and generalizability compared with state-of-the-art registration methods, including those based on deep learning.",
keywords = "Deformable image registration, Fully convolutional neural network, Generative adversarial network",
author = "Jingfan Fan and Xiaohuan Cao and Qian Wang and Yap, {Pew Thian} and Dinggang Shen",
year = "2019",
month = "12",
doi = "10.1016/j.media.2019.101545",
language = "English",
volume = "58",
journal = "Medical Image Analysis",
issn = "1361-8415",
publisher = "Elsevier",

}

TY - JOUR

T1 - Adversarial learning for mono- or multi-modal registration

AU - Fan, Jingfan

AU - Cao, Xiaohuan

AU - Wang, Qian

AU - Yap, Pew Thian

AU - Shen, Dinggang

PY - 2019/12

Y1 - 2019/12

N2 - This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration methods, our approach can train a deformable registration network without the need of ground-truth deformations and specific similarity metrics. We connect a registration network and a discrimination network with a deformable transformation layer. The registration network is trained with the feedback from the discrimination network, which is designed to judge whether a pair of registered images are sufficiently similar. Using adversarial training, the registration network is trained to predict deformations that are accurate enough to fool the discrimination network. The proposed method is thus a general registration framework, which can be applied for both mono-modal and multi-modal image registration. Experiments on four brain MRI datasets and a multi-modal pelvic image dataset indicate that our method yields promising registration performance in accuracy, efficiency and generalizability compared with state-of-the-art registration methods, including those based on deep learning.

AB - This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration methods, our approach can train a deformable registration network without the need of ground-truth deformations and specific similarity metrics. We connect a registration network and a discrimination network with a deformable transformation layer. The registration network is trained with the feedback from the discrimination network, which is designed to judge whether a pair of registered images are sufficiently similar. Using adversarial training, the registration network is trained to predict deformations that are accurate enough to fool the discrimination network. The proposed method is thus a general registration framework, which can be applied for both mono-modal and multi-modal image registration. Experiments on four brain MRI datasets and a multi-modal pelvic image dataset indicate that our method yields promising registration performance in accuracy, efficiency and generalizability compared with state-of-the-art registration methods, including those based on deep learning.

KW - Deformable image registration

KW - Fully convolutional neural network

KW - Generative adversarial network

UR - http://www.scopus.com/inward/record.url?scp=85072580106&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072580106&partnerID=8YFLogxK

U2 - 10.1016/j.media.2019.101545

DO - 10.1016/j.media.2019.101545

M3 - Article

C2 - 31557633

AN - SCOPUS:85072580106

VL - 58

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

M1 - 101545

ER -