Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker

Ruikai Zhang, Yali Zheng, Carmen C.Y. Poon, Dinggang Shen, James Y.W. Lau

Research output: Contribution to journalArticlepeer-review

99 Citations (Scopus)


A computer-aided detection (CAD) tool for locating and detecting polyps can help reduce the chance of missing polyps during colonoscopy. Nevertheless, state-of-the-art algorithms were either computationally complex or suffered from low sensitivity and therefore unsuitable to be used in real clinical setting. In this paper, a novel regression-based Convolutional Neural Network (CNN) pipeline is presented for polyp detection during colonoscopy. The proposed pipeline was constructed in two parts: 1) to learn the spatial features of colorectal polyps, a fast object detection algorithm named ResYOLO was pre-trained with a large non-medical image database and further fine-tuned with colonoscopic images extracted from videos; and 2) temporal information was incorporated via a tracker named Efficient Convolution Operators (ECO) for refining the detection results given by ResYOLO. Evaluated on 17,574 frames extracted from 18 endoscopic videos of the AsuMayoDB, the proposed method was able to detect frames with polyps with a precision of 88.6%, recall of 71.6% and processing speed of 6.5 frames per second, i.e. the method can accurately locate polyps in more frames and at a faster speed compared to existing methods. In conclusion, the proposed method has great potential to be used to assist endoscopists in tracking polyps during colonoscopy.

Original languageEnglish
Pages (from-to)209-219
Number of pages11
JournalPattern Recognition
Publication statusPublished - 2018 Nov


  • Body Sensor Network
  • Deep Learning
  • Endoscopic Informatics
  • Health Informatics
  • Smart cancer screening
  • Therapeutic endoscopy

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker'. Together they form a unique fingerprint.

Cite this