Deep Neural Network Regression for Automated Retinal Layer Segmentation in Optical Coherence Tomography Images

Lua Ngo, Jaepyeong Cha, Jae Ho Han

Research output: Contribution to journalArticlepeer-review

28 Citations (Scopus)


Segmenting the retinal layers in optical coherence tomography (OCT) images helps to quantify the layer information in early diagnosis of retinal diseases, which are the main cause of permanent blindness. Thus, the segmentation process plays a critical role in preventing vision impairment. However, because there is a lack of practical automated techniques, expert ophthalmologists still have to manually segment the retinal layers. In this paper, we propose an automated segmentation method for OCT images based on a feature-learning regression network without human bias. The proposed deep neural network regression takes the intensity, gradient, and adaptive normalized intensity score (ANIS) of an image segment as features for learning, and then predicts the corresponding retinal boundary pixel. Reformulating the segmentation as a regression problem obviates the need for a huge dataset and reduces the complexity significantly, as shown in the analysis of computational complexity given here. In addition, assisted by ANIS, the method operates robustly on OCT images containing intensity variances, low-contrast regions, speckle noise, and blood vessels, yet remains accurate and time-efficient. In the evaluation of the method conducted using 114 images, the processing time was approximately 10.596 s per image for identifying eight boundaries, and the training phase for each boundary line took only 30 s. Further, the Dice similarity coefficient used for assessing accuracy gave a computed value of approximately 0.966. The absolute pixel distance of manual and automatic segmentation using the proposed scheme was 0.612, which is less than a one-pixel difference, on average.

Original languageEnglish
Article number8784409
Pages (from-to)303-312
Number of pages10
JournalIEEE Transactions on Image Processing
Publication statusPublished - 2020


  • Artificial intelligence
  • biomedical optical imaging
  • image segmentation
  • neural network
  • optical coherence tomography

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Deep Neural Network Regression for Automated Retinal Layer Segmentation in Optical Coherence Tomography Images'. Together they form a unique fingerprint.

Cite this