Depth Map Super-Resolution Using Guided Deformable Convolution

Joon Yeon Kim, Seowon Ji, Seung Jin Baek, Seung Won Jung, Sung Jea Ko

Research output: Contribution to journalArticlepeer-review

Abstract

Depth maps acquired by low-cost sensors have low spatial resolution, which restricts their usefulness in many image processing and computer vision tasks. To increase the spatial resolution of the depth map, most state-of-the-art depth map super-resolution methods based on deep learning extract the features from a high-resolution guidance image and concatenate them with the features from the depth map. However, such simple concatenation can transfer unnecessary textures, known as texture copying artifacts, of the guidance image to the depth map. To address this problem, we propose a novel depth map super-resolution method using guided deformable convolution. Unlike standard deformable convolution, guided deformable convolution obtains 2D kernel offsets of the depth features from the guidance features. Because the guidance features are not explicitly concatenated with the depth features but are used only to determine the kernel offsets for the depth features, the proposed method can significantly alleviate the texture copying artifacts in the resultant depth map. Experimental results show that the proposed method outperforms the state-of-the-art methods in terms of qualitative and quantitative evaluations.

Original languageEnglish
Article number9420066
Pages (from-to)66626-66635
Number of pages10
JournalIEEE Access
Volume9
DOIs
Publication statusPublished - 2021

Keywords

  • Convolutional neural network
  • depth map
  • super-resolution

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint

Dive into the research topics of 'Depth Map Super-Resolution Using Guided Deformable Convolution'. Together they form a unique fingerprint.

Cite this