Multi-Scale Warping for Video Frame Interpolation

Whan Choi, Yeong Jun Koh, Chang Su Kim

Research output: Contribution to journalArticlepeer-review

Abstract

A novel video interpolation network to improve the temporal resolutions of video sequences is proposed in this work. We develop a multi-scale warping module to interpolate intermediate frames robustly for both small and large motions. Specifically, the proposed multi-scale warping module deals with large motions between two consecutive frames using coarse-scale features, while estimating detailed local motions by exploring fine-scale features. To this end, it takes multi-scale features from the encoder and estimates kernel weights and offset vectors for each scale. Finally, it synthesizes multi-scale warping frames and combines them to obtain an intermediate frame. Extensive experimental results demonstrate that the proposed algorithm outperforms state-of-the-art video interpolation algorithms on various benchmark datasets.

Original languageEnglish
Pages (from-to)150470-150479
Number of pages10
JournalIEEE Access
Volume9
DOIs
Publication statusPublished - 2021

Keywords

  • adaptive convolution
  • convolutional neural network
  • deformable convolution
  • kernel-based approach
  • multi-scale feature
  • Video frame interpolation

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint

Dive into the research topics of 'Multi-Scale Warping for Video Frame Interpolation'. Together they form a unique fingerprint.

Cite this