Video inpainting algorithm using Spatio-temporal consistency

Sang Heon Lee, Soon Young Lee, Jun Hee Heu, Chang-Su Kim, Sang Uk Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A new video inpainting algorithm is proposed for removing unwanted or erroneous objects from video data. The proposed algorithm fills a mask region with source blocks from unmasked areas, while keeping spatio-temporal consistency. First, a 3-dimensional graph is constructed over consecutive frames. It defines a structure of nodes over which the source blocks are pasted. Then, we form temporal block bundles using the motion information. The best block bundles, which minimize an objective function, are arranged in the 3-dimensional graph. Extensive simulation results demonstrate that the proposed algorithm can yield visually pleasing video inpainting results even for dynamic sequences.

Original languageEnglish
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
Volume7246
DOIs
Publication statusPublished - 2009
EventComputational Imaging VII - San Jose, CA, United States
Duration: 2009 Jan 192009 Jan 20

Other

OtherComputational Imaging VII
CountryUnited States
CitySan Jose, CA
Period09/1/1909/1/20

Fingerprint

Inpainting
bundles
Bundle
video data
Graph in graph theory
Mask
Consecutive
Masks
masks
Objective function
Minimise
Motion
Vertex of a graph
Demonstrate
Simulation
simulation

Keywords

  • Inpainting
  • Optimization
  • Restoration
  • Spatio-temporal inpainting
  • Video editing

ASJC Scopus subject areas

  • Applied Mathematics
  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics

Cite this

Lee, S. H., Lee, S. Y., Heu, J. H., Kim, C-S., & Lee, S. U. (2009). Video inpainting algorithm using Spatio-temporal consistency. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 7246). [72460N] https://doi.org/10.1117/12.805892

Video inpainting algorithm using Spatio-temporal consistency. / Lee, Sang Heon; Lee, Soon Young; Heu, Jun Hee; Kim, Chang-Su; Lee, Sang Uk.

Proceedings of SPIE - The International Society for Optical Engineering. Vol. 7246 2009. 72460N.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Lee, SH, Lee, SY, Heu, JH, Kim, C-S & Lee, SU 2009, Video inpainting algorithm using Spatio-temporal consistency. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 7246, 72460N, Computational Imaging VII, San Jose, CA, United States, 09/1/19. https://doi.org/10.1117/12.805892
Lee SH, Lee SY, Heu JH, Kim C-S, Lee SU. Video inpainting algorithm using Spatio-temporal consistency. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 7246. 2009. 72460N https://doi.org/10.1117/12.805892
Lee, Sang Heon ; Lee, Soon Young ; Heu, Jun Hee ; Kim, Chang-Su ; Lee, Sang Uk. / Video inpainting algorithm using Spatio-temporal consistency. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 7246 2009.
@inproceedings{b98434fdfb6249ff9f2fdbef38737e12,
title = "Video inpainting algorithm using Spatio-temporal consistency",
abstract = "A new video inpainting algorithm is proposed for removing unwanted or erroneous objects from video data. The proposed algorithm fills a mask region with source blocks from unmasked areas, while keeping spatio-temporal consistency. First, a 3-dimensional graph is constructed over consecutive frames. It defines a structure of nodes over which the source blocks are pasted. Then, we form temporal block bundles using the motion information. The best block bundles, which minimize an objective function, are arranged in the 3-dimensional graph. Extensive simulation results demonstrate that the proposed algorithm can yield visually pleasing video inpainting results even for dynamic sequences.",
keywords = "Inpainting, Optimization, Restoration, Spatio-temporal inpainting, Video editing",
author = "Lee, {Sang Heon} and Lee, {Soon Young} and Heu, {Jun Hee} and Chang-Su Kim and Lee, {Sang Uk}",
year = "2009",
doi = "10.1117/12.805892",
language = "English",
volume = "7246",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

}

TY - GEN

T1 - Video inpainting algorithm using Spatio-temporal consistency

AU - Lee, Sang Heon

AU - Lee, Soon Young

AU - Heu, Jun Hee

AU - Kim, Chang-Su

AU - Lee, Sang Uk

PY - 2009

Y1 - 2009

N2 - A new video inpainting algorithm is proposed for removing unwanted or erroneous objects from video data. The proposed algorithm fills a mask region with source blocks from unmasked areas, while keeping spatio-temporal consistency. First, a 3-dimensional graph is constructed over consecutive frames. It defines a structure of nodes over which the source blocks are pasted. Then, we form temporal block bundles using the motion information. The best block bundles, which minimize an objective function, are arranged in the 3-dimensional graph. Extensive simulation results demonstrate that the proposed algorithm can yield visually pleasing video inpainting results even for dynamic sequences.

AB - A new video inpainting algorithm is proposed for removing unwanted or erroneous objects from video data. The proposed algorithm fills a mask region with source blocks from unmasked areas, while keeping spatio-temporal consistency. First, a 3-dimensional graph is constructed over consecutive frames. It defines a structure of nodes over which the source blocks are pasted. Then, we form temporal block bundles using the motion information. The best block bundles, which minimize an objective function, are arranged in the 3-dimensional graph. Extensive simulation results demonstrate that the proposed algorithm can yield visually pleasing video inpainting results even for dynamic sequences.

KW - Inpainting

KW - Optimization

KW - Restoration

KW - Spatio-temporal inpainting

KW - Video editing

UR - http://www.scopus.com/inward/record.url?scp=65649125657&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=65649125657&partnerID=8YFLogxK

U2 - 10.1117/12.805892

DO - 10.1117/12.805892

M3 - Conference contribution

AN - SCOPUS:65649125657

VL - 7246

BT - Proceedings of SPIE - The International Society for Optical Engineering

ER -