Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping

Yeong Jun Koh, Chulwoo Lee, Chang-Su Kim

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

We propose a video stabilization algorithm, which extracts a guaranteed number of reliable feature trajectories for robust mesh grid warping. We first estimate feature trajectories through a video sequence and transform the feature positions into rolling-free smoothed positions. When the number of the estimated trajectories is insufficient, we generate virtual trajectories by augmenting incomplete trajectories using a low-rank matrix completion scheme. Next, we detect feature points on a large moving object and exclude them so as to stabilize camera movements, rather than object movements. With the selected feature points, we set a mesh grid on each frame and warp each grid cell by moving the original feature positions to the smoothed ones. For robust warping, we formulate a cost function based on the reliability weights of each feature point and each grid cell. The cost function consists of a data term, a structure-preserving term, and a regularization term. By minimizing the cost function, we determine the robust mesh grid warping and achieve the stabilization. Experimental results demonstrate that the proposed algorithm reconstructs videos more stably than the conventional algorithms.

Original languageEnglish
Article number7271056
Pages (from-to)5260-5273
Number of pages14
JournalIEEE Transactions on Image Processing
Volume24
Issue number12
DOIs
Publication statusPublished - 2015 Dec 1

Fingerprint

Stabilization
Trajectories
Costs and Cost Analysis
Cost functions
Weights and Measures
Cameras
Grid Cells

Keywords

  • and mesh grid warping
  • low-rank matrix completion
  • reliable feature selection
  • rolling shutter distortion
  • Video stabilization

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping. / Koh, Yeong Jun; Lee, Chulwoo; Kim, Chang-Su.

In: IEEE Transactions on Image Processing, Vol. 24, No. 12, 7271056, 01.12.2015, p. 5260-5273.

Research output: Contribution to journalArticle

@article{e95b5139d6f74cd695ab811154b96f77,
title = "Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping",
abstract = "We propose a video stabilization algorithm, which extracts a guaranteed number of reliable feature trajectories for robust mesh grid warping. We first estimate feature trajectories through a video sequence and transform the feature positions into rolling-free smoothed positions. When the number of the estimated trajectories is insufficient, we generate virtual trajectories by augmenting incomplete trajectories using a low-rank matrix completion scheme. Next, we detect feature points on a large moving object and exclude them so as to stabilize camera movements, rather than object movements. With the selected feature points, we set a mesh grid on each frame and warp each grid cell by moving the original feature positions to the smoothed ones. For robust warping, we formulate a cost function based on the reliability weights of each feature point and each grid cell. The cost function consists of a data term, a structure-preserving term, and a regularization term. By minimizing the cost function, we determine the robust mesh grid warping and achieve the stabilization. Experimental results demonstrate that the proposed algorithm reconstructs videos more stably than the conventional algorithms.",
keywords = "and mesh grid warping, low-rank matrix completion, reliable feature selection, rolling shutter distortion, Video stabilization",
author = "Koh, {Yeong Jun} and Chulwoo Lee and Chang-Su Kim",
year = "2015",
month = "12",
day = "1",
doi = "10.1109/TIP.2015.2479918",
language = "English",
volume = "24",
pages = "5260--5273",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "12",

}

TY - JOUR

T1 - Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping

AU - Koh, Yeong Jun

AU - Lee, Chulwoo

AU - Kim, Chang-Su

PY - 2015/12/1

Y1 - 2015/12/1

N2 - We propose a video stabilization algorithm, which extracts a guaranteed number of reliable feature trajectories for robust mesh grid warping. We first estimate feature trajectories through a video sequence and transform the feature positions into rolling-free smoothed positions. When the number of the estimated trajectories is insufficient, we generate virtual trajectories by augmenting incomplete trajectories using a low-rank matrix completion scheme. Next, we detect feature points on a large moving object and exclude them so as to stabilize camera movements, rather than object movements. With the selected feature points, we set a mesh grid on each frame and warp each grid cell by moving the original feature positions to the smoothed ones. For robust warping, we formulate a cost function based on the reliability weights of each feature point and each grid cell. The cost function consists of a data term, a structure-preserving term, and a regularization term. By minimizing the cost function, we determine the robust mesh grid warping and achieve the stabilization. Experimental results demonstrate that the proposed algorithm reconstructs videos more stably than the conventional algorithms.

AB - We propose a video stabilization algorithm, which extracts a guaranteed number of reliable feature trajectories for robust mesh grid warping. We first estimate feature trajectories through a video sequence and transform the feature positions into rolling-free smoothed positions. When the number of the estimated trajectories is insufficient, we generate virtual trajectories by augmenting incomplete trajectories using a low-rank matrix completion scheme. Next, we detect feature points on a large moving object and exclude them so as to stabilize camera movements, rather than object movements. With the selected feature points, we set a mesh grid on each frame and warp each grid cell by moving the original feature positions to the smoothed ones. For robust warping, we formulate a cost function based on the reliability weights of each feature point and each grid cell. The cost function consists of a data term, a structure-preserving term, and a regularization term. By minimizing the cost function, we determine the robust mesh grid warping and achieve the stabilization. Experimental results demonstrate that the proposed algorithm reconstructs videos more stably than the conventional algorithms.

KW - and mesh grid warping

KW - low-rank matrix completion

KW - reliable feature selection

KW - rolling shutter distortion

KW - Video stabilization

UR - http://www.scopus.com/inward/record.url?scp=84943752476&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84943752476&partnerID=8YFLogxK

U2 - 10.1109/TIP.2015.2479918

DO - 10.1109/TIP.2015.2479918

M3 - Article

AN - SCOPUS:84943752476

VL - 24

SP - 5260

EP - 5273

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 12

M1 - 7271056

ER -