Spatiotemporal saliency detection for video sequences based on random walk with restart

Hansang Kim, Youngbae Kim, Jae Young Sim, Chang-Su Kim

Research output: Contribution to journalArticle

60 Citations (Scopus)

Abstract

A novel saliency detection algorithm for video sequences based on the random walk with restart (RWR) is proposed in this paper. We adopt RWR to detect spatially and temporally salient regions. More specifically, we first find a temporal saliency distribution using the features of motion distinctiveness, temporal consistency, and abrupt change. Among them, the motion distinctiveness is derived by comparing the motion profiles of image patches. Then, we employ the temporal saliency distribution as a restarting distribution of the random walker. In addition, we design the transition probability matrix for the walker using the spatial features of intensity, color, and compactness. Finally, we estimate the spatiotemporal saliency distribution by finding the steady-state distribution of the walker. The proposed algorithm detects foreground salient objects faithfully, while suppressing cluttered backgrounds effectively, by incorporating the spatial transition matrix and the temporal restarting distribution systematically. Experimental results on various video sequences demonstrate that the proposed algorithm outperforms conventional saliency detection algorithms qualitatively and quantitatively.

Original languageEnglish
Article number7091884
Pages (from-to)2552-2564
Number of pages13
JournalIEEE Transactions on Image Processing
Volume24
Issue number8
DOIs
Publication statusPublished - 2015 Aug 1

Fingerprint

Color

Keywords

  • motion profile
  • random walk with restart
  • Saliency detection
  • spatiotemporal feature
  • video saliency

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Spatiotemporal saliency detection for video sequences based on random walk with restart. / Kim, Hansang; Kim, Youngbae; Sim, Jae Young; Kim, Chang-Su.

In: IEEE Transactions on Image Processing, Vol. 24, No. 8, 7091884, 01.08.2015, p. 2552-2564.

Research output: Contribution to journalArticle

Kim, Hansang ; Kim, Youngbae ; Sim, Jae Young ; Kim, Chang-Su. / Spatiotemporal saliency detection for video sequences based on random walk with restart. In: IEEE Transactions on Image Processing. 2015 ; Vol. 24, No. 8. pp. 2552-2564.
@article{4d5b4b2513c24c73b67847a1f2bcc1f5,
title = "Spatiotemporal saliency detection for video sequences based on random walk with restart",
abstract = "A novel saliency detection algorithm for video sequences based on the random walk with restart (RWR) is proposed in this paper. We adopt RWR to detect spatially and temporally salient regions. More specifically, we first find a temporal saliency distribution using the features of motion distinctiveness, temporal consistency, and abrupt change. Among them, the motion distinctiveness is derived by comparing the motion profiles of image patches. Then, we employ the temporal saliency distribution as a restarting distribution of the random walker. In addition, we design the transition probability matrix for the walker using the spatial features of intensity, color, and compactness. Finally, we estimate the spatiotemporal saliency distribution by finding the steady-state distribution of the walker. The proposed algorithm detects foreground salient objects faithfully, while suppressing cluttered backgrounds effectively, by incorporating the spatial transition matrix and the temporal restarting distribution systematically. Experimental results on various video sequences demonstrate that the proposed algorithm outperforms conventional saliency detection algorithms qualitatively and quantitatively.",
keywords = "motion profile, random walk with restart, Saliency detection, spatiotemporal feature, video saliency",
author = "Hansang Kim and Youngbae Kim and Sim, {Jae Young} and Chang-Su Kim",
year = "2015",
month = "8",
day = "1",
doi = "10.1109/TIP.2015.2425544",
language = "English",
volume = "24",
pages = "2552--2564",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "8",

}

TY - JOUR

T1 - Spatiotemporal saliency detection for video sequences based on random walk with restart

AU - Kim, Hansang

AU - Kim, Youngbae

AU - Sim, Jae Young

AU - Kim, Chang-Su

PY - 2015/8/1

Y1 - 2015/8/1

N2 - A novel saliency detection algorithm for video sequences based on the random walk with restart (RWR) is proposed in this paper. We adopt RWR to detect spatially and temporally salient regions. More specifically, we first find a temporal saliency distribution using the features of motion distinctiveness, temporal consistency, and abrupt change. Among them, the motion distinctiveness is derived by comparing the motion profiles of image patches. Then, we employ the temporal saliency distribution as a restarting distribution of the random walker. In addition, we design the transition probability matrix for the walker using the spatial features of intensity, color, and compactness. Finally, we estimate the spatiotemporal saliency distribution by finding the steady-state distribution of the walker. The proposed algorithm detects foreground salient objects faithfully, while suppressing cluttered backgrounds effectively, by incorporating the spatial transition matrix and the temporal restarting distribution systematically. Experimental results on various video sequences demonstrate that the proposed algorithm outperforms conventional saliency detection algorithms qualitatively and quantitatively.

AB - A novel saliency detection algorithm for video sequences based on the random walk with restart (RWR) is proposed in this paper. We adopt RWR to detect spatially and temporally salient regions. More specifically, we first find a temporal saliency distribution using the features of motion distinctiveness, temporal consistency, and abrupt change. Among them, the motion distinctiveness is derived by comparing the motion profiles of image patches. Then, we employ the temporal saliency distribution as a restarting distribution of the random walker. In addition, we design the transition probability matrix for the walker using the spatial features of intensity, color, and compactness. Finally, we estimate the spatiotemporal saliency distribution by finding the steady-state distribution of the walker. The proposed algorithm detects foreground salient objects faithfully, while suppressing cluttered backgrounds effectively, by incorporating the spatial transition matrix and the temporal restarting distribution systematically. Experimental results on various video sequences demonstrate that the proposed algorithm outperforms conventional saliency detection algorithms qualitatively and quantitatively.

KW - motion profile

KW - random walk with restart

KW - Saliency detection

KW - spatiotemporal feature

KW - video saliency

UR - http://www.scopus.com/inward/record.url?scp=84929378573&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84929378573&partnerID=8YFLogxK

U2 - 10.1109/TIP.2015.2425544

DO - 10.1109/TIP.2015.2425544

M3 - Article

VL - 24

SP - 2552

EP - 2564

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 8

M1 - 7091884

ER -