Multiscale saliency detection using random walk with restart

Jun Seong Kim, Jae Young Sim, Chang-Su Kim

Research output: Contribution to journalArticle

44 Citations (Scopus)

Abstract

In this paper, we propose a graph-based multiscale saliency-detection algorithm by modeling eye movements as a random walk on a graph. The proposed algorithm first extracts intensity, color, and compactness features from an input image. It then constructs a fully connected graph by employing image blocks as the nodes. It assigns a high edge weight if the two connected nodes have dissimilar intensity and color features and if the ending node is more compact than the starting node. Then, the proposed algorithm computes the stationary distribution of the Markov chain on the graph as the saliency map. However, the performance of the saliency detection depends on the relative block size in an image. To provide a more reliable saliency map, we develop a coarse-to-fine refinement technique for multiscale saliency maps based on the random walk with restart (RWR). Specifically, we use the saliency map at a coarse scale as the restarting distribution of RWR at a fine scale. Experimental results demonstrate that the proposed algorithm detects visual saliency precisely and reliably. Moreover, the proposed algorithm can be efficiently used in the applications of proto-object extraction and image retargeting.

Original languageEnglish
Article number6544572
Pages (from-to)198-210
Number of pages13
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume24
Issue number2
DOIs
Publication statusPublished - 2014 Feb 1

Fingerprint

Color
Eye movements
Markov processes

Keywords

  • Compactness feature
  • hierarchical saliency refinement
  • Markov chain
  • multiscale saliency detection
  • random walk with restart

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Media Technology

Cite this

Multiscale saliency detection using random walk with restart. / Kim, Jun Seong; Sim, Jae Young; Kim, Chang-Su.

In: IEEE Transactions on Circuits and Systems for Video Technology, Vol. 24, No. 2, 6544572, 01.02.2014, p. 198-210.

Research output: Contribution to journalArticle

@article{ee9dcc22bd694c2a81d07c79eea48276,
title = "Multiscale saliency detection using random walk with restart",
abstract = "In this paper, we propose a graph-based multiscale saliency-detection algorithm by modeling eye movements as a random walk on a graph. The proposed algorithm first extracts intensity, color, and compactness features from an input image. It then constructs a fully connected graph by employing image blocks as the nodes. It assigns a high edge weight if the two connected nodes have dissimilar intensity and color features and if the ending node is more compact than the starting node. Then, the proposed algorithm computes the stationary distribution of the Markov chain on the graph as the saliency map. However, the performance of the saliency detection depends on the relative block size in an image. To provide a more reliable saliency map, we develop a coarse-to-fine refinement technique for multiscale saliency maps based on the random walk with restart (RWR). Specifically, we use the saliency map at a coarse scale as the restarting distribution of RWR at a fine scale. Experimental results demonstrate that the proposed algorithm detects visual saliency precisely and reliably. Moreover, the proposed algorithm can be efficiently used in the applications of proto-object extraction and image retargeting.",
keywords = "Compactness feature, hierarchical saliency refinement, Markov chain, multiscale saliency detection, random walk with restart",
author = "Kim, {Jun Seong} and Sim, {Jae Young} and Chang-Su Kim",
year = "2014",
month = "2",
day = "1",
doi = "10.1109/TCSVT.2013.2270366",
language = "English",
volume = "24",
pages = "198--210",
journal = "IEEE Transactions on Circuits and Systems for Video Technology",
issn = "1051-8215",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

TY - JOUR

T1 - Multiscale saliency detection using random walk with restart

AU - Kim, Jun Seong

AU - Sim, Jae Young

AU - Kim, Chang-Su

PY - 2014/2/1

Y1 - 2014/2/1

N2 - In this paper, we propose a graph-based multiscale saliency-detection algorithm by modeling eye movements as a random walk on a graph. The proposed algorithm first extracts intensity, color, and compactness features from an input image. It then constructs a fully connected graph by employing image blocks as the nodes. It assigns a high edge weight if the two connected nodes have dissimilar intensity and color features and if the ending node is more compact than the starting node. Then, the proposed algorithm computes the stationary distribution of the Markov chain on the graph as the saliency map. However, the performance of the saliency detection depends on the relative block size in an image. To provide a more reliable saliency map, we develop a coarse-to-fine refinement technique for multiscale saliency maps based on the random walk with restart (RWR). Specifically, we use the saliency map at a coarse scale as the restarting distribution of RWR at a fine scale. Experimental results demonstrate that the proposed algorithm detects visual saliency precisely and reliably. Moreover, the proposed algorithm can be efficiently used in the applications of proto-object extraction and image retargeting.

AB - In this paper, we propose a graph-based multiscale saliency-detection algorithm by modeling eye movements as a random walk on a graph. The proposed algorithm first extracts intensity, color, and compactness features from an input image. It then constructs a fully connected graph by employing image blocks as the nodes. It assigns a high edge weight if the two connected nodes have dissimilar intensity and color features and if the ending node is more compact than the starting node. Then, the proposed algorithm computes the stationary distribution of the Markov chain on the graph as the saliency map. However, the performance of the saliency detection depends on the relative block size in an image. To provide a more reliable saliency map, we develop a coarse-to-fine refinement technique for multiscale saliency maps based on the random walk with restart (RWR). Specifically, we use the saliency map at a coarse scale as the restarting distribution of RWR at a fine scale. Experimental results demonstrate that the proposed algorithm detects visual saliency precisely and reliably. Moreover, the proposed algorithm can be efficiently used in the applications of proto-object extraction and image retargeting.

KW - Compactness feature

KW - hierarchical saliency refinement

KW - Markov chain

KW - multiscale saliency detection

KW - random walk with restart

UR - http://www.scopus.com/inward/record.url?scp=84894516003&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84894516003&partnerID=8YFLogxK

U2 - 10.1109/TCSVT.2013.2270366

DO - 10.1109/TCSVT.2013.2270366

M3 - Article

VL - 24

SP - 198

EP - 210

JO - IEEE Transactions on Circuits and Systems for Video Technology

JF - IEEE Transactions on Circuits and Systems for Video Technology

SN - 1051-8215

IS - 2

M1 - 6544572

ER -