Compressed domain video saliency detection using global and local spatiotemporal features

Se Ho Lee, Je Won Kang, Chang-Su Kim

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.

Original languageEnglish
Pages (from-to)169-183
Number of pages15
JournalJournal of Visual Communication and Image Representation
Volume35
DOIs
Publication statusPublished - 2016 Feb 1

Fingerprint

Textures
Electric fuses
Decoding
Color

Keywords

  • Compressed domain
  • Image analysis
  • Image understanding
  • Motion analysis
  • Partial decoding
  • Spatiotemporal feature
  • Video saliency detection
  • Visual attention

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Media Technology
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Compressed domain video saliency detection using global and local spatiotemporal features. / Lee, Se Ho; Kang, Je Won; Kim, Chang-Su.

In: Journal of Visual Communication and Image Representation, Vol. 35, 01.02.2016, p. 169-183.

Research output: Contribution to journalArticle

@article{e66c56cdc1f449b0aa319d9b039fb092,
title = "Compressed domain video saliency detection using global and local spatiotemporal features",
abstract = "A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.",
keywords = "Compressed domain, Image analysis, Image understanding, Motion analysis, Partial decoding, Spatiotemporal feature, Video saliency detection, Visual attention",
author = "Lee, {Se Ho} and Kang, {Je Won} and Chang-Su Kim",
year = "2016",
month = "2",
day = "1",
doi = "10.1016/j.jvcir.2015.12.011",
language = "English",
volume = "35",
pages = "169--183",
journal = "Journal of Visual Communication and Image Representation",
issn = "1047-3203",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Compressed domain video saliency detection using global and local spatiotemporal features

AU - Lee, Se Ho

AU - Kang, Je Won

AU - Kim, Chang-Su

PY - 2016/2/1

Y1 - 2016/2/1

N2 - A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.

AB - A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.

KW - Compressed domain

KW - Image analysis

KW - Image understanding

KW - Motion analysis

KW - Partial decoding

KW - Spatiotemporal feature

KW - Video saliency detection

KW - Visual attention

UR - http://www.scopus.com/inward/record.url?scp=84952905942&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84952905942&partnerID=8YFLogxK

U2 - 10.1016/j.jvcir.2015.12.011

DO - 10.1016/j.jvcir.2015.12.011

M3 - Article

VL - 35

SP - 169

EP - 183

JO - Journal of Visual Communication and Image Representation

JF - Journal of Visual Communication and Image Representation

SN - 1047-3203

ER -