Streaming video segmentation via short-term hierarchical segmentation and frame-by-frame Markov random field optimization

Won Dong Jang, Chang-Su Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

An online video segmentation algorithm, based on shortterm hierarchical segmentation (STHS) and frame-by-frame Markov random field (MRF) optimization, is proposed in this work. We develop the STHS technique, which generates initial segments by sliding a short window of frames. In STHS, we apply spatial agglomerative clustering to each frame, and then adopt inter-frame bipartite graph matching to construct initial segments. Then, we partition each frame into final segments, by minimizing an MRF energy function composed of unary and pair wise costs. We compute the unary cost using the STHS initial segments and the segmentation result at the previous frame. We set the pair wise cost to encourage similar nodes to have the same segment label. Experimental results on a video segmentation benchmark dataset, VSB100, demonstrate that the proposed algorithm outperforms state-of-the-art online video segmentation techniques significantly.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
PublisherSpringer Verlag
Pages599-615
Number of pages17
Volume9910 LNCS
ISBN (Print)9783319464657
DOIs
Publication statusPublished - 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9910 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

    Fingerprint

Keywords

  • Agglomerative clustering
  • Graph matching
  • Online segmentation
  • Streaming segmentation
  • Video segmentation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Jang, W. D., & Kim, C-S. (2016). Streaming video segmentation via short-term hierarchical segmentation and frame-by-frame Markov random field optimization. In Computer Vision - 14th European Conference, ECCV 2016, Proceedings (Vol. 9910 LNCS, pp. 599-615). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46466-4_36