Streaming video segmentation via short-term hierarchical segmentation and frame-by-frame Markov random field optimization

Won Dong Jang, Chang-Su Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

An online video segmentation algorithm, based on shortterm hierarchical segmentation (STHS) and frame-by-frame Markov random field (MRF) optimization, is proposed in this work. We develop the STHS technique, which generates initial segments by sliding a short window of frames. In STHS, we apply spatial agglomerative clustering to each frame, and then adopt inter-frame bipartite graph matching to construct initial segments. Then, we partition each frame into final segments, by minimizing an MRF energy function composed of unary and pair wise costs. We compute the unary cost using the STHS initial segments and the segmentation result at the previous frame. We set the pair wise cost to encourage similar nodes to have the same segment label. Experimental results on a video segmentation benchmark dataset, VSB100, demonstrate that the proposed algorithm outperforms state-of-the-art online video segmentation techniques significantly.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
PublisherSpringer Verlag
Pages599-615
Number of pages17
ISBN (Print)9783319464657
DOIs
Publication statusPublished - 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: 2016 Oct 82016 Oct 16

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9910 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th European Conference on Computer Vision, ECCV 2016
CountryNetherlands
CityAmsterdam
Period16/10/816/10/16

Keywords

  • Agglomerative clustering
  • Graph matching
  • Online segmentation
  • Streaming segmentation
  • Video segmentation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Streaming video segmentation via short-term hierarchical segmentation and frame-by-frame Markov random field optimization'. Together they form a unique fingerprint.

Cite this