Correspondence matching of multi-view video sequences using mutual information based similarity measure

Soon Young Lee, Jae Young Sim, Chang-Su Kim, Sang Uk Lee

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

We propose a correspondence matching algorithm for multi-view video sequences, which provides reliable performance even when the multiple cameras have significantly different parameters, such as viewing angles and positions. We use an activity vector, which represents the temporal occurrence pattern of moving foreground objects at a pixel position, as an invariant feature for correspondence matching. We first devise a novel similarity measure between activity vectors by considering the joint and individual behavior of the activity vectors. Specifically, we define random variables associated with the activity vectors and measure their similarity using the mutual information between the random variables. Moreover, to find a reliable homography transform between views, we find consistent pixel positions by employing the iterative bidirectional matching. We also refine the matching results of multiple source pixel positions by minimizing a matching cost function based on the Markov random field. Experimental results show that the proposed algorithm provides more accurate and reliable matching performance than the conventional activity-based and feature-based matching algorithms, and therefore can facilitate various applications of visual sensor networks.

Original languageEnglish
Article number6553124
Pages (from-to)1719-1731
Number of pages13
JournalIEEE Transactions on Multimedia
Volume15
Issue number8
DOIs
Publication statusPublished - 2013 Dec 2

Fingerprint

Pixels
Random variables
Cost functions
Sensor networks
Cameras

Keywords

  • Correspondence matching
  • Markov random field
  • Multi-view videos
  • Mutual information based similarity measure
  • Visual sensor networks

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing
  • Media Technology
  • Computer Science Applications

Cite this

Correspondence matching of multi-view video sequences using mutual information based similarity measure. / Lee, Soon Young; Sim, Jae Young; Kim, Chang-Su; Lee, Sang Uk.

In: IEEE Transactions on Multimedia, Vol. 15, No. 8, 6553124, 02.12.2013, p. 1719-1731.

Research output: Contribution to journalArticle

@article{0189d023c21046d9b3c647bd2c58fc6b,
title = "Correspondence matching of multi-view video sequences using mutual information based similarity measure",
abstract = "We propose a correspondence matching algorithm for multi-view video sequences, which provides reliable performance even when the multiple cameras have significantly different parameters, such as viewing angles and positions. We use an activity vector, which represents the temporal occurrence pattern of moving foreground objects at a pixel position, as an invariant feature for correspondence matching. We first devise a novel similarity measure between activity vectors by considering the joint and individual behavior of the activity vectors. Specifically, we define random variables associated with the activity vectors and measure their similarity using the mutual information between the random variables. Moreover, to find a reliable homography transform between views, we find consistent pixel positions by employing the iterative bidirectional matching. We also refine the matching results of multiple source pixel positions by minimizing a matching cost function based on the Markov random field. Experimental results show that the proposed algorithm provides more accurate and reliable matching performance than the conventional activity-based and feature-based matching algorithms, and therefore can facilitate various applications of visual sensor networks.",
keywords = "Correspondence matching, Markov random field, Multi-view videos, Mutual information based similarity measure, Visual sensor networks",
author = "Lee, {Soon Young} and Sim, {Jae Young} and Chang-Su Kim and Lee, {Sang Uk}",
year = "2013",
month = "12",
day = "2",
doi = "10.1109/TMM.2013.2271747",
language = "English",
volume = "15",
pages = "1719--1731",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "8",

}

TY - JOUR

T1 - Correspondence matching of multi-view video sequences using mutual information based similarity measure

AU - Lee, Soon Young

AU - Sim, Jae Young

AU - Kim, Chang-Su

AU - Lee, Sang Uk

PY - 2013/12/2

Y1 - 2013/12/2

N2 - We propose a correspondence matching algorithm for multi-view video sequences, which provides reliable performance even when the multiple cameras have significantly different parameters, such as viewing angles and positions. We use an activity vector, which represents the temporal occurrence pattern of moving foreground objects at a pixel position, as an invariant feature for correspondence matching. We first devise a novel similarity measure between activity vectors by considering the joint and individual behavior of the activity vectors. Specifically, we define random variables associated with the activity vectors and measure their similarity using the mutual information between the random variables. Moreover, to find a reliable homography transform between views, we find consistent pixel positions by employing the iterative bidirectional matching. We also refine the matching results of multiple source pixel positions by minimizing a matching cost function based on the Markov random field. Experimental results show that the proposed algorithm provides more accurate and reliable matching performance than the conventional activity-based and feature-based matching algorithms, and therefore can facilitate various applications of visual sensor networks.

AB - We propose a correspondence matching algorithm for multi-view video sequences, which provides reliable performance even when the multiple cameras have significantly different parameters, such as viewing angles and positions. We use an activity vector, which represents the temporal occurrence pattern of moving foreground objects at a pixel position, as an invariant feature for correspondence matching. We first devise a novel similarity measure between activity vectors by considering the joint and individual behavior of the activity vectors. Specifically, we define random variables associated with the activity vectors and measure their similarity using the mutual information between the random variables. Moreover, to find a reliable homography transform between views, we find consistent pixel positions by employing the iterative bidirectional matching. We also refine the matching results of multiple source pixel positions by minimizing a matching cost function based on the Markov random field. Experimental results show that the proposed algorithm provides more accurate and reliable matching performance than the conventional activity-based and feature-based matching algorithms, and therefore can facilitate various applications of visual sensor networks.

KW - Correspondence matching

KW - Markov random field

KW - Multi-view videos

KW - Mutual information based similarity measure

KW - Visual sensor networks

UR - http://www.scopus.com/inward/record.url?scp=84888381448&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84888381448&partnerID=8YFLogxK

U2 - 10.1109/TMM.2013.2271747

DO - 10.1109/TMM.2013.2271747

M3 - Article

VL - 15

SP - 1719

EP - 1731

JO - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

IS - 8

M1 - 6553124

ER -