Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter

Chang Beom Park, Seong Whan Lee

Research output: Contribution to journalArticle

46 Citations (Scopus)

Abstract

In this paper, we present a real-time 3D pointing gesture recognition algorithm for mobile robots, based on a cascade hidden Markov model (HMM) and a particle filter. Among the various human gestures, the pointing gesture is very useful to human-robot interaction (HRI). In fact, it is highly intuitive, does not involve a-priori assumptions, and has no substitute in other modes of interaction. A major issue in pointing gesture recognition is the difficultly of accurate estimation of the pointing direction, caused by the difficulty of hand tracking and the unreliability of the direction estimation. The proposed method involves the use of a stereo camera and 3D particle filters for reliable hand tracking, and a cascade of two HMMs for a robust estimate of the pointing direction. When a subject enters the field of view of the camera, his or her face and two hands are located and tracked using particle filters. The first stage HMM takes the hand position estimate and maps it to a more accurate position by modeling the kinematic characteristics of finger pointing. The resulting 3D coordinates are used as input into the second stage HMM that discriminates pointing gestures from other types. Finally, the pointing direction is estimated for the pointing state. The proposed method can deal with both large and small pointing gestures. The experimental results show gesture recognition and target selection rates of better than 89% and 99% respectively, during human-robot interaction.

Original languageEnglish
Pages (from-to)51-63
Number of pages13
JournalImage and Vision Computing
Volume29
Issue number1
DOIs
Publication statusPublished - 2011 Jan 1

Fingerprint

Gesture recognition
Hidden Markov models
Mobile robots
Human robot interaction
Cameras
Kinematics

Keywords

  • 3D particle filter
  • Cascade HMM
  • Human-robot interaction
  • Pointing gesture recognition

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter. / Park, Chang Beom; Lee, Seong Whan.

In: Image and Vision Computing, Vol. 29, No. 1, 01.01.2011, p. 51-63.

Research output: Contribution to journalArticle

@article{e687d4aef3234345bcd0dbe917eb64ef,
title = "Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter",
abstract = "In this paper, we present a real-time 3D pointing gesture recognition algorithm for mobile robots, based on a cascade hidden Markov model (HMM) and a particle filter. Among the various human gestures, the pointing gesture is very useful to human-robot interaction (HRI). In fact, it is highly intuitive, does not involve a-priori assumptions, and has no substitute in other modes of interaction. A major issue in pointing gesture recognition is the difficultly of accurate estimation of the pointing direction, caused by the difficulty of hand tracking and the unreliability of the direction estimation. The proposed method involves the use of a stereo camera and 3D particle filters for reliable hand tracking, and a cascade of two HMMs for a robust estimate of the pointing direction. When a subject enters the field of view of the camera, his or her face and two hands are located and tracked using particle filters. The first stage HMM takes the hand position estimate and maps it to a more accurate position by modeling the kinematic characteristics of finger pointing. The resulting 3D coordinates are used as input into the second stage HMM that discriminates pointing gestures from other types. Finally, the pointing direction is estimated for the pointing state. The proposed method can deal with both large and small pointing gestures. The experimental results show gesture recognition and target selection rates of better than 89{\%} and 99{\%} respectively, during human-robot interaction.",
keywords = "3D particle filter, Cascade HMM, Human-robot interaction, Pointing gesture recognition",
author = "Park, {Chang Beom} and Lee, {Seong Whan}",
year = "2011",
month = "1",
day = "1",
doi = "10.1016/j.imavis.2010.08.006",
language = "English",
volume = "29",
pages = "51--63",
journal = "Image and Vision Computing",
issn = "0262-8856",
publisher = "Elsevier Limited",
number = "1",

}

TY - JOUR

T1 - Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter

AU - Park, Chang Beom

AU - Lee, Seong Whan

PY - 2011/1/1

Y1 - 2011/1/1

N2 - In this paper, we present a real-time 3D pointing gesture recognition algorithm for mobile robots, based on a cascade hidden Markov model (HMM) and a particle filter. Among the various human gestures, the pointing gesture is very useful to human-robot interaction (HRI). In fact, it is highly intuitive, does not involve a-priori assumptions, and has no substitute in other modes of interaction. A major issue in pointing gesture recognition is the difficultly of accurate estimation of the pointing direction, caused by the difficulty of hand tracking and the unreliability of the direction estimation. The proposed method involves the use of a stereo camera and 3D particle filters for reliable hand tracking, and a cascade of two HMMs for a robust estimate of the pointing direction. When a subject enters the field of view of the camera, his or her face and two hands are located and tracked using particle filters. The first stage HMM takes the hand position estimate and maps it to a more accurate position by modeling the kinematic characteristics of finger pointing. The resulting 3D coordinates are used as input into the second stage HMM that discriminates pointing gestures from other types. Finally, the pointing direction is estimated for the pointing state. The proposed method can deal with both large and small pointing gestures. The experimental results show gesture recognition and target selection rates of better than 89% and 99% respectively, during human-robot interaction.

AB - In this paper, we present a real-time 3D pointing gesture recognition algorithm for mobile robots, based on a cascade hidden Markov model (HMM) and a particle filter. Among the various human gestures, the pointing gesture is very useful to human-robot interaction (HRI). In fact, it is highly intuitive, does not involve a-priori assumptions, and has no substitute in other modes of interaction. A major issue in pointing gesture recognition is the difficultly of accurate estimation of the pointing direction, caused by the difficulty of hand tracking and the unreliability of the direction estimation. The proposed method involves the use of a stereo camera and 3D particle filters for reliable hand tracking, and a cascade of two HMMs for a robust estimate of the pointing direction. When a subject enters the field of view of the camera, his or her face and two hands are located and tracked using particle filters. The first stage HMM takes the hand position estimate and maps it to a more accurate position by modeling the kinematic characteristics of finger pointing. The resulting 3D coordinates are used as input into the second stage HMM that discriminates pointing gestures from other types. Finally, the pointing direction is estimated for the pointing state. The proposed method can deal with both large and small pointing gestures. The experimental results show gesture recognition and target selection rates of better than 89% and 99% respectively, during human-robot interaction.

KW - 3D particle filter

KW - Cascade HMM

KW - Human-robot interaction

KW - Pointing gesture recognition

UR - http://www.scopus.com/inward/record.url?scp=77957697336&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77957697336&partnerID=8YFLogxK

U2 - 10.1016/j.imavis.2010.08.006

DO - 10.1016/j.imavis.2010.08.006

M3 - Article

AN - SCOPUS:77957697336

VL - 29

SP - 51

EP - 63

JO - Image and Vision Computing

JF - Image and Vision Computing

SN - 0262-8856

IS - 1

ER -