Continuous hand gesture recognition based on trajectory shape information

Cheoljong Yang, David K. Han, Hanseok Ko

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures.

Original languageEnglish
JournalPattern Recognition Letters
DOIs
Publication statusAccepted/In press - 2016 Oct 15

Fingerprint

Gesture recognition
Trajectories
Fusion reactions

Keywords

  • Conditional random fields
  • Convolution neural network
  • Gesture recognition
  • Human robot interaction
  • Trajectory segmentation
  • Trajectory shape modeling

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Continuous hand gesture recognition based on trajectory shape information. / Yang, Cheoljong; Han, David K.; Ko, Hanseok.

In: Pattern Recognition Letters, 15.10.2016.

Research output: Contribution to journalArticle

@article{14864dd3515f44f2976556a366746c4b,
title = "Continuous hand gesture recognition based on trajectory shape information",
abstract = "In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures.",
keywords = "Conditional random fields, Convolution neural network, Gesture recognition, Human robot interaction, Trajectory segmentation, Trajectory shape modeling",
author = "Cheoljong Yang and Han, {David K.} and Hanseok Ko",
year = "2016",
month = "10",
day = "15",
doi = "10.1016/j.patrec.2017.05.016",
language = "English",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",

}

TY - JOUR

T1 - Continuous hand gesture recognition based on trajectory shape information

AU - Yang, Cheoljong

AU - Han, David K.

AU - Ko, Hanseok

PY - 2016/10/15

Y1 - 2016/10/15

N2 - In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures.

AB - In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures.

KW - Conditional random fields

KW - Convolution neural network

KW - Gesture recognition

KW - Human robot interaction

KW - Trajectory segmentation

KW - Trajectory shape modeling

UR - http://www.scopus.com/inward/record.url?scp=85019950586&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019950586&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2017.05.016

DO - 10.1016/j.patrec.2017.05.016

M3 - Article

AN - SCOPUS:85019950586

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

ER -