Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

Hee Deok Yang, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

27 Citations (Scopus)

Abstract

The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.

Original languageEnglish
Pages (from-to)2051-2056
Number of pages6
JournalPattern Recognition Letters
Volume34
Issue number16
DOIs
Publication statusPublished - 2013

Keywords

  • BoostMap embedding
  • Conditional random field
  • Sign language recognition
  • Support vector machine

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine'. Together they form a unique fingerprint.

Cite this