Robust sign language recognition with hierarchical conditional random fields

Hee Deok Yang, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

Sign language spotting is the task of detection and recognition of signs (words in the predefined vocabulary) and fingerspellings (a combination of continuous alphabets that are not found in signs) in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed, which can distinguish signs, fingerspellings, and nonsign patterns. This is achieved through a hierarchical framework consisting of three steps; (1) Candidate segments of signs and fingerspellings are discriminated with a two-layer conditional random field (CRF). (2) Hand shapes of detected signs and fingerspellings are verified by BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and differ only in hand trajectories. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83% and 78%, respectively.

Original languageEnglish
Title of host publicationProceedings - 2010 20th International Conference on Pattern Recognition, ICPR 2010
Pages2202-2205
Number of pages4
DOIs
Publication statusPublished - 2010
Event2010 20th International Conference on Pattern Recognition, ICPR 2010 - Istanbul, Turkey
Duration: 2010 Aug 232010 Aug 26

Publication series

NameProceedings - International Conference on Pattern Recognition
ISSN (Print)1051-4651

Other

Other2010 20th International Conference on Pattern Recognition, ICPR 2010
CountryTurkey
CityIstanbul
Period10/8/2310/8/26

Keywords

  • Conditional random field
  • Fingerspelling spotting
  • Sign language spotting

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Robust sign language recognition with hierarchical conditional random fields'. Together they form a unique fingerprint.

Cite this