Robust sign language recognition with hierarchical conditional random fields

Hee Deok Yang, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

Sign language spotting is the task of detection and recognition of signs (words in the predefined vocabulary) and fingerspellings (a combination of continuous alphabets that are not found in signs) in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed, which can distinguish signs, fingerspellings, and nonsign patterns. This is achieved through a hierarchical framework consisting of three steps; (1) Candidate segments of signs and fingerspellings are discriminated with a two-layer conditional random field (CRF). (2) Hand shapes of detected signs and fingerspellings are verified by BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and differ only in hand trajectories. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83% and 78%, respectively.

Original languageEnglish
Title of host publicationProceedings - International Conference on Pattern Recognition
Pages2202-2205
Number of pages4
DOIs
Publication statusPublished - 2010 Nov 18
Event2010 20th International Conference on Pattern Recognition, ICPR 2010 - Istanbul, Turkey
Duration: 2010 Aug 232010 Aug 26

Other

Other2010 20th International Conference on Pattern Recognition, ICPR 2010
CountryTurkey
CityIstanbul
Period10/8/2310/8/26

Fingerprint

Trajectories
Experiments

Keywords

  • Conditional random field
  • Fingerspelling spotting
  • Sign language spotting

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Yang, H. D., & Lee, S. W. (2010). Robust sign language recognition with hierarchical conditional random fields. In Proceedings - International Conference on Pattern Recognition (pp. 2202-2205). [5595973] https://doi.org/10.1109/ICPR.2010.539

Robust sign language recognition with hierarchical conditional random fields. / Yang, Hee Deok; Lee, Seong Whan.

Proceedings - International Conference on Pattern Recognition. 2010. p. 2202-2205 5595973.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, HD & Lee, SW 2010, Robust sign language recognition with hierarchical conditional random fields. in Proceedings - International Conference on Pattern Recognition., 5595973, pp. 2202-2205, 2010 20th International Conference on Pattern Recognition, ICPR 2010, Istanbul, Turkey, 10/8/23. https://doi.org/10.1109/ICPR.2010.539
Yang HD, Lee SW. Robust sign language recognition with hierarchical conditional random fields. In Proceedings - International Conference on Pattern Recognition. 2010. p. 2202-2205. 5595973 https://doi.org/10.1109/ICPR.2010.539
Yang, Hee Deok ; Lee, Seong Whan. / Robust sign language recognition with hierarchical conditional random fields. Proceedings - International Conference on Pattern Recognition. 2010. pp. 2202-2205
@inproceedings{4832bbec7d40410897f5e5b280b595cd,
title = "Robust sign language recognition with hierarchical conditional random fields",
abstract = "Sign language spotting is the task of detection and recognition of signs (words in the predefined vocabulary) and fingerspellings (a combination of continuous alphabets that are not found in signs) in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed, which can distinguish signs, fingerspellings, and nonsign patterns. This is achieved through a hierarchical framework consisting of three steps; (1) Candidate segments of signs and fingerspellings are discriminated with a two-layer conditional random field (CRF). (2) Hand shapes of detected signs and fingerspellings are verified by BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and differ only in hand trajectories. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83{\%} and 78{\%}, respectively.",
keywords = "Conditional random field, Fingerspelling spotting, Sign language spotting",
author = "Yang, {Hee Deok} and Lee, {Seong Whan}",
year = "2010",
month = "11",
day = "18",
doi = "10.1109/ICPR.2010.539",
language = "English",
isbn = "9780769541099",
pages = "2202--2205",
booktitle = "Proceedings - International Conference on Pattern Recognition",

}

TY - GEN

T1 - Robust sign language recognition with hierarchical conditional random fields

AU - Yang, Hee Deok

AU - Lee, Seong Whan

PY - 2010/11/18

Y1 - 2010/11/18

N2 - Sign language spotting is the task of detection and recognition of signs (words in the predefined vocabulary) and fingerspellings (a combination of continuous alphabets that are not found in signs) in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed, which can distinguish signs, fingerspellings, and nonsign patterns. This is achieved through a hierarchical framework consisting of three steps; (1) Candidate segments of signs and fingerspellings are discriminated with a two-layer conditional random field (CRF). (2) Hand shapes of detected signs and fingerspellings are verified by BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and differ only in hand trajectories. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83% and 78%, respectively.

AB - Sign language spotting is the task of detection and recognition of signs (words in the predefined vocabulary) and fingerspellings (a combination of continuous alphabets that are not found in signs) in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed, which can distinguish signs, fingerspellings, and nonsign patterns. This is achieved through a hierarchical framework consisting of three steps; (1) Candidate segments of signs and fingerspellings are discriminated with a two-layer conditional random field (CRF). (2) Hand shapes of detected signs and fingerspellings are verified by BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and differ only in hand trajectories. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83% and 78%, respectively.

KW - Conditional random field

KW - Fingerspelling spotting

KW - Sign language spotting

UR - http://www.scopus.com/inward/record.url?scp=78149488216&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78149488216&partnerID=8YFLogxK

U2 - 10.1109/ICPR.2010.539

DO - 10.1109/ICPR.2010.539

M3 - Conference contribution

AN - SCOPUS:78149488216

SN - 9780769541099

SP - 2202

EP - 2205

BT - Proceedings - International Conference on Pattern Recognition

ER -