Recognizing conversational expressions using latent dynamic conditional random fields

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Facial expressions are one of the most important elements for our social interaction. Automatic processing and recognition of facial expressions is hence one of the core areas in computer vision, computer graphics, and social signal processing. Conditional Random Fields (CRFs) and their extensions are widely used for recognizing facial expressions. Most research in this area, however, is done either with a limited set of emotional expressions (such as the six universal expressions), or it concentrates on extracting facial action units (individual muscle movements) from video sequences. Little research has been conducted to analyze the complex facial movements that occur in conversational contexts. Conversational expressions such as 'agree', 'disagree', 'thinking', 'looking confused', however, form an integral part of non-verbal communication and systems that can automatically parse and understand such expressions are a key ingredient for the development of efficient human-computer interaction systems. Since conversational expressions may consists of several sub-expressions and contain complex dynamics, however, standard CRF approaches are not suited for the task. In this paper, we conduct a detailed comparison of CRFs and Latent Dynamic Conditional Random Fields (LDCRFs) for recognizing complex conversational expressions. We show the importance of modeling sub-expression dynamics and discuss challenges for applying LDCRFs to recognize a large set of conversational expressions.

Original languageEnglish
Title of host publicationProceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013
PublisherIEEE Computer Society
Pages697-701
Number of pages5
DOIs
Publication statusPublished - 2013 Jan 1
Event2013 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013 - Naha, Okinawa, Japan
Duration: 2013 Nov 52013 Nov 8

Other

Other2013 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013
CountryJapan
CityNaha, Okinawa
Period13/11/513/11/8

Fingerprint

Computer graphics
Human computer interaction
Computer vision
Muscle
Signal processing
Communication
Processing

Keywords

  • Facial Expression Analysis
  • Human Computer Interaction
  • Sequence Modeling

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Hur, D., Wallraven, C., & Lee, S. W. (2013). Recognizing conversational expressions using latent dynamic conditional random fields. In Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013 (pp. 697-701). [6778408] IEEE Computer Society. https://doi.org/10.1109/ACPR.2013.98

Recognizing conversational expressions using latent dynamic conditional random fields. / Hur, Dongcheol; Wallraven, Christian; Lee, Seong Whan.

Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013. IEEE Computer Society, 2013. p. 697-701 6778408.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hur, D, Wallraven, C & Lee, SW 2013, Recognizing conversational expressions using latent dynamic conditional random fields. in Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013., 6778408, IEEE Computer Society, pp. 697-701, 2013 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013, Naha, Okinawa, Japan, 13/11/5. https://doi.org/10.1109/ACPR.2013.98
Hur D, Wallraven C, Lee SW. Recognizing conversational expressions using latent dynamic conditional random fields. In Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013. IEEE Computer Society. 2013. p. 697-701. 6778408 https://doi.org/10.1109/ACPR.2013.98
Hur, Dongcheol ; Wallraven, Christian ; Lee, Seong Whan. / Recognizing conversational expressions using latent dynamic conditional random fields. Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013. IEEE Computer Society, 2013. pp. 697-701
@inproceedings{24aa2f1d00de43429c85a227fd7e84bb,
title = "Recognizing conversational expressions using latent dynamic conditional random fields",
abstract = "Facial expressions are one of the most important elements for our social interaction. Automatic processing and recognition of facial expressions is hence one of the core areas in computer vision, computer graphics, and social signal processing. Conditional Random Fields (CRFs) and their extensions are widely used for recognizing facial expressions. Most research in this area, however, is done either with a limited set of emotional expressions (such as the six universal expressions), or it concentrates on extracting facial action units (individual muscle movements) from video sequences. Little research has been conducted to analyze the complex facial movements that occur in conversational contexts. Conversational expressions such as 'agree', 'disagree', 'thinking', 'looking confused', however, form an integral part of non-verbal communication and systems that can automatically parse and understand such expressions are a key ingredient for the development of efficient human-computer interaction systems. Since conversational expressions may consists of several sub-expressions and contain complex dynamics, however, standard CRF approaches are not suited for the task. In this paper, we conduct a detailed comparison of CRFs and Latent Dynamic Conditional Random Fields (LDCRFs) for recognizing complex conversational expressions. We show the importance of modeling sub-expression dynamics and discuss challenges for applying LDCRFs to recognize a large set of conversational expressions.",
keywords = "Facial Expression Analysis, Human Computer Interaction, Sequence Modeling",
author = "Dongcheol Hur and Christian Wallraven and Lee, {Seong Whan}",
year = "2013",
month = "1",
day = "1",
doi = "10.1109/ACPR.2013.98",
language = "English",
pages = "697--701",
booktitle = "Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Recognizing conversational expressions using latent dynamic conditional random fields

AU - Hur, Dongcheol

AU - Wallraven, Christian

AU - Lee, Seong Whan

PY - 2013/1/1

Y1 - 2013/1/1

N2 - Facial expressions are one of the most important elements for our social interaction. Automatic processing and recognition of facial expressions is hence one of the core areas in computer vision, computer graphics, and social signal processing. Conditional Random Fields (CRFs) and their extensions are widely used for recognizing facial expressions. Most research in this area, however, is done either with a limited set of emotional expressions (such as the six universal expressions), or it concentrates on extracting facial action units (individual muscle movements) from video sequences. Little research has been conducted to analyze the complex facial movements that occur in conversational contexts. Conversational expressions such as 'agree', 'disagree', 'thinking', 'looking confused', however, form an integral part of non-verbal communication and systems that can automatically parse and understand such expressions are a key ingredient for the development of efficient human-computer interaction systems. Since conversational expressions may consists of several sub-expressions and contain complex dynamics, however, standard CRF approaches are not suited for the task. In this paper, we conduct a detailed comparison of CRFs and Latent Dynamic Conditional Random Fields (LDCRFs) for recognizing complex conversational expressions. We show the importance of modeling sub-expression dynamics and discuss challenges for applying LDCRFs to recognize a large set of conversational expressions.

AB - Facial expressions are one of the most important elements for our social interaction. Automatic processing and recognition of facial expressions is hence one of the core areas in computer vision, computer graphics, and social signal processing. Conditional Random Fields (CRFs) and their extensions are widely used for recognizing facial expressions. Most research in this area, however, is done either with a limited set of emotional expressions (such as the six universal expressions), or it concentrates on extracting facial action units (individual muscle movements) from video sequences. Little research has been conducted to analyze the complex facial movements that occur in conversational contexts. Conversational expressions such as 'agree', 'disagree', 'thinking', 'looking confused', however, form an integral part of non-verbal communication and systems that can automatically parse and understand such expressions are a key ingredient for the development of efficient human-computer interaction systems. Since conversational expressions may consists of several sub-expressions and contain complex dynamics, however, standard CRF approaches are not suited for the task. In this paper, we conduct a detailed comparison of CRFs and Latent Dynamic Conditional Random Fields (LDCRFs) for recognizing complex conversational expressions. We show the importance of modeling sub-expression dynamics and discuss challenges for applying LDCRFs to recognize a large set of conversational expressions.

KW - Facial Expression Analysis

KW - Human Computer Interaction

KW - Sequence Modeling

UR - http://www.scopus.com/inward/record.url?scp=84899082368&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84899082368&partnerID=8YFLogxK

U2 - 10.1109/ACPR.2013.98

DO - 10.1109/ACPR.2013.98

M3 - Conference contribution

AN - SCOPUS:84899082368

SP - 697

EP - 701

BT - Proceedings - 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013

PB - IEEE Computer Society

ER -