The face speaks: Contextual and temporal sensitivity to backchannel responses

Andrew J. Aubrey, Douglas W. Cunningham, David Marshall, Paul L. Rosin, AhYoung Shin, Christian Wallraven

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

It is often assumed that one person in a conversation is active (the speaker) and the rest passive (the listeners). Conversational analysis has shown, however, that listeners take an active part in the conversation, providing feedback signals that can control conversational flow. The face plays a vital role in these backchannel responses. A deeper understanding of facial backchannel signals is crucial for many applications in social signal processing, including automatic modeling and analysis of conversations, or in the development of life-like, effective conversational agents. Here, we present results from two experiments testing the sensitivity to the context and the timing of backchannel responses. We utilised sequences from a newly recorded database of 5-minute, two-person conversations. Experiment 1 tested how well participants would be able to match backchannel sequences to their corresponding speaker sequence. On average, participants performed well above chance. Experiment 2 tested how sensitive participants would be to temporal misalignments of the backchannel sequence. Interestingly, participants were able to estimate the correct temporal alignment for the sequence pairs. Taken together, our results show that human conversational skills are highly tuned both towards context and temporal alignment, showing the need for accurate modeling of conversations in social signal processing.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages248-259
Number of pages12
Volume7729 LNCS
EditionPART 2
DOIs
Publication statusPublished - 2013 Apr 15
Event11th Asian Conference on Computer Vision, ACCV 2012 - Daejeon, Korea, Republic of
Duration: 2012 Nov 52012 Nov 6

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 2
Volume7729 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other11th Asian Conference on Computer Vision, ACCV 2012
CountryKorea, Republic of
CityDaejeon
Period12/11/512/11/6

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Fingerprint Dive into the research topics of 'The face speaks: Contextual and temporal sensitivity to backchannel responses'. Together they form a unique fingerprint.

  • Cite this

    Aubrey, A. J., Cunningham, D. W., Marshall, D., Rosin, P. L., Shin, A., & Wallraven, C. (2013). The face speaks: Contextual and temporal sensitivity to backchannel responses. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (PART 2 ed., Vol. 7729 LNCS, pp. 248-259). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7729 LNCS, No. PART 2). https://doi.org/10.1007/978-3-642-37484-5_21