The POETICON corpus

Capturing language use and sensorimotor experience in everyday interaction

K. Pastra, Christian Wallraven, M. Schultze, A. Vatakis, K. Kaulard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Natural language use, acquisition, and understanding takes place usually in multisensory and multimedia communication environments. Therefore, for one to model language in its interaction and integration with sensorimotor experiences, one needs a representative corpus of such interplay. In this paper, we will present the first corpus of language use and sensorimotor experience recordings in everyday human: human interaction, in which spontaneous language communication has been recorded along with corresponding multiview video recordings, recordings of 3D full body kinematics, and 3D tracking of objects in focus. It is a twelve-hour corpus which comprises of six everyday human: human interaction scenes, each one performed 3 times by 4 different English-speaking couples (interaction between a male and a female actor), each couple acting each scene in two settings: a fully naturalistic setting in which 5-camera multi-view video recordings take place, and a high-tech setting, with full body motion capture for both individuals, a 2-camera multiview video recording, and 3D tracking of focus objects. The corpus has been developed within an EU-funded cognitive systems research project, POETICON (http://www.poeticon.eu), and represents a new type of language resources for cognitive systems. Namely, a corpus that reveals the dynamic role of language in its interplay with sensorimotor experiences and which allows one to computationally model this interplay.

Original languageEnglish
Title of host publicationProceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010
PublisherEuropean Language Resources Association (ELRA)
Pages3031-3036
Number of pages6
ISBN (Electronic)2951740867, 9782951740860
Publication statusPublished - 2010 Jan 1
Externally publishedYes
Event7th International Conference on Language Resources and Evaluation, LREC 2010 - Valletta, Malta
Duration: 2010 May 172010 May 23

Other

Other7th International Conference on Language Resources and Evaluation, LREC 2010
CountryMalta
CityValletta
Period10/5/1710/5/23

Fingerprint

video recording
interaction
language
experience
recording
communication
systems research
multimedia
Interaction
Language Use
Sensorimotor
speaking
research project
EU
Language
resources
Communication
Cognitive Systems
Resources
Natural Language

ASJC Scopus subject areas

  • Education
  • Library and Information Sciences
  • Linguistics and Language
  • Language and Linguistics

Cite this

Pastra, K., Wallraven, C., Schultze, M., Vatakis, A., & Kaulard, K. (2010). The POETICON corpus: Capturing language use and sensorimotor experience in everyday interaction. In Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010 (pp. 3031-3036). European Language Resources Association (ELRA).

The POETICON corpus : Capturing language use and sensorimotor experience in everyday interaction. / Pastra, K.; Wallraven, Christian; Schultze, M.; Vatakis, A.; Kaulard, K.

Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010. European Language Resources Association (ELRA), 2010. p. 3031-3036.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Pastra, K, Wallraven, C, Schultze, M, Vatakis, A & Kaulard, K 2010, The POETICON corpus: Capturing language use and sensorimotor experience in everyday interaction. in Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010. European Language Resources Association (ELRA), pp. 3031-3036, 7th International Conference on Language Resources and Evaluation, LREC 2010, Valletta, Malta, 10/5/17.
Pastra K, Wallraven C, Schultze M, Vatakis A, Kaulard K. The POETICON corpus: Capturing language use and sensorimotor experience in everyday interaction. In Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010. European Language Resources Association (ELRA). 2010. p. 3031-3036
Pastra, K. ; Wallraven, Christian ; Schultze, M. ; Vatakis, A. ; Kaulard, K. / The POETICON corpus : Capturing language use and sensorimotor experience in everyday interaction. Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010. European Language Resources Association (ELRA), 2010. pp. 3031-3036
@inproceedings{6ae64061d26c4006ad0294e64630f570,
title = "The POETICON corpus: Capturing language use and sensorimotor experience in everyday interaction",
abstract = "Natural language use, acquisition, and understanding takes place usually in multisensory and multimedia communication environments. Therefore, for one to model language in its interaction and integration with sensorimotor experiences, one needs a representative corpus of such interplay. In this paper, we will present the first corpus of language use and sensorimotor experience recordings in everyday human: human interaction, in which spontaneous language communication has been recorded along with corresponding multiview video recordings, recordings of 3D full body kinematics, and 3D tracking of objects in focus. It is a twelve-hour corpus which comprises of six everyday human: human interaction scenes, each one performed 3 times by 4 different English-speaking couples (interaction between a male and a female actor), each couple acting each scene in two settings: a fully naturalistic setting in which 5-camera multi-view video recordings take place, and a high-tech setting, with full body motion capture for both individuals, a 2-camera multiview video recording, and 3D tracking of focus objects. The corpus has been developed within an EU-funded cognitive systems research project, POETICON (http://www.poeticon.eu), and represents a new type of language resources for cognitive systems. Namely, a corpus that reveals the dynamic role of language in its interplay with sensorimotor experiences and which allows one to computationally model this interplay.",
author = "K. Pastra and Christian Wallraven and M. Schultze and A. Vatakis and K. Kaulard",
year = "2010",
month = "1",
day = "1",
language = "English",
pages = "3031--3036",
booktitle = "Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010",
publisher = "European Language Resources Association (ELRA)",

}

TY - GEN

T1 - The POETICON corpus

T2 - Capturing language use and sensorimotor experience in everyday interaction

AU - Pastra, K.

AU - Wallraven, Christian

AU - Schultze, M.

AU - Vatakis, A.

AU - Kaulard, K.

PY - 2010/1/1

Y1 - 2010/1/1

N2 - Natural language use, acquisition, and understanding takes place usually in multisensory and multimedia communication environments. Therefore, for one to model language in its interaction and integration with sensorimotor experiences, one needs a representative corpus of such interplay. In this paper, we will present the first corpus of language use and sensorimotor experience recordings in everyday human: human interaction, in which spontaneous language communication has been recorded along with corresponding multiview video recordings, recordings of 3D full body kinematics, and 3D tracking of objects in focus. It is a twelve-hour corpus which comprises of six everyday human: human interaction scenes, each one performed 3 times by 4 different English-speaking couples (interaction between a male and a female actor), each couple acting each scene in two settings: a fully naturalistic setting in which 5-camera multi-view video recordings take place, and a high-tech setting, with full body motion capture for both individuals, a 2-camera multiview video recording, and 3D tracking of focus objects. The corpus has been developed within an EU-funded cognitive systems research project, POETICON (http://www.poeticon.eu), and represents a new type of language resources for cognitive systems. Namely, a corpus that reveals the dynamic role of language in its interplay with sensorimotor experiences and which allows one to computationally model this interplay.

AB - Natural language use, acquisition, and understanding takes place usually in multisensory and multimedia communication environments. Therefore, for one to model language in its interaction and integration with sensorimotor experiences, one needs a representative corpus of such interplay. In this paper, we will present the first corpus of language use and sensorimotor experience recordings in everyday human: human interaction, in which spontaneous language communication has been recorded along with corresponding multiview video recordings, recordings of 3D full body kinematics, and 3D tracking of objects in focus. It is a twelve-hour corpus which comprises of six everyday human: human interaction scenes, each one performed 3 times by 4 different English-speaking couples (interaction between a male and a female actor), each couple acting each scene in two settings: a fully naturalistic setting in which 5-camera multi-view video recordings take place, and a high-tech setting, with full body motion capture for both individuals, a 2-camera multiview video recording, and 3D tracking of focus objects. The corpus has been developed within an EU-funded cognitive systems research project, POETICON (http://www.poeticon.eu), and represents a new type of language resources for cognitive systems. Namely, a corpus that reveals the dynamic role of language in its interplay with sensorimotor experiences and which allows one to computationally model this interplay.

UR - http://www.scopus.com/inward/record.url?scp=85006551330&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85006551330&partnerID=8YFLogxK

M3 - Conference contribution

SP - 3031

EP - 3036

BT - Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010

PB - European Language Resources Association (ELRA)

ER -