Visual and Haptic Shape Processing in the Human Brain: Unisensory Processing, Multisensory Convergence, and Top-Down Influences

Haemy Lee Masson, Jessica Bulthé, Hans P. Op De Beeck, Christian Wallraven

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing.

Original languageEnglish
Pages (from-to)3402-3412
Number of pages11
JournalCerebral Cortex
Volume26
Issue number8
DOIs
Publication statusPublished - 2016 Aug 1

Fingerprint

Occipital Lobe
Parietal Lobe
Imagery (Psychotherapy)
Brain
Touch
Frontal Lobe
Visual Cortex
Temporal Lobe
Magnetic Resonance Imaging
icodextrin

Keywords

  • haptics
  • multisensory
  • multivoxel analysis
  • shape perception

ASJC Scopus subject areas

  • Medicine(all)
  • Cognitive Neuroscience
  • Cellular and Molecular Neuroscience

Cite this

Visual and Haptic Shape Processing in the Human Brain : Unisensory Processing, Multisensory Convergence, and Top-Down Influences. / Lee Masson, Haemy; Bulthé, Jessica; Op De Beeck, Hans P.; Wallraven, Christian.

In: Cerebral Cortex, Vol. 26, No. 8, 01.08.2016, p. 3402-3412.

Research output: Contribution to journalArticle

@article{dec0cc7bb7d247359e806b17867356e3,
title = "Visual and Haptic Shape Processing in the Human Brain: Unisensory Processing, Multisensory Convergence, and Top-Down Influences",
abstract = "Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing.",
keywords = "haptics, multisensory, multivoxel analysis, shape perception",
author = "{Lee Masson}, Haemy and Jessica Bulth{\'e} and {Op De Beeck}, {Hans P.} and Christian Wallraven",
year = "2016",
month = "8",
day = "1",
doi = "10.1093/cercor/bhv170",
language = "English",
volume = "26",
pages = "3402--3412",
journal = "Cerebral Cortex",
issn = "1047-3211",
publisher = "Oxford University Press",
number = "8",

}

TY - JOUR

T1 - Visual and Haptic Shape Processing in the Human Brain

T2 - Unisensory Processing, Multisensory Convergence, and Top-Down Influences

AU - Lee Masson, Haemy

AU - Bulthé, Jessica

AU - Op De Beeck, Hans P.

AU - Wallraven, Christian

PY - 2016/8/1

Y1 - 2016/8/1

N2 - Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing.

AB - Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing.

KW - haptics

KW - multisensory

KW - multivoxel analysis

KW - shape perception

UR - http://www.scopus.com/inward/record.url?scp=84981262626&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84981262626&partnerID=8YFLogxK

U2 - 10.1093/cercor/bhv170

DO - 10.1093/cercor/bhv170

M3 - Article

C2 - 26223258

AN - SCOPUS:84981262626

VL - 26

SP - 3402

EP - 3412

JO - Cerebral Cortex

JF - Cerebral Cortex

SN - 1047-3211

IS - 8

ER -