Estimation of 3D shape from image orientations

Roland W. Fleming, Daniel Holtmann-Rice, Heinrich Bulthoff

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

One of the main functions of vision is to estimate the 3D shape of objects in our environment. Many different visual cues, such as stereopsis, motion parallax, and shading, are thought to be involved. One important cue that remains poorly understood comes from surface texture markings. When a textured surface is slanted in 3D relative to the observer, the surface patterns appear compressed in the retinal image, providing potentially important information about 3D shape. What is not known, however, is how the brain actually measures this information from the retinal image. Here, we explain how the key information could be extracted by populations of cells tuned to different orientations and spatial frequencies, like those found in the primary visual cortex. To test this theory, we created stimuli that selectively stimulate such cell populations, by "smearing" (filtering) images of 2D random noise into specific oriented patterns. We find that the resulting patterns appear vividly 3D, and that increasing the strength of the orientation signals progressively increases the sense of 3D shape, even though the filtering we apply is physically inconsistent with what would occur with a real object. This finding suggests we have isolated key mechanisms used by the brain to estimate shape from texture. Crucially, we also find that adapting the visual system's orientation detectors to orthogonal patterns causes unoriented random noise to look like a specific 3D shape. Together these findings demonstrate a crucial role of orientation detectors in the perception of 3D shape.

Original languageEnglish
Pages (from-to)20438-20443
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume108
Issue number51
DOIs
Publication statusPublished - 2011 Dec 20

Fingerprint

Cues
Depth Perception
Brain
Visual Cortex
Population

Keywords

  • Complex cells
  • Orientation field
  • Shape perception
  • Surface perception

ASJC Scopus subject areas

  • General

Cite this

Estimation of 3D shape from image orientations. / Fleming, Roland W.; Holtmann-Rice, Daniel; Bulthoff, Heinrich.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 108, No. 51, 20.12.2011, p. 20438-20443.

Research output: Contribution to journalArticle

@article{8ef9607603e540c28e0750da384c5546,
title = "Estimation of 3D shape from image orientations",
abstract = "One of the main functions of vision is to estimate the 3D shape of objects in our environment. Many different visual cues, such as stereopsis, motion parallax, and shading, are thought to be involved. One important cue that remains poorly understood comes from surface texture markings. When a textured surface is slanted in 3D relative to the observer, the surface patterns appear compressed in the retinal image, providing potentially important information about 3D shape. What is not known, however, is how the brain actually measures this information from the retinal image. Here, we explain how the key information could be extracted by populations of cells tuned to different orientations and spatial frequencies, like those found in the primary visual cortex. To test this theory, we created stimuli that selectively stimulate such cell populations, by {"}smearing{"} (filtering) images of 2D random noise into specific oriented patterns. We find that the resulting patterns appear vividly 3D, and that increasing the strength of the orientation signals progressively increases the sense of 3D shape, even though the filtering we apply is physically inconsistent with what would occur with a real object. This finding suggests we have isolated key mechanisms used by the brain to estimate shape from texture. Crucially, we also find that adapting the visual system's orientation detectors to orthogonal patterns causes unoriented random noise to look like a specific 3D shape. Together these findings demonstrate a crucial role of orientation detectors in the perception of 3D shape.",
keywords = "Complex cells, Orientation field, Shape perception, Surface perception",
author = "Fleming, {Roland W.} and Daniel Holtmann-Rice and Heinrich Bulthoff",
year = "2011",
month = "12",
day = "20",
doi = "10.1073/pnas.1114619109",
language = "English",
volume = "108",
pages = "20438--20443",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "51",

}

TY - JOUR

T1 - Estimation of 3D shape from image orientations

AU - Fleming, Roland W.

AU - Holtmann-Rice, Daniel

AU - Bulthoff, Heinrich

PY - 2011/12/20

Y1 - 2011/12/20

N2 - One of the main functions of vision is to estimate the 3D shape of objects in our environment. Many different visual cues, such as stereopsis, motion parallax, and shading, are thought to be involved. One important cue that remains poorly understood comes from surface texture markings. When a textured surface is slanted in 3D relative to the observer, the surface patterns appear compressed in the retinal image, providing potentially important information about 3D shape. What is not known, however, is how the brain actually measures this information from the retinal image. Here, we explain how the key information could be extracted by populations of cells tuned to different orientations and spatial frequencies, like those found in the primary visual cortex. To test this theory, we created stimuli that selectively stimulate such cell populations, by "smearing" (filtering) images of 2D random noise into specific oriented patterns. We find that the resulting patterns appear vividly 3D, and that increasing the strength of the orientation signals progressively increases the sense of 3D shape, even though the filtering we apply is physically inconsistent with what would occur with a real object. This finding suggests we have isolated key mechanisms used by the brain to estimate shape from texture. Crucially, we also find that adapting the visual system's orientation detectors to orthogonal patterns causes unoriented random noise to look like a specific 3D shape. Together these findings demonstrate a crucial role of orientation detectors in the perception of 3D shape.

AB - One of the main functions of vision is to estimate the 3D shape of objects in our environment. Many different visual cues, such as stereopsis, motion parallax, and shading, are thought to be involved. One important cue that remains poorly understood comes from surface texture markings. When a textured surface is slanted in 3D relative to the observer, the surface patterns appear compressed in the retinal image, providing potentially important information about 3D shape. What is not known, however, is how the brain actually measures this information from the retinal image. Here, we explain how the key information could be extracted by populations of cells tuned to different orientations and spatial frequencies, like those found in the primary visual cortex. To test this theory, we created stimuli that selectively stimulate such cell populations, by "smearing" (filtering) images of 2D random noise into specific oriented patterns. We find that the resulting patterns appear vividly 3D, and that increasing the strength of the orientation signals progressively increases the sense of 3D shape, even though the filtering we apply is physically inconsistent with what would occur with a real object. This finding suggests we have isolated key mechanisms used by the brain to estimate shape from texture. Crucially, we also find that adapting the visual system's orientation detectors to orthogonal patterns causes unoriented random noise to look like a specific 3D shape. Together these findings demonstrate a crucial role of orientation detectors in the perception of 3D shape.

KW - Complex cells

KW - Orientation field

KW - Shape perception

KW - Surface perception

UR - http://www.scopus.com/inward/record.url?scp=84855505934&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84855505934&partnerID=8YFLogxK

U2 - 10.1073/pnas.1114619109

DO - 10.1073/pnas.1114619109

M3 - Article

C2 - 22147916

AN - SCOPUS:84855505934

VL - 108

SP - 20438

EP - 20443

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 51

ER -