Categorization of Natural Scenes: Local Versus Global Information and the Role of Color

Julia Vogel, Adrian Schwaninger, Christian Wallraven, Heinrich Bulthoff

Research output: Contribution to journalArticle

33 Citations (Scopus)

Abstract

Categorization of scenes is a fundamental process of human vision that allows us to efficiently and rapidly analyze our surroundings. Several studies have explored the processes underlying human scene categorization, but they have focused on processing global image information. In this study, we present both psychophysical and computational experiments that investigate the role of local versus global image information in scene categorization. In a first set of human experiments, categorization performance is tested when only local or only global image information is present. Our results suggest that humans rely on local, region-based information as much as on global, configural information. In addition, humans seem to integrate both types of information for intact scene categorization. In a set of computational experiments, human performance is compared to two state-of-the-art computer vision approaches that have been shown to be psychophysically plausible and that model either local or global information. In addition to the influence of local versus global information, in a second series of experiments, we investigated the effect of color on the categorization performance of both the human observers and the computational model. Analysis of the human data suggests that color is an additional channel of perceptual information that leads to higher categorization results at the expense of increased reaction times in the intact condition. However, it does not affect reaction times when only local information is present. When color is removed, the employed computational model follows the relative performance decrease of human observers for each scene category and can thus be seen as a perceptually plausible model for human scene categorization based on local image information.

Original languageEnglish
Pages (from-to)19
Number of pages1
JournalACM Transactions on Applied Perception
Volume4
Issue number3
DOIs
Publication statusPublished - 2007
Externally publishedYes

Fingerprint

Categorization
Color
Experiments
Computer vision
Reaction Time
Computational Experiments
Computational Model
Observer
Processing
Human Vision
Human Performance
Human
Computer Vision
Experiment
Integrate
Decrease
Series

Keywords

  • Algorithms
  • computational modeling
  • computationalgist
  • global configural information
  • Human perception
  • local region-based information
  • scene classification
  • Scene perception
  • semantic modeling

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)
  • Experimental and Cognitive Psychology

Cite this

Categorization of Natural Scenes : Local Versus Global Information and the Role of Color. / Vogel, Julia; Schwaninger, Adrian; Wallraven, Christian; Bulthoff, Heinrich.

In: ACM Transactions on Applied Perception, Vol. 4, No. 3, 2007, p. 19.

Research output: Contribution to journalArticle

@article{438f81ef3b0a4da7aa9446b4ed405256,
title = "Categorization of Natural Scenes: Local Versus Global Information and the Role of Color",
abstract = "Categorization of scenes is a fundamental process of human vision that allows us to efficiently and rapidly analyze our surroundings. Several studies have explored the processes underlying human scene categorization, but they have focused on processing global image information. In this study, we present both psychophysical and computational experiments that investigate the role of local versus global image information in scene categorization. In a first set of human experiments, categorization performance is tested when only local or only global image information is present. Our results suggest that humans rely on local, region-based information as much as on global, configural information. In addition, humans seem to integrate both types of information for intact scene categorization. In a set of computational experiments, human performance is compared to two state-of-the-art computer vision approaches that have been shown to be psychophysically plausible and that model either local or global information. In addition to the influence of local versus global information, in a second series of experiments, we investigated the effect of color on the categorization performance of both the human observers and the computational model. Analysis of the human data suggests that color is an additional channel of perceptual information that leads to higher categorization results at the expense of increased reaction times in the intact condition. However, it does not affect reaction times when only local information is present. When color is removed, the employed computational model follows the relative performance decrease of human observers for each scene category and can thus be seen as a perceptually plausible model for human scene categorization based on local image information.",
keywords = "Algorithms, computational modeling, computationalgist, global configural information, Human perception, local region-based information, scene classification, Scene perception, semantic modeling",
author = "Julia Vogel and Adrian Schwaninger and Christian Wallraven and Heinrich Bulthoff",
year = "2007",
doi = "10.1145/1278387.1278393",
language = "English",
volume = "4",
pages = "19",
journal = "ACM Transactions on Applied Perception",
issn = "1544-3558",
publisher = "Association for Computing Machinery (ACM)",
number = "3",

}

TY - JOUR

T1 - Categorization of Natural Scenes

T2 - Local Versus Global Information and the Role of Color

AU - Vogel, Julia

AU - Schwaninger, Adrian

AU - Wallraven, Christian

AU - Bulthoff, Heinrich

PY - 2007

Y1 - 2007

N2 - Categorization of scenes is a fundamental process of human vision that allows us to efficiently and rapidly analyze our surroundings. Several studies have explored the processes underlying human scene categorization, but they have focused on processing global image information. In this study, we present both psychophysical and computational experiments that investigate the role of local versus global image information in scene categorization. In a first set of human experiments, categorization performance is tested when only local or only global image information is present. Our results suggest that humans rely on local, region-based information as much as on global, configural information. In addition, humans seem to integrate both types of information for intact scene categorization. In a set of computational experiments, human performance is compared to two state-of-the-art computer vision approaches that have been shown to be psychophysically plausible and that model either local or global information. In addition to the influence of local versus global information, in a second series of experiments, we investigated the effect of color on the categorization performance of both the human observers and the computational model. Analysis of the human data suggests that color is an additional channel of perceptual information that leads to higher categorization results at the expense of increased reaction times in the intact condition. However, it does not affect reaction times when only local information is present. When color is removed, the employed computational model follows the relative performance decrease of human observers for each scene category and can thus be seen as a perceptually plausible model for human scene categorization based on local image information.

AB - Categorization of scenes is a fundamental process of human vision that allows us to efficiently and rapidly analyze our surroundings. Several studies have explored the processes underlying human scene categorization, but they have focused on processing global image information. In this study, we present both psychophysical and computational experiments that investigate the role of local versus global image information in scene categorization. In a first set of human experiments, categorization performance is tested when only local or only global image information is present. Our results suggest that humans rely on local, region-based information as much as on global, configural information. In addition, humans seem to integrate both types of information for intact scene categorization. In a set of computational experiments, human performance is compared to two state-of-the-art computer vision approaches that have been shown to be psychophysically plausible and that model either local or global information. In addition to the influence of local versus global information, in a second series of experiments, we investigated the effect of color on the categorization performance of both the human observers and the computational model. Analysis of the human data suggests that color is an additional channel of perceptual information that leads to higher categorization results at the expense of increased reaction times in the intact condition. However, it does not affect reaction times when only local information is present. When color is removed, the employed computational model follows the relative performance decrease of human observers for each scene category and can thus be seen as a perceptually plausible model for human scene categorization based on local image information.

KW - Algorithms

KW - computational modeling

KW - computationalgist

KW - global configural information

KW - Human perception

KW - local region-based information

KW - scene classification

KW - Scene perception

KW - semantic modeling

UR - http://www.scopus.com/inward/record.url?scp=85010289707&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010289707&partnerID=8YFLogxK

U2 - 10.1145/1278387.1278393

DO - 10.1145/1278387.1278393

M3 - Article

AN - SCOPUS:85010289707

VL - 4

SP - 19

JO - ACM Transactions on Applied Perception

JF - ACM Transactions on Applied Perception

SN - 1544-3558

IS - 3

ER -