Classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition

Daehoon Kim, Seungmin Rho, Sanghoon Jun, Een Jun Hwang

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

In this paper, we propose a classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition using the local features, GPS data and user tags of images. For spatio-temporal landmark image classification, we first divide Earth's entire surface into unit grid cells and collect pictures taken in each cell through Flickr. The collected images contain information such as location, titles and other user tags. Usually, the titles or user tags of landmark images include landmark names. Hence, by analyzing such tags, we can identify promising landmark names in the region and create a collection of images for each landmark using Flickr API. Even though each landmark class contains images of the same landmark, their spatio-temporal features could be different depending on shooting time, distance or angle. Therefore, we further divide the images in each landmark class into several subclasses according to their spatio-temporal characteristics using their color and local features. Especially, we detect the interest points of the images in the class, construct their feature descriptors using SURF and perform statistical analysis to select their representative points. Similar representative points are merged for fast comparison. Finally, we construct an index on the representative points using k-d tree. To identify the landmark in a user query image, we extract its SURF features and search for them in the index. Most similar matches are returned, along with descriptive text and GPS information. We implemented a prototype system based on a client-server architecture and performed various experiments to demonstrate that our scheme can achieve reasonable precision and scalability and provide a new browsing experience to the user.

Original languageEnglish
Pages (from-to)201-213
Number of pages13
JournalIntegrated Computer-Aided Engineering
Volume22
Issue number2
DOIs
Publication statusPublished - 2015

Fingerprint

Landmarks
Indexing
Repository
Global positioning system
Image classification
Application programming interfaces (API)
Scalability
Statistical methods
Servers
Earth (planet)
Color
Local Features
Experiments
Divides
Shooting
Client/server
Image Classification
Cell
Browsing
Descriptors

Keywords

  • Landmark
  • local feature descriptor
  • object recognition
  • spatio-temporal
  • user-aware

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Software
  • Computational Theory and Mathematics
  • Theoretical Computer Science

Cite this

Classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition. / Kim, Daehoon; Rho, Seungmin; Jun, Sanghoon; Hwang, Een Jun.

In: Integrated Computer-Aided Engineering, Vol. 22, No. 2, 2015, p. 201-213.

Research output: Contribution to journalArticle

@article{c3552dc5dcc84f24a6b368faef1a533e,
title = "Classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition",
abstract = "In this paper, we propose a classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition using the local features, GPS data and user tags of images. For spatio-temporal landmark image classification, we first divide Earth's entire surface into unit grid cells and collect pictures taken in each cell through Flickr. The collected images contain information such as location, titles and other user tags. Usually, the titles or user tags of landmark images include landmark names. Hence, by analyzing such tags, we can identify promising landmark names in the region and create a collection of images for each landmark using Flickr API. Even though each landmark class contains images of the same landmark, their spatio-temporal features could be different depending on shooting time, distance or angle. Therefore, we further divide the images in each landmark class into several subclasses according to their spatio-temporal characteristics using their color and local features. Especially, we detect the interest points of the images in the class, construct their feature descriptors using SURF and perform statistical analysis to select their representative points. Similar representative points are merged for fast comparison. Finally, we construct an index on the representative points using k-d tree. To identify the landmark in a user query image, we extract its SURF features and search for them in the index. Most similar matches are returned, along with descriptive text and GPS information. We implemented a prototype system based on a client-server architecture and performed various experiments to demonstrate that our scheme can achieve reasonable precision and scalability and provide a new browsing experience to the user.",
keywords = "Landmark, local feature descriptor, object recognition, spatio-temporal, user-aware",
author = "Daehoon Kim and Seungmin Rho and Sanghoon Jun and Hwang, {Een Jun}",
year = "2015",
doi = "10.3233/ICA-140478",
language = "English",
volume = "22",
pages = "201--213",
journal = "Integrated Computer-Aided Engineering",
issn = "1069-2509",
publisher = "IOS Press",
number = "2",

}

TY - JOUR

T1 - Classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition

AU - Kim, Daehoon

AU - Rho, Seungmin

AU - Jun, Sanghoon

AU - Hwang, Een Jun

PY - 2015

Y1 - 2015

N2 - In this paper, we propose a classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition using the local features, GPS data and user tags of images. For spatio-temporal landmark image classification, we first divide Earth's entire surface into unit grid cells and collect pictures taken in each cell through Flickr. The collected images contain information such as location, titles and other user tags. Usually, the titles or user tags of landmark images include landmark names. Hence, by analyzing such tags, we can identify promising landmark names in the region and create a collection of images for each landmark using Flickr API. Even though each landmark class contains images of the same landmark, their spatio-temporal features could be different depending on shooting time, distance or angle. Therefore, we further divide the images in each landmark class into several subclasses according to their spatio-temporal characteristics using their color and local features. Especially, we detect the interest points of the images in the class, construct their feature descriptors using SURF and perform statistical analysis to select their representative points. Similar representative points are merged for fast comparison. Finally, we construct an index on the representative points using k-d tree. To identify the landmark in a user query image, we extract its SURF features and search for them in the index. Most similar matches are returned, along with descriptive text and GPS information. We implemented a prototype system based on a client-server architecture and performed various experiments to demonstrate that our scheme can achieve reasonable precision and scalability and provide a new browsing experience to the user.

AB - In this paper, we propose a classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition using the local features, GPS data and user tags of images. For spatio-temporal landmark image classification, we first divide Earth's entire surface into unit grid cells and collect pictures taken in each cell through Flickr. The collected images contain information such as location, titles and other user tags. Usually, the titles or user tags of landmark images include landmark names. Hence, by analyzing such tags, we can identify promising landmark names in the region and create a collection of images for each landmark using Flickr API. Even though each landmark class contains images of the same landmark, their spatio-temporal features could be different depending on shooting time, distance or angle. Therefore, we further divide the images in each landmark class into several subclasses according to their spatio-temporal characteristics using their color and local features. Especially, we detect the interest points of the images in the class, construct their feature descriptors using SURF and perform statistical analysis to select their representative points. Similar representative points are merged for fast comparison. Finally, we construct an index on the representative points using k-d tree. To identify the landmark in a user query image, we extract its SURF features and search for them in the index. Most similar matches are returned, along with descriptive text and GPS information. We implemented a prototype system based on a client-server architecture and performed various experiments to demonstrate that our scheme can achieve reasonable precision and scalability and provide a new browsing experience to the user.

KW - Landmark

KW - local feature descriptor

KW - object recognition

KW - spatio-temporal

KW - user-aware

UR - http://www.scopus.com/inward/record.url?scp=84925714082&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84925714082&partnerID=8YFLogxK

U2 - 10.3233/ICA-140478

DO - 10.3233/ICA-140478

M3 - Article

VL - 22

SP - 201

EP - 213

JO - Integrated Computer-Aided Engineering

JF - Integrated Computer-Aided Engineering

SN - 1069-2509

IS - 2

ER -