CAPTCHA-based image annotation

Shinil Kwon, Sungdeok Cha

Research output: Contribution to journalArticle

Abstract

Although crowdsourcing-based image annotation services are accurate, they easily become too costly to assign proper labels on all the images on the Internet. In this paper, we propose a practical, efficient, and accurate CAPTCHA-based image annotation technique that requires no or little additional cost. If challenge sessions can be conducted such that bots are successfully defeated and only humans are likely to pass, inclusion of unlabeled images into the challenges and analysis of successful responses offer benefit equivalent to obtaining free and motivated crowdsourcing services. We conducted an experiment using 25 individuals in order to evaluate the effectiveness of the approach. Results are highly positive except that some participants occasionally made, as expected mistakes. In order to further improve accuracy of image labeling, crosschecking mechanism was introduced to effectively eliminate the risk of potential human errors.

Original languageEnglish
Pages (from-to)27-31
Number of pages5
JournalInformation Processing Letters
Volume128
DOIs
Publication statusPublished - 2017 Dec 1

Fingerprint

Image Annotation
Labeling
Labels
Internet
Costs
Human Error
Experiments
Assign
Eliminate
Inclusion
Likely
Evaluate
Experiment

Keywords

  • CAPTCHA
  • Crowdsourcing
  • Image annotation
  • Safety/security in digital systems

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Signal Processing
  • Information Systems
  • Computer Science Applications

Cite this

CAPTCHA-based image annotation. / Kwon, Shinil; Cha, Sungdeok.

In: Information Processing Letters, Vol. 128, 01.12.2017, p. 27-31.

Research output: Contribution to journalArticle

Kwon, Shinil ; Cha, Sungdeok. / CAPTCHA-based image annotation. In: Information Processing Letters. 2017 ; Vol. 128. pp. 27-31.
@article{87a8f8cf4a6742cd84970b22b304151b,
title = "CAPTCHA-based image annotation",
abstract = "Although crowdsourcing-based image annotation services are accurate, they easily become too costly to assign proper labels on all the images on the Internet. In this paper, we propose a practical, efficient, and accurate CAPTCHA-based image annotation technique that requires no or little additional cost. If challenge sessions can be conducted such that bots are successfully defeated and only humans are likely to pass, inclusion of unlabeled images into the challenges and analysis of successful responses offer benefit equivalent to obtaining free and motivated crowdsourcing services. We conducted an experiment using 25 individuals in order to evaluate the effectiveness of the approach. Results are highly positive except that some participants occasionally made, as expected mistakes. In order to further improve accuracy of image labeling, crosschecking mechanism was introduced to effectively eliminate the risk of potential human errors.",
keywords = "CAPTCHA, Crowdsourcing, Image annotation, Safety/security in digital systems",
author = "Shinil Kwon and Sungdeok Cha",
year = "2017",
month = "12",
day = "1",
doi = "10.1016/j.ipl.2017.07.009",
language = "English",
volume = "128",
pages = "27--31",
journal = "Information Processing Letters",
issn = "0020-0190",
publisher = "Elsevier",

}

TY - JOUR

T1 - CAPTCHA-based image annotation

AU - Kwon, Shinil

AU - Cha, Sungdeok

PY - 2017/12/1

Y1 - 2017/12/1

N2 - Although crowdsourcing-based image annotation services are accurate, they easily become too costly to assign proper labels on all the images on the Internet. In this paper, we propose a practical, efficient, and accurate CAPTCHA-based image annotation technique that requires no or little additional cost. If challenge sessions can be conducted such that bots are successfully defeated and only humans are likely to pass, inclusion of unlabeled images into the challenges and analysis of successful responses offer benefit equivalent to obtaining free and motivated crowdsourcing services. We conducted an experiment using 25 individuals in order to evaluate the effectiveness of the approach. Results are highly positive except that some participants occasionally made, as expected mistakes. In order to further improve accuracy of image labeling, crosschecking mechanism was introduced to effectively eliminate the risk of potential human errors.

AB - Although crowdsourcing-based image annotation services are accurate, they easily become too costly to assign proper labels on all the images on the Internet. In this paper, we propose a practical, efficient, and accurate CAPTCHA-based image annotation technique that requires no or little additional cost. If challenge sessions can be conducted such that bots are successfully defeated and only humans are likely to pass, inclusion of unlabeled images into the challenges and analysis of successful responses offer benefit equivalent to obtaining free and motivated crowdsourcing services. We conducted an experiment using 25 individuals in order to evaluate the effectiveness of the approach. Results are highly positive except that some participants occasionally made, as expected mistakes. In order to further improve accuracy of image labeling, crosschecking mechanism was introduced to effectively eliminate the risk of potential human errors.

KW - CAPTCHA

KW - Crowdsourcing

KW - Image annotation

KW - Safety/security in digital systems

UR - http://www.scopus.com/inward/record.url?scp=85026779259&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85026779259&partnerID=8YFLogxK

U2 - 10.1016/j.ipl.2017.07.009

DO - 10.1016/j.ipl.2017.07.009

M3 - Article

VL - 128

SP - 27

EP - 31

JO - Information Processing Letters

JF - Information Processing Letters

SN - 0020-0190

ER -