AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks

Minsuk Choi, Cheonbok Park, Soyoung Yang, Yonggyu Kim, Jaegul Choo, Sungsoo Hong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Document labeling is a critical step in building various machine learning applications. However, the step can be time-consuming and arduous, requiring a significant amount of human effort. To support an efficient document labeling environment, we present a system called Attentive Interactive Labeling Assistant (AILA). At its core, AILA uses Interactive Attention Module (IAM), a novel module that visually highlights words in a document that labelers may pay attention to when labeling a document. IAM utilizes attention-based Deep Neural Networks, which not only support a prediction of which words to highlight, but also enable labelers to indicate words that should be assigned high attention weights while labeling to improve the future quality of word prediction. We evaluated the labeling efficiency and accuracy by comparing the conditions with and without IAM in our study. The results showed that the participants’ labeling efficiency increased significantly under the condition with IAM than under the condition without IAM, while the two conditions maintained roughly the same labeling accuracy.

Original languageEnglish
Title of host publicationCHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450359702
DOIs
Publication statusPublished - 2019 May 2
Event2019 CHI Conference on Human Factors in Computing Systems, CHI 2019 - Glasgow, United Kingdom
Duration: 2019 May 42019 May 9

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2019 CHI Conference on Human Factors in Computing Systems, CHI 2019
CountryUnited Kingdom
CityGlasgow
Period19/5/419/5/9

Fingerprint

Labeling
Deep neural networks
Learning systems

Keywords

  • Attention model
  • Deep neural networks
  • Document classification
  • Document labeling
  • Natural language processing

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Choi, M., Park, C., Yang, S., Kim, Y., Choo, J., & Hong, S. (2019). AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks. In CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3290605.3300460

AILA : Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks. / Choi, Minsuk; Park, Cheonbok; Yang, Soyoung; Kim, Yonggyu; Choo, Jaegul; Hong, Sungsoo.

CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2019. (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Choi, M, Park, C, Yang, S, Kim, Y, Choo, J & Hong, S 2019, AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks. in CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, United Kingdom, 19/5/4. https://doi.org/10.1145/3290605.3300460
Choi M, Park C, Yang S, Kim Y, Choo J, Hong S. AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks. In CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2019. (Conference on Human Factors in Computing Systems - Proceedings). https://doi.org/10.1145/3290605.3300460
Choi, Minsuk ; Park, Cheonbok ; Yang, Soyoung ; Kim, Yonggyu ; Choo, Jaegul ; Hong, Sungsoo. / AILA : Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks. CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2019. (Conference on Human Factors in Computing Systems - Proceedings).
@inproceedings{e7991ef12e984740b37ec84833c2667e,
title = "AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks",
abstract = "Document labeling is a critical step in building various machine learning applications. However, the step can be time-consuming and arduous, requiring a significant amount of human effort. To support an efficient document labeling environment, we present a system called Attentive Interactive Labeling Assistant (AILA). At its core, AILA uses Interactive Attention Module (IAM), a novel module that visually highlights words in a document that labelers may pay attention to when labeling a document. IAM utilizes attention-based Deep Neural Networks, which not only support a prediction of which words to highlight, but also enable labelers to indicate words that should be assigned high attention weights while labeling to improve the future quality of word prediction. We evaluated the labeling efficiency and accuracy by comparing the conditions with and without IAM in our study. The results showed that the participants’ labeling efficiency increased significantly under the condition with IAM than under the condition without IAM, while the two conditions maintained roughly the same labeling accuracy.",
keywords = "Attention model, Deep neural networks, Document classification, Document labeling, Natural language processing",
author = "Minsuk Choi and Cheonbok Park and Soyoung Yang and Yonggyu Kim and Jaegul Choo and Sungsoo Hong",
year = "2019",
month = "5",
day = "2",
doi = "10.1145/3290605.3300460",
language = "English",
series = "Conference on Human Factors in Computing Systems - Proceedings",
publisher = "Association for Computing Machinery",
booktitle = "CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems",

}

TY - GEN

T1 - AILA

T2 - Attentive Interactive Labeling Assistant for Document Classification through Attention-based Deep Neural Networks

AU - Choi, Minsuk

AU - Park, Cheonbok

AU - Yang, Soyoung

AU - Kim, Yonggyu

AU - Choo, Jaegul

AU - Hong, Sungsoo

PY - 2019/5/2

Y1 - 2019/5/2

N2 - Document labeling is a critical step in building various machine learning applications. However, the step can be time-consuming and arduous, requiring a significant amount of human effort. To support an efficient document labeling environment, we present a system called Attentive Interactive Labeling Assistant (AILA). At its core, AILA uses Interactive Attention Module (IAM), a novel module that visually highlights words in a document that labelers may pay attention to when labeling a document. IAM utilizes attention-based Deep Neural Networks, which not only support a prediction of which words to highlight, but also enable labelers to indicate words that should be assigned high attention weights while labeling to improve the future quality of word prediction. We evaluated the labeling efficiency and accuracy by comparing the conditions with and without IAM in our study. The results showed that the participants’ labeling efficiency increased significantly under the condition with IAM than under the condition without IAM, while the two conditions maintained roughly the same labeling accuracy.

AB - Document labeling is a critical step in building various machine learning applications. However, the step can be time-consuming and arduous, requiring a significant amount of human effort. To support an efficient document labeling environment, we present a system called Attentive Interactive Labeling Assistant (AILA). At its core, AILA uses Interactive Attention Module (IAM), a novel module that visually highlights words in a document that labelers may pay attention to when labeling a document. IAM utilizes attention-based Deep Neural Networks, which not only support a prediction of which words to highlight, but also enable labelers to indicate words that should be assigned high attention weights while labeling to improve the future quality of word prediction. We evaluated the labeling efficiency and accuracy by comparing the conditions with and without IAM in our study. The results showed that the participants’ labeling efficiency increased significantly under the condition with IAM than under the condition without IAM, while the two conditions maintained roughly the same labeling accuracy.

KW - Attention model

KW - Deep neural networks

KW - Document classification

KW - Document labeling

KW - Natural language processing

UR - http://www.scopus.com/inward/record.url?scp=85067603145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067603145&partnerID=8YFLogxK

U2 - 10.1145/3290605.3300460

DO - 10.1145/3290605.3300460

M3 - Conference contribution

AN - SCOPUS:85067603145

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

ER -