From small-scale to large-scale text classification

Kang Min Kim, Yeachan Kim, Jungho Lee, Ji Min Lee, Sang-Geun Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Neural network models have achieved impressive results in the field of text classification. However, existing approaches often suffer from insufficient training data in a large-scale text classification involving a large number of categories (e.g., several thousands of categories). Several neural network models have utilized multi-task learning to overcome the limited amount of training data. However, these approaches are also limited to small-scale text classification. In this paper, we propose a novel neural network-based multi-task learning framework for large-scale text classification. To this end, we first treat the different scales of text classification (i.e., large and small numbers of categories) as multiple, related tasks. Then, we train the proposed neural network, which learns small- and large-scale text classification tasks simultaneously. In particular, we further enhance this multi-task learning architecture by using a gate mechanism, which controls the flow of features between the small- and large-scale text classification tasks. Experimental results clearly show that our proposed model improves the performance of the large-scale text classification task with the help of the small-scale text classification task. The proposed scheme exhibits significant improvements of as much as 14% and 5% in terms of micro-averaging and macro-averaging F1-score, respectively, over state-of-the-art techniques.

Original languageEnglish
Title of host publicationThe Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019
PublisherAssociation for Computing Machinery, Inc
Pages853-862
Number of pages10
ISBN (Electronic)9781450366748
DOIs
Publication statusPublished - 2019 May 13
Event2019 World Wide Web Conference, WWW 2019 - San Francisco, United States
Duration: 2019 May 132019 May 17

Publication series

NameThe Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019

Conference

Conference2019 World Wide Web Conference, WWW 2019
CountryUnited States
CitySan Francisco
Period19/5/1319/5/17

Fingerprint

Neural networks
Macros

Keywords

  • Deep Neural Networks
  • Large-scale Text Classification
  • Multi-task Learning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this

Kim, K. M., Kim, Y., Lee, J., Lee, J. M., & Lee, S-G. (2019). From small-scale to large-scale text classification. In The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019 (pp. 853-862). (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019). Association for Computing Machinery, Inc. https://doi.org/10.1145/3308558.3313563

From small-scale to large-scale text classification. / Kim, Kang Min; Kim, Yeachan; Lee, Jungho; Lee, Ji Min; Lee, Sang-Geun.

The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. Association for Computing Machinery, Inc, 2019. p. 853-862 (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, KM, Kim, Y, Lee, J, Lee, JM & Lee, S-G 2019, From small-scale to large-scale text classification. in The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019, Association for Computing Machinery, Inc, pp. 853-862, 2019 World Wide Web Conference, WWW 2019, San Francisco, United States, 19/5/13. https://doi.org/10.1145/3308558.3313563
Kim KM, Kim Y, Lee J, Lee JM, Lee S-G. From small-scale to large-scale text classification. In The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. Association for Computing Machinery, Inc. 2019. p. 853-862. (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019). https://doi.org/10.1145/3308558.3313563
Kim, Kang Min ; Kim, Yeachan ; Lee, Jungho ; Lee, Ji Min ; Lee, Sang-Geun. / From small-scale to large-scale text classification. The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. Association for Computing Machinery, Inc, 2019. pp. 853-862 (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019).
@inproceedings{250b389c58c24d36841528e2ad2e45a4,
title = "From small-scale to large-scale text classification",
abstract = "Neural network models have achieved impressive results in the field of text classification. However, existing approaches often suffer from insufficient training data in a large-scale text classification involving a large number of categories (e.g., several thousands of categories). Several neural network models have utilized multi-task learning to overcome the limited amount of training data. However, these approaches are also limited to small-scale text classification. In this paper, we propose a novel neural network-based multi-task learning framework for large-scale text classification. To this end, we first treat the different scales of text classification (i.e., large and small numbers of categories) as multiple, related tasks. Then, we train the proposed neural network, which learns small- and large-scale text classification tasks simultaneously. In particular, we further enhance this multi-task learning architecture by using a gate mechanism, which controls the flow of features between the small- and large-scale text classification tasks. Experimental results clearly show that our proposed model improves the performance of the large-scale text classification task with the help of the small-scale text classification task. The proposed scheme exhibits significant improvements of as much as 14{\%} and 5{\%} in terms of micro-averaging and macro-averaging F1-score, respectively, over state-of-the-art techniques.",
keywords = "Deep Neural Networks, Large-scale Text Classification, Multi-task Learning",
author = "Kim, {Kang Min} and Yeachan Kim and Jungho Lee and Lee, {Ji Min} and Sang-Geun Lee",
year = "2019",
month = "5",
day = "13",
doi = "10.1145/3308558.3313563",
language = "English",
series = "The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019",
publisher = "Association for Computing Machinery, Inc",
pages = "853--862",
booktitle = "The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019",

}

TY - GEN

T1 - From small-scale to large-scale text classification

AU - Kim, Kang Min

AU - Kim, Yeachan

AU - Lee, Jungho

AU - Lee, Ji Min

AU - Lee, Sang-Geun

PY - 2019/5/13

Y1 - 2019/5/13

N2 - Neural network models have achieved impressive results in the field of text classification. However, existing approaches often suffer from insufficient training data in a large-scale text classification involving a large number of categories (e.g., several thousands of categories). Several neural network models have utilized multi-task learning to overcome the limited amount of training data. However, these approaches are also limited to small-scale text classification. In this paper, we propose a novel neural network-based multi-task learning framework for large-scale text classification. To this end, we first treat the different scales of text classification (i.e., large and small numbers of categories) as multiple, related tasks. Then, we train the proposed neural network, which learns small- and large-scale text classification tasks simultaneously. In particular, we further enhance this multi-task learning architecture by using a gate mechanism, which controls the flow of features between the small- and large-scale text classification tasks. Experimental results clearly show that our proposed model improves the performance of the large-scale text classification task with the help of the small-scale text classification task. The proposed scheme exhibits significant improvements of as much as 14% and 5% in terms of micro-averaging and macro-averaging F1-score, respectively, over state-of-the-art techniques.

AB - Neural network models have achieved impressive results in the field of text classification. However, existing approaches often suffer from insufficient training data in a large-scale text classification involving a large number of categories (e.g., several thousands of categories). Several neural network models have utilized multi-task learning to overcome the limited amount of training data. However, these approaches are also limited to small-scale text classification. In this paper, we propose a novel neural network-based multi-task learning framework for large-scale text classification. To this end, we first treat the different scales of text classification (i.e., large and small numbers of categories) as multiple, related tasks. Then, we train the proposed neural network, which learns small- and large-scale text classification tasks simultaneously. In particular, we further enhance this multi-task learning architecture by using a gate mechanism, which controls the flow of features between the small- and large-scale text classification tasks. Experimental results clearly show that our proposed model improves the performance of the large-scale text classification task with the help of the small-scale text classification task. The proposed scheme exhibits significant improvements of as much as 14% and 5% in terms of micro-averaging and macro-averaging F1-score, respectively, over state-of-the-art techniques.

KW - Deep Neural Networks

KW - Large-scale Text Classification

KW - Multi-task Learning

UR - http://www.scopus.com/inward/record.url?scp=85066910847&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066910847&partnerID=8YFLogxK

U2 - 10.1145/3308558.3313563

DO - 10.1145/3308558.3313563

M3 - Conference contribution

T3 - The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019

SP - 853

EP - 862

BT - The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019

PB - Association for Computing Machinery, Inc

ER -