Multi-co-training for document classification using various document representations: TF–IDF, LDA, and Doc2Vec

Donghwa Kim, Deokseong Seo, Suhyoun Cho, Pilsung Kang

Research output: Contribution to journalArticle

38 Citations (Scopus)

Abstract

The purpose of document classification is to assign the most appropriate label to a specified document. The main challenges in document classification are insufficient label information and unstructured sparse format. A semi-supervised learning (SSL) approach could be an effective solution to the former problem, whereas the consideration of multiple document representation schemes can resolve the latter problem. Co-training is a popular SSL method that attempts to exploit various perspectives in terms of feature subsets for the same example. In this paper, we propose multi-co-training (MCT) for improving the performance of document classification. In order to increase the variety of feature sets for classification, we transform a document using three document representation methods: term frequency–inverse document frequency (TF–IDF) based on the bag-of-words scheme, topic distribution based on latent Dirichlet allocation (LDA), and neural-network-based document embedding known as document to vector (Doc2Vec). The experimental results demonstrate that the proposed MCT is robust to parameter changes and outperforms benchmark methods under various conditions.

Original languageEnglish
Pages (from-to)15-29
Number of pages15
JournalInformation Sciences
Volume477
DOIs
Publication statusPublished - 2019 Mar 1

Keywords

  • Co-training
  • Doc2vec
  • Document classification
  • LDA
  • Semi-supervised learning
  • TF–IDF

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Multi-co-training for document classification using various document representations: TF–IDF, LDA, and Doc2Vec'. Together they form a unique fingerprint.

  • Cite this