Supervised paragraph vector: Distributed representations of words, documents and class labels

Eunjeong L. Park, Sungzoon Cho, Pilsung Kang

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

While the traditional method of deriving representations for documents was bag-of-words, they suffered from high dimensionality and sparsity. Recently, many methods to obtain lower dimensional and densely distributed representations were proposed. Paragraph Vector is one of such algorithms, which extends the word2vec algorithm by considering the paragraph as an additional word. However, it generates a single representation for all tasks, while different tasks may require different representations. In this paper, we propose a Supervised Paragraph Vector, a task-specific variant of Paragraph Vector for situations where class labels exist. Essentially, Supervised Paragraph Vector uses class labels along with words and documents and obtains corresponding representations with respect to the particular classification task. In order to prove the benefits of the proposed algorithm, three performance criteria are used: Interpretability, discriminative power, and computational efficiency. To test interpretability, we find words that are close and far to class vectors and demonstrate that such words are closely related to the corresponding class. We also use principal component analysis to visualize all words, documents, and class labels at the same time and show that our method effectively displays the related words and documents for each class label. To evaluate discriminative power and computational efficiency, we perform document classification on four commonly used datasets with various classifiers and achieve comparable classification accuracies to bag-of-words and Paragraph Vector.

Original languageEnglish
Article number8653834
Pages (from-to)29051-29064
Number of pages14
JournalIEEE Access
Volume7
DOIs
Publication statusPublished - 2019 Jan 1

Keywords

  • Class label
  • distributed representations
  • document embedding
  • representation learning
  • word embedding

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint Dive into the research topics of 'Supervised paragraph vector: Distributed representations of words, documents and class labels'. Together they form a unique fingerprint.

  • Cite this