A Unified Approach to Word Sense Representation and Disambiguation

Do Myoung Lee, Yeachan Kim, Ji Min Lee, Sang-Geun Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The lexical ambiguity of words has been successfully clarified by representing words at a sense level instead of a word level. This is known as word sense representation (WSR). However, WSR models are typically trained in an unsupervised fashion without any guidance from sense inventories. Therefore, the number of sense vectors assigned to a word varies from model to model. This implies that some senses are missed or unnecessarily added. Moreover, to utilize their sense vectors in natural language processing tasks, we must determine which sense of a word to choose. In this paper, we introduce a unified neural model that incorporates WSR into word sense disambiguation (WSD), thereby leveraging the sense inventory. We use bidirectional long short-term memory networks to capture the sequential information of contexts effectively. To overcome the limitation of size with the labeled dataset, we train our model in a semi-supervised fashion to scale up the size of the dataset by leveraging a large-scale unlabeled dataset. We evaluate our proposed model on both WSR and WSD tasks. The experimental results demonstrate that our model outperforms state-of-the-art on WSR task by 0.27%, while, on WSD task, by 1.4% in terms of Spearman's correlation and F'l-score, respectively.

Original languageEnglish
Title of host publicationProceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018
EditorsNewton Howard, Sam Kwong, Yingxu Wang, Jerome Feldman, Bernard Widrow, Phillip Sheu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages330-336
Number of pages7
ISBN (Electronic)9781538633601
DOIs
Publication statusPublished - 2018 Oct 4
Event17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018 - Berkeley, United States
Duration: 2018 Jul 162018 Jul 18

Other

Other17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018
CountryUnited States
CityBerkeley
Period18/7/1618/7/18

Fingerprint

Natural Language Processing
Equipment and Supplies
Long-Term Memory
Short-Term Memory
Datasets
Processing

Keywords

  • Artificial neural nets
  • Computational Intelligence
  • Natural language processing
  • Recurrent neural networks

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems
  • Cognitive Neuroscience

Cite this

Lee, D. M., Kim, Y., Lee, J. M., & Lee, S-G. (2018). A Unified Approach to Word Sense Representation and Disambiguation. In N. Howard, S. Kwong, Y. Wang, J. Feldman, B. Widrow, & P. Sheu (Eds.), Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018 (pp. 330-336). [8482041] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCI-CC.2018.8482041

A Unified Approach to Word Sense Representation and Disambiguation. / Lee, Do Myoung; Kim, Yeachan; Lee, Ji Min; Lee, Sang-Geun.

Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018. ed. / Newton Howard; Sam Kwong; Yingxu Wang; Jerome Feldman; Bernard Widrow; Phillip Sheu. Institute of Electrical and Electronics Engineers Inc., 2018. p. 330-336 8482041.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Lee, DM, Kim, Y, Lee, JM & Lee, S-G 2018, A Unified Approach to Word Sense Representation and Disambiguation. in N Howard, S Kwong, Y Wang, J Feldman, B Widrow & P Sheu (eds), Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018., 8482041, Institute of Electrical and Electronics Engineers Inc., pp. 330-336, 17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018, Berkeley, United States, 18/7/16. https://doi.org/10.1109/ICCI-CC.2018.8482041
Lee DM, Kim Y, Lee JM, Lee S-G. A Unified Approach to Word Sense Representation and Disambiguation. In Howard N, Kwong S, Wang Y, Feldman J, Widrow B, Sheu P, editors, Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 330-336. 8482041 https://doi.org/10.1109/ICCI-CC.2018.8482041
Lee, Do Myoung ; Kim, Yeachan ; Lee, Ji Min ; Lee, Sang-Geun. / A Unified Approach to Word Sense Representation and Disambiguation. Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018. editor / Newton Howard ; Sam Kwong ; Yingxu Wang ; Jerome Feldman ; Bernard Widrow ; Phillip Sheu. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 330-336
@inproceedings{ae15fa67ed664fefb58140e1262cb98f,
title = "A Unified Approach to Word Sense Representation and Disambiguation",
abstract = "The lexical ambiguity of words has been successfully clarified by representing words at a sense level instead of a word level. This is known as word sense representation (WSR). However, WSR models are typically trained in an unsupervised fashion without any guidance from sense inventories. Therefore, the number of sense vectors assigned to a word varies from model to model. This implies that some senses are missed or unnecessarily added. Moreover, to utilize their sense vectors in natural language processing tasks, we must determine which sense of a word to choose. In this paper, we introduce a unified neural model that incorporates WSR into word sense disambiguation (WSD), thereby leveraging the sense inventory. We use bidirectional long short-term memory networks to capture the sequential information of contexts effectively. To overcome the limitation of size with the labeled dataset, we train our model in a semi-supervised fashion to scale up the size of the dataset by leveraging a large-scale unlabeled dataset. We evaluate our proposed model on both WSR and WSD tasks. The experimental results demonstrate that our model outperforms state-of-the-art on WSR task by 0.27{\%}, while, on WSD task, by 1.4{\%} in terms of Spearman's correlation and F'l-score, respectively.",
keywords = "Artificial neural nets, Computational Intelligence, Natural language processing, Recurrent neural networks",
author = "Lee, {Do Myoung} and Yeachan Kim and Lee, {Ji Min} and Sang-Geun Lee",
year = "2018",
month = "10",
day = "4",
doi = "10.1109/ICCI-CC.2018.8482041",
language = "English",
pages = "330--336",
editor = "Newton Howard and Sam Kwong and Yingxu Wang and Jerome Feldman and Bernard Widrow and Phillip Sheu",
booktitle = "Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - A Unified Approach to Word Sense Representation and Disambiguation

AU - Lee, Do Myoung

AU - Kim, Yeachan

AU - Lee, Ji Min

AU - Lee, Sang-Geun

PY - 2018/10/4

Y1 - 2018/10/4

N2 - The lexical ambiguity of words has been successfully clarified by representing words at a sense level instead of a word level. This is known as word sense representation (WSR). However, WSR models are typically trained in an unsupervised fashion without any guidance from sense inventories. Therefore, the number of sense vectors assigned to a word varies from model to model. This implies that some senses are missed or unnecessarily added. Moreover, to utilize their sense vectors in natural language processing tasks, we must determine which sense of a word to choose. In this paper, we introduce a unified neural model that incorporates WSR into word sense disambiguation (WSD), thereby leveraging the sense inventory. We use bidirectional long short-term memory networks to capture the sequential information of contexts effectively. To overcome the limitation of size with the labeled dataset, we train our model in a semi-supervised fashion to scale up the size of the dataset by leveraging a large-scale unlabeled dataset. We evaluate our proposed model on both WSR and WSD tasks. The experimental results demonstrate that our model outperforms state-of-the-art on WSR task by 0.27%, while, on WSD task, by 1.4% in terms of Spearman's correlation and F'l-score, respectively.

AB - The lexical ambiguity of words has been successfully clarified by representing words at a sense level instead of a word level. This is known as word sense representation (WSR). However, WSR models are typically trained in an unsupervised fashion without any guidance from sense inventories. Therefore, the number of sense vectors assigned to a word varies from model to model. This implies that some senses are missed or unnecessarily added. Moreover, to utilize their sense vectors in natural language processing tasks, we must determine which sense of a word to choose. In this paper, we introduce a unified neural model that incorporates WSR into word sense disambiguation (WSD), thereby leveraging the sense inventory. We use bidirectional long short-term memory networks to capture the sequential information of contexts effectively. To overcome the limitation of size with the labeled dataset, we train our model in a semi-supervised fashion to scale up the size of the dataset by leveraging a large-scale unlabeled dataset. We evaluate our proposed model on both WSR and WSD tasks. The experimental results demonstrate that our model outperforms state-of-the-art on WSR task by 0.27%, while, on WSD task, by 1.4% in terms of Spearman's correlation and F'l-score, respectively.

KW - Artificial neural nets

KW - Computational Intelligence

KW - Natural language processing

KW - Recurrent neural networks

UR - http://www.scopus.com/inward/record.url?scp=85056479283&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056479283&partnerID=8YFLogxK

U2 - 10.1109/ICCI-CC.2018.8482041

DO - 10.1109/ICCI-CC.2018.8482041

M3 - Conference contribution

AN - SCOPUS:85056479283

SP - 330

EP - 336

BT - Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018

A2 - Howard, Newton

A2 - Kwong, Sam

A2 - Wang, Yingxu

A2 - Feldman, Jerome

A2 - Widrow, Bernard

A2 - Sheu, Phillip

PB - Institute of Electrical and Electronics Engineers Inc.

ER -