Predicting unseen labels using label hierarchies in large-scale multi-label learning

Jinseok Nam, Eneldo Loza Mencía, Hyun Woo Kim, Johannes Fürnkranz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. One way of learning underlying structures over labels is to project both instances and labels into the same space where an instance and its relevant labels tend to have similar representations. In this paper, we present a novel method to learn a joint space of instances and labels by leveraging a hierarchy of labels. We also present an efficient method for pretraining vector representations of labels, namely label embeddings, from large amounts of label co-occurrence patterns and hierarchical structures of labels. This approach also allows us to make predictions on labels that have not been seen during training. We empirically show that the use of pretrained label embeddings allows us to obtain higher accuracies on unseen labels even when the number of labels are quite large. Our experimental results also demonstrate qualitatively that the proposed method is able to learn regularities among labels by exploiting a label hierarchy as well as label co-occurrences.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings
EditorsAnnalisa Appice, João Gama, Vitor Santos Costa, João Gama, Alípio Jorge, Annalisa Appice, Annalisa Appice, Vitor Santos Costa, Alípio Jorge, Annalisa Appice, Pedro Pereira Rodrigues, Pedro Pereira Rodrigues, João Gama, Vitor Santos Costa, Soares Soares, Pedro Pereira Rodrigues, Soares Soares, Soares Soares, João Gama, Soares Soares, Alípio Jorge, Alípio Jorge, Pedro Pereira Rodrigues, Vitor Santos Costa
PublisherSpringer Verlag
Pages102-118
Number of pages17
ISBN (Print)9783319235271, 9783319235271, 9783319235271, 9783319235271
DOIs
Publication statusPublished - 2015 Jan 1
Externally publishedYes
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015 - Porto, Portugal
Duration: 2015 Sep 72015 Sep 11

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9284
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015
CountryPortugal
CityPorto
Period15/9/715/9/11

Fingerprint

Labels
Structure Learning
Hierarchical Structure
High Accuracy
Regularity
Tend
Prediction
Experimental Results
Demonstrate
Learning
Hierarchy
Training

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Nam, J., Mencía, E. L., Kim, H. W., & Fürnkranz, J. (2015). Predicting unseen labels using label hierarchies in large-scale multi-label learning. In A. Appice, J. Gama, V. S. Costa, J. Gama, A. Jorge, A. Appice, A. Appice, V. S. Costa, A. Jorge, A. Appice, P. P. Rodrigues, P. P. Rodrigues, J. Gama, V. S. Costa, S. Soares, P. P. Rodrigues, S. Soares, S. Soares, J. Gama, S. Soares, A. Jorge, A. Jorge, P. P. Rodrigues, ... V. S. Costa (Eds.), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings (pp. 102-118). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9284). Springer Verlag. https://doi.org/10.1007/978-3-319-23528-8_7

Predicting unseen labels using label hierarchies in large-scale multi-label learning. / Nam, Jinseok; Mencía, Eneldo Loza; Kim, Hyun Woo; Fürnkranz, Johannes.

Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. ed. / Annalisa Appice; João Gama; Vitor Santos Costa; João Gama; Alípio Jorge; Annalisa Appice; Annalisa Appice; Vitor Santos Costa; Alípio Jorge; Annalisa Appice; Pedro Pereira Rodrigues; Pedro Pereira Rodrigues; João Gama; Vitor Santos Costa; Soares Soares; Pedro Pereira Rodrigues; Soares Soares; Soares Soares; João Gama; Soares Soares; Alípio Jorge; Alípio Jorge; Pedro Pereira Rodrigues; Vitor Santos Costa. Springer Verlag, 2015. p. 102-118 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9284).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nam, J, Mencía, EL, Kim, HW & Fürnkranz, J 2015, Predicting unseen labels using label hierarchies in large-scale multi-label learning. in A Appice, J Gama, VS Costa, J Gama, A Jorge, A Appice, A Appice, VS Costa, A Jorge, A Appice, PP Rodrigues, PP Rodrigues, J Gama, VS Costa, S Soares, PP Rodrigues, S Soares, S Soares, J Gama, S Soares, A Jorge, A Jorge, PP Rodrigues & VS Costa (eds), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9284, Springer Verlag, pp. 102-118, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015, Porto, Portugal, 15/9/7. https://doi.org/10.1007/978-3-319-23528-8_7
Nam J, Mencía EL, Kim HW, Fürnkranz J. Predicting unseen labels using label hierarchies in large-scale multi-label learning. In Appice A, Gama J, Costa VS, Gama J, Jorge A, Appice A, Appice A, Costa VS, Jorge A, Appice A, Rodrigues PP, Rodrigues PP, Gama J, Costa VS, Soares S, Rodrigues PP, Soares S, Soares S, Gama J, Soares S, Jorge A, Jorge A, Rodrigues PP, Costa VS, editors, Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. Springer Verlag. 2015. p. 102-118. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-23528-8_7
Nam, Jinseok ; Mencía, Eneldo Loza ; Kim, Hyun Woo ; Fürnkranz, Johannes. / Predicting unseen labels using label hierarchies in large-scale multi-label learning. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. editor / Annalisa Appice ; João Gama ; Vitor Santos Costa ; João Gama ; Alípio Jorge ; Annalisa Appice ; Annalisa Appice ; Vitor Santos Costa ; Alípio Jorge ; Annalisa Appice ; Pedro Pereira Rodrigues ; Pedro Pereira Rodrigues ; João Gama ; Vitor Santos Costa ; Soares Soares ; Pedro Pereira Rodrigues ; Soares Soares ; Soares Soares ; João Gama ; Soares Soares ; Alípio Jorge ; Alípio Jorge ; Pedro Pereira Rodrigues ; Vitor Santos Costa. Springer Verlag, 2015. pp. 102-118 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{7c53da73731f45baa51eeb8fcae64dc8,
title = "Predicting unseen labels using label hierarchies in large-scale multi-label learning",
abstract = "An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. One way of learning underlying structures over labels is to project both instances and labels into the same space where an instance and its relevant labels tend to have similar representations. In this paper, we present a novel method to learn a joint space of instances and labels by leveraging a hierarchy of labels. We also present an efficient method for pretraining vector representations of labels, namely label embeddings, from large amounts of label co-occurrence patterns and hierarchical structures of labels. This approach also allows us to make predictions on labels that have not been seen during training. We empirically show that the use of pretrained label embeddings allows us to obtain higher accuracies on unseen labels even when the number of labels are quite large. Our experimental results also demonstrate qualitatively that the proposed method is able to learn regularities among labels by exploiting a label hierarchy as well as label co-occurrences.",
author = "Jinseok Nam and Menc{\'i}a, {Eneldo Loza} and Kim, {Hyun Woo} and Johannes F{\"u}rnkranz",
year = "2015",
month = "1",
day = "1",
doi = "10.1007/978-3-319-23528-8_7",
language = "English",
isbn = "9783319235271",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "102--118",
editor = "Annalisa Appice and Jo{\~a}o Gama and Costa, {Vitor Santos} and Jo{\~a}o Gama and Al{\'i}pio Jorge and Annalisa Appice and Annalisa Appice and Costa, {Vitor Santos} and Al{\'i}pio Jorge and Annalisa Appice and Rodrigues, {Pedro Pereira} and Rodrigues, {Pedro Pereira} and Jo{\~a}o Gama and Costa, {Vitor Santos} and Soares Soares and Rodrigues, {Pedro Pereira} and Soares Soares and Soares Soares and Jo{\~a}o Gama and Soares Soares and Al{\'i}pio Jorge and Al{\'i}pio Jorge and Rodrigues, {Pedro Pereira} and Costa, {Vitor Santos}",
booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings",

}

TY - GEN

T1 - Predicting unseen labels using label hierarchies in large-scale multi-label learning

AU - Nam, Jinseok

AU - Mencía, Eneldo Loza

AU - Kim, Hyun Woo

AU - Fürnkranz, Johannes

PY - 2015/1/1

Y1 - 2015/1/1

N2 - An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. One way of learning underlying structures over labels is to project both instances and labels into the same space where an instance and its relevant labels tend to have similar representations. In this paper, we present a novel method to learn a joint space of instances and labels by leveraging a hierarchy of labels. We also present an efficient method for pretraining vector representations of labels, namely label embeddings, from large amounts of label co-occurrence patterns and hierarchical structures of labels. This approach also allows us to make predictions on labels that have not been seen during training. We empirically show that the use of pretrained label embeddings allows us to obtain higher accuracies on unseen labels even when the number of labels are quite large. Our experimental results also demonstrate qualitatively that the proposed method is able to learn regularities among labels by exploiting a label hierarchy as well as label co-occurrences.

AB - An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. One way of learning underlying structures over labels is to project both instances and labels into the same space where an instance and its relevant labels tend to have similar representations. In this paper, we present a novel method to learn a joint space of instances and labels by leveraging a hierarchy of labels. We also present an efficient method for pretraining vector representations of labels, namely label embeddings, from large amounts of label co-occurrence patterns and hierarchical structures of labels. This approach also allows us to make predictions on labels that have not been seen during training. We empirically show that the use of pretrained label embeddings allows us to obtain higher accuracies on unseen labels even when the number of labels are quite large. Our experimental results also demonstrate qualitatively that the proposed method is able to learn regularities among labels by exploiting a label hierarchy as well as label co-occurrences.

UR - http://www.scopus.com/inward/record.url?scp=84959373230&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959373230&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-23528-8_7

DO - 10.1007/978-3-319-23528-8_7

M3 - Conference contribution

AN - SCOPUS:84959373230

SN - 9783319235271

SN - 9783319235271

SN - 9783319235271

SN - 9783319235271

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 102

EP - 118

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings

A2 - Appice, Annalisa

A2 - Gama, João

A2 - Costa, Vitor Santos

A2 - Gama, João

A2 - Jorge, Alípio

A2 - Appice, Annalisa

A2 - Appice, Annalisa

A2 - Costa, Vitor Santos

A2 - Jorge, Alípio

A2 - Appice, Annalisa

A2 - Rodrigues, Pedro Pereira

A2 - Rodrigues, Pedro Pereira

A2 - Gama, João

A2 - Costa, Vitor Santos

A2 - Soares, Soares

A2 - Rodrigues, Pedro Pereira

A2 - Soares, Soares

A2 - Soares, Soares

A2 - Gama, João

A2 - Soares, Soares

A2 - Jorge, Alípio

A2 - Jorge, Alípio

A2 - Rodrigues, Pedro Pereira

A2 - Costa, Vitor Santos

PB - Springer Verlag

ER -