Enhanced representation and multi-task learning for image annotation

Alexander Binder, Wojciech Samek, Klaus Muller, Motoaki Kawanabe

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

In this paper we propose a novel biased random sampling strategy for image representation in Bag-of-Words models. We evaluate its impact on the feature properties and the ranking quality for a set of semantic concepts and show that it improves performance of classifiers in image annotation tasks and increases the correlation between kernels and labels. As second contribution we propose a method called Output Kernel Multi-Task Learning (MTL) to improve ranking performance by transfer information between classes. The main advantages of output kernel MTL are that it permits asymmetric information transfer between tasks and scales to training sets of several thousand images. We give a theoretical interpretation of the method and show that the learned contributions of source tasks to target tasks are semantically consistent. Both strategies are evaluated on the ImageCLEF PhotoAnnotation dataset. Our best visual result which used the MTL method was ranked first according to mean Average Precision (mAP) within the purely visual submissions in the ImageCLEF 2011 PhotoAnnotation Challenge. Our multi-modal submission achieved the first rank by mAP among all submissions in the same competition.

Original languageEnglish
Pages (from-to)466-478
Number of pages13
JournalComputer Vision and Image Understanding
Volume117
Issue number5
DOIs
Publication statusPublished - 2013 Jan 14

Fingerprint

Labels
Classifiers
Semantics
Sampling

Keywords

  • Bag-of-Words representation
  • Biased random sampling
  • Image classification
  • Image ranking
  • ImageCLEF
  • Multi task learning
  • Multiple kernel learning
  • Mutual information

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Enhanced representation and multi-task learning for image annotation. / Binder, Alexander; Samek, Wojciech; Muller, Klaus; Kawanabe, Motoaki.

In: Computer Vision and Image Understanding, Vol. 117, No. 5, 14.01.2013, p. 466-478.

Research output: Contribution to journalArticle

Binder, Alexander ; Samek, Wojciech ; Muller, Klaus ; Kawanabe, Motoaki. / Enhanced representation and multi-task learning for image annotation. In: Computer Vision and Image Understanding. 2013 ; Vol. 117, No. 5. pp. 466-478.
@article{449507c0d33649058d70b46a271054c8,
title = "Enhanced representation and multi-task learning for image annotation",
abstract = "In this paper we propose a novel biased random sampling strategy for image representation in Bag-of-Words models. We evaluate its impact on the feature properties and the ranking quality for a set of semantic concepts and show that it improves performance of classifiers in image annotation tasks and increases the correlation between kernels and labels. As second contribution we propose a method called Output Kernel Multi-Task Learning (MTL) to improve ranking performance by transfer information between classes. The main advantages of output kernel MTL are that it permits asymmetric information transfer between tasks and scales to training sets of several thousand images. We give a theoretical interpretation of the method and show that the learned contributions of source tasks to target tasks are semantically consistent. Both strategies are evaluated on the ImageCLEF PhotoAnnotation dataset. Our best visual result which used the MTL method was ranked first according to mean Average Precision (mAP) within the purely visual submissions in the ImageCLEF 2011 PhotoAnnotation Challenge. Our multi-modal submission achieved the first rank by mAP among all submissions in the same competition.",
keywords = "Bag-of-Words representation, Biased random sampling, Image classification, Image ranking, ImageCLEF, Multi task learning, Multiple kernel learning, Mutual information",
author = "Alexander Binder and Wojciech Samek and Klaus Muller and Motoaki Kawanabe",
year = "2013",
month = "1",
day = "14",
doi = "10.1016/j.cviu.2012.09.006",
language = "English",
volume = "117",
pages = "466--478",
journal = "Computer Vision and Image Understanding",
issn = "1077-3142",
publisher = "Academic Press Inc.",
number = "5",

}

TY - JOUR

T1 - Enhanced representation and multi-task learning for image annotation

AU - Binder, Alexander

AU - Samek, Wojciech

AU - Muller, Klaus

AU - Kawanabe, Motoaki

PY - 2013/1/14

Y1 - 2013/1/14

N2 - In this paper we propose a novel biased random sampling strategy for image representation in Bag-of-Words models. We evaluate its impact on the feature properties and the ranking quality for a set of semantic concepts and show that it improves performance of classifiers in image annotation tasks and increases the correlation between kernels and labels. As second contribution we propose a method called Output Kernel Multi-Task Learning (MTL) to improve ranking performance by transfer information between classes. The main advantages of output kernel MTL are that it permits asymmetric information transfer between tasks and scales to training sets of several thousand images. We give a theoretical interpretation of the method and show that the learned contributions of source tasks to target tasks are semantically consistent. Both strategies are evaluated on the ImageCLEF PhotoAnnotation dataset. Our best visual result which used the MTL method was ranked first according to mean Average Precision (mAP) within the purely visual submissions in the ImageCLEF 2011 PhotoAnnotation Challenge. Our multi-modal submission achieved the first rank by mAP among all submissions in the same competition.

AB - In this paper we propose a novel biased random sampling strategy for image representation in Bag-of-Words models. We evaluate its impact on the feature properties and the ranking quality for a set of semantic concepts and show that it improves performance of classifiers in image annotation tasks and increases the correlation between kernels and labels. As second contribution we propose a method called Output Kernel Multi-Task Learning (MTL) to improve ranking performance by transfer information between classes. The main advantages of output kernel MTL are that it permits asymmetric information transfer between tasks and scales to training sets of several thousand images. We give a theoretical interpretation of the method and show that the learned contributions of source tasks to target tasks are semantically consistent. Both strategies are evaluated on the ImageCLEF PhotoAnnotation dataset. Our best visual result which used the MTL method was ranked first according to mean Average Precision (mAP) within the purely visual submissions in the ImageCLEF 2011 PhotoAnnotation Challenge. Our multi-modal submission achieved the first rank by mAP among all submissions in the same competition.

KW - Bag-of-Words representation

KW - Biased random sampling

KW - Image classification

KW - Image ranking

KW - ImageCLEF

KW - Multi task learning

KW - Multiple kernel learning

KW - Mutual information

UR - http://www.scopus.com/inward/record.url?scp=84875913890&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84875913890&partnerID=8YFLogxK

U2 - 10.1016/j.cviu.2012.09.006

DO - 10.1016/j.cviu.2012.09.006

M3 - Article

AN - SCOPUS:84875913890

VL - 117

SP - 466

EP - 478

JO - Computer Vision and Image Understanding

JF - Computer Vision and Image Understanding

SN - 1077-3142

IS - 5

ER -