Enhanced representation and multi-task learning for image annotation

Alexander Binder, Wojciech Samek, Klaus Robert Müller, Motoaki Kawanabe

Research output: Contribution to journalArticlepeer-review

13 Citations (Scopus)


In this paper we propose a novel biased random sampling strategy for image representation in Bag-of-Words models. We evaluate its impact on the feature properties and the ranking quality for a set of semantic concepts and show that it improves performance of classifiers in image annotation tasks and increases the correlation between kernels and labels. As second contribution we propose a method called Output Kernel Multi-Task Learning (MTL) to improve ranking performance by transfer information between classes. The main advantages of output kernel MTL are that it permits asymmetric information transfer between tasks and scales to training sets of several thousand images. We give a theoretical interpretation of the method and show that the learned contributions of source tasks to target tasks are semantically consistent. Both strategies are evaluated on the ImageCLEF PhotoAnnotation dataset. Our best visual result which used the MTL method was ranked first according to mean Average Precision (mAP) within the purely visual submissions in the ImageCLEF 2011 PhotoAnnotation Challenge. Our multi-modal submission achieved the first rank by mAP among all submissions in the same competition.

Original languageEnglish
Pages (from-to)466-478
Number of pages13
JournalComputer Vision and Image Understanding
Issue number5
Publication statusPublished - 2013


  • Bag-of-Words representation
  • Biased random sampling
  • Image classification
  • Image ranking
  • ImageCLEF
  • Multi task learning
  • Multiple kernel learning
  • Mutual information

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Enhanced representation and multi-task learning for image annotation'. Together they form a unique fingerprint.

Cite this