Music emotion classification and context-based music recommendation

Byeong Jun Han, Seungmin Rho, Sanghoon Jun, Een Jun Hwang

Research output: Contribution to journalArticle

72 Citations (Scopus)

Abstract

Context-based music recommendation is one of rapidly emerging applications in the advent of ubiquitous era and requires multidisciplinary efforts including low level feature extraction and music classification, human emotion description and prediction, ontology-based representation and recommendation, and the establishment of connections among them. In this paper, we contributed in three distinctive ways to take into account the idea of context awareness in the music recommendation field. Firstly, we propose a novel emotion state transition model (ESTM) to model human emotional states and their transitions by music. ESTM acts like a bridge between user situation information along with his/her emotion and low-level music features. With ESTM, we can recommend the most appropriate music to the user for transiting to the desired emotional state. Secondly, we present context-based music recommendation (COMUS) ontology for modeling user's musical preferences and context, and for supporting reasoning about the user's desired emotion and preferences. The COMUS is music-dedicated ontology in OWL constructed by incorporating domain-specific classes for music recommendation into the Music Ontology, which includes situation, mood, and musical features. Thirdly, for mapping low-level features to ESTM, we collected various high-dimensional music feature data and applied nonnegative matrix factorization (NMF) for their dimension reduction. We also used support vector machine (SVM) as emotional state transition classifier. We constructed a prototype music recommendation system based on these features and carried out various experiments to measure its performance. We report some of the experimental results.

Original languageEnglish
Pages (from-to)433-460
Number of pages28
JournalMultimedia Tools and Applications
Volume47
Issue number3
DOIs
Publication statusPublished - 2010 May 1

Fingerprint

Computer music
Ontology
Recommender systems
Factorization
Support vector machines
Feature extraction
Classifiers
Computer simulation

Keywords

  • Classification
  • Emotion
  • Emotion state transition model
  • Mood
  • Music information retrieval
  • Recommendation

ASJC Scopus subject areas

  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications
  • Software

Cite this

Music emotion classification and context-based music recommendation. / Han, Byeong Jun; Rho, Seungmin; Jun, Sanghoon; Hwang, Een Jun.

In: Multimedia Tools and Applications, Vol. 47, No. 3, 01.05.2010, p. 433-460.

Research output: Contribution to journalArticle

Han, Byeong Jun ; Rho, Seungmin ; Jun, Sanghoon ; Hwang, Een Jun. / Music emotion classification and context-based music recommendation. In: Multimedia Tools and Applications. 2010 ; Vol. 47, No. 3. pp. 433-460.
@article{92ffe470c0bd4710bed863e58e604dfa,
title = "Music emotion classification and context-based music recommendation",
abstract = "Context-based music recommendation is one of rapidly emerging applications in the advent of ubiquitous era and requires multidisciplinary efforts including low level feature extraction and music classification, human emotion description and prediction, ontology-based representation and recommendation, and the establishment of connections among them. In this paper, we contributed in three distinctive ways to take into account the idea of context awareness in the music recommendation field. Firstly, we propose a novel emotion state transition model (ESTM) to model human emotional states and their transitions by music. ESTM acts like a bridge between user situation information along with his/her emotion and low-level music features. With ESTM, we can recommend the most appropriate music to the user for transiting to the desired emotional state. Secondly, we present context-based music recommendation (COMUS) ontology for modeling user's musical preferences and context, and for supporting reasoning about the user's desired emotion and preferences. The COMUS is music-dedicated ontology in OWL constructed by incorporating domain-specific classes for music recommendation into the Music Ontology, which includes situation, mood, and musical features. Thirdly, for mapping low-level features to ESTM, we collected various high-dimensional music feature data and applied nonnegative matrix factorization (NMF) for their dimension reduction. We also used support vector machine (SVM) as emotional state transition classifier. We constructed a prototype music recommendation system based on these features and carried out various experiments to measure its performance. We report some of the experimental results.",
keywords = "Classification, Emotion, Emotion state transition model, Mood, Music information retrieval, Recommendation",
author = "Han, {Byeong Jun} and Seungmin Rho and Sanghoon Jun and Hwang, {Een Jun}",
year = "2010",
month = "5",
day = "1",
doi = "10.1007/s11042-009-0332-6",
language = "English",
volume = "47",
pages = "433--460",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer Netherlands",
number = "3",

}

TY - JOUR

T1 - Music emotion classification and context-based music recommendation

AU - Han, Byeong Jun

AU - Rho, Seungmin

AU - Jun, Sanghoon

AU - Hwang, Een Jun

PY - 2010/5/1

Y1 - 2010/5/1

N2 - Context-based music recommendation is one of rapidly emerging applications in the advent of ubiquitous era and requires multidisciplinary efforts including low level feature extraction and music classification, human emotion description and prediction, ontology-based representation and recommendation, and the establishment of connections among them. In this paper, we contributed in three distinctive ways to take into account the idea of context awareness in the music recommendation field. Firstly, we propose a novel emotion state transition model (ESTM) to model human emotional states and their transitions by music. ESTM acts like a bridge between user situation information along with his/her emotion and low-level music features. With ESTM, we can recommend the most appropriate music to the user for transiting to the desired emotional state. Secondly, we present context-based music recommendation (COMUS) ontology for modeling user's musical preferences and context, and for supporting reasoning about the user's desired emotion and preferences. The COMUS is music-dedicated ontology in OWL constructed by incorporating domain-specific classes for music recommendation into the Music Ontology, which includes situation, mood, and musical features. Thirdly, for mapping low-level features to ESTM, we collected various high-dimensional music feature data and applied nonnegative matrix factorization (NMF) for their dimension reduction. We also used support vector machine (SVM) as emotional state transition classifier. We constructed a prototype music recommendation system based on these features and carried out various experiments to measure its performance. We report some of the experimental results.

AB - Context-based music recommendation is one of rapidly emerging applications in the advent of ubiquitous era and requires multidisciplinary efforts including low level feature extraction and music classification, human emotion description and prediction, ontology-based representation and recommendation, and the establishment of connections among them. In this paper, we contributed in three distinctive ways to take into account the idea of context awareness in the music recommendation field. Firstly, we propose a novel emotion state transition model (ESTM) to model human emotional states and their transitions by music. ESTM acts like a bridge between user situation information along with his/her emotion and low-level music features. With ESTM, we can recommend the most appropriate music to the user for transiting to the desired emotional state. Secondly, we present context-based music recommendation (COMUS) ontology for modeling user's musical preferences and context, and for supporting reasoning about the user's desired emotion and preferences. The COMUS is music-dedicated ontology in OWL constructed by incorporating domain-specific classes for music recommendation into the Music Ontology, which includes situation, mood, and musical features. Thirdly, for mapping low-level features to ESTM, we collected various high-dimensional music feature data and applied nonnegative matrix factorization (NMF) for their dimension reduction. We also used support vector machine (SVM) as emotional state transition classifier. We constructed a prototype music recommendation system based on these features and carried out various experiments to measure its performance. We report some of the experimental results.

KW - Classification

KW - Emotion

KW - Emotion state transition model

KW - Mood

KW - Music information retrieval

KW - Recommendation

UR - http://www.scopus.com/inward/record.url?scp=77950189462&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950189462&partnerID=8YFLogxK

U2 - 10.1007/s11042-009-0332-6

DO - 10.1007/s11042-009-0332-6

M3 - Article

VL - 47

SP - 433

EP - 460

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

IS - 3

ER -