Emotion recognition using convolutional neural network with selected statistical photoplethysmogram features

Min Seop Lee, Yun Kyu Lee, Myo Taeg Lim, Tae Koo Kang

Research output: Contribution to journalArticle

Abstract

Emotion recognition research has been conducted using various physiological signals. In this paper, we propose an efficient photoplethysmogram-based method that fuses the deep features extracted by two deep convolutional neural networks and the statistical features selected by Pearson's correlation technique. A photoplethysmogram (PPG) signal can be easily obtained through many devices, and the procedure for recording this signal is simpler than that for other physiological signals. The normal-to-normal (NN) interval values of heart rate variability (HRV) were utilized to extract the time domain features, and the normalized PPG signal was used to acquire the frequency domain features. Then, we selected features that correlated highly with an emotion through Pearson's correlation. These statistical features were fused with deep-learning features extracted from a convolutional neural network (CNN). The PPG signal and the NN interval were used as the inputs of the CNN to extract the features, and the total concatenated features were utilized to classify the valence and the arousal, which are the basic parameters of emotion. The Database for Emotion Analysis using Physiological signals (DEAP) was chosen for the experiment, and the results demonstrated that the proposed method achieved a noticeable performance with a short recognition interval.

Original languageEnglish
Article number3501
JournalApplied Sciences (Switzerland)
Volume10
Issue number10
DOIs
Publication statusPublished - 2020 May 1

Keywords

  • Convolutional neural network
  • Emotion recognition
  • Feature fusion
  • PPG
  • Statistical feature

ASJC Scopus subject areas

  • Materials Science(all)
  • Instrumentation
  • Engineering(all)
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes

Fingerprint Dive into the research topics of 'Emotion recognition using convolutional neural network with selected statistical photoplethysmogram features'. Together they form a unique fingerprint.

  • Cite this