Learning Spatiotemporal Graph Representations for Visual Perception using EEG Signals

Jenifer Kalafatovich, Minji Lee, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

Abstract

Perceiving and recognizing objects enable interaction with the external environment. Recently, decoding brain signals based on brain-computer interface (BCI) that recognize the user’s intentions by just looking at objects has attracted attention as a next-generation intuitive interface. However, classifying signals from different objects is very challenging, and in practice, decoding performance for visual perception is not yet high enough to be used in real environments. In this study, we aimed to classify single-trial electroencephalography signals evoked by visual stimuli into their corresponding semantic category. We proposed a two-stream convolutional neural network to increase classification performance. The model consists of a spatial stream and a temporal stream that use graph convolutional neural network and channel-wise convolutional neural network respectively. Two public datasets were used to evaluate the proposed model; (i) SU DB (a set of 72 photographs of objects belonging to 6 semantic categories) and MPI DB (8 exemplars belonging to two categories). Our results outperform state-of-the-art methods, with accuracies of 54.28 ± 7.89% for SU DB (6-class) and 84.40 ± 8.03% for MPI DB (2-class). These results could facilitate the application of intuitive BCI systems based on visual perception.

Original languageEnglish
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Neural Systems and Rehabilitation Engineering
DOIs
Publication statusAccepted/In press - 2022

Keywords

  • Brain modeling
  • Convolution
  • Convolutional neural networks
  • Electrodes
  • Electroencephalography
  • Feature extraction
  • Semantics
  • Visual perception
  • brain-computer interface (BCI)
  • convolutional neural network (CNN)
  • electroencephalography (EEG)

ASJC Scopus subject areas

  • Internal Medicine
  • Neuroscience(all)
  • Biomedical Engineering
  • Rehabilitation

Fingerprint

Dive into the research topics of 'Learning Spatiotemporal Graph Representations for Visual Perception using EEG Signals'. Together they form a unique fingerprint.

Cite this