Continuous EEG Decoding of Pilots' Mental States Using Multiple Feature Block-Based Convolutional Neural Network

Dae Hyeok Lee, Ji Hoon Jeong, Kiduk Kim, Baek Woon Yu, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

Abstract

Non-invasive brain-computer interface (BCI) has been developed for recognizing and classifying human mental states with high performances. Specifically, classifying pilots' mental states accurately is a critical issue because their cognitive states, which are induced by mental fatigue, workload, and distraction, may be fundamental in catastrophic accidents. In this study, we present an electroencephalogram (EEG) classification of four mental states (fatigue, workload, distraction, and the normal state) from EEG signals in both offline and pseudo-online analyses. To the best of our knowledge, this study is the first attempt to classify pilots' mental states using only EEG signals during continuous decoding. We recorded EEG signals from seven pilots under various simulated flight conditions. We proposed a multiple feature block-based convolutional neural network (MFB-CNN) with temporal-spatio EEG filters to recognize the pilot's current mental states. We validated the proposed method for two analyses across all subjects. In the offline analysis, we confirmed the classification accuracy of 0.75 (±0.04). Also, in the pseudo-online analysis, we obtained the detection accuracy of 0.72 (±0.20), 0.72 (±0.27), and 0.61 (±0.18) for fatigue, workload, and distraction, respectively. Hence, we demonstrate the feasibility of classifying various types of mental states for implementation in real-world environments.

Original languageEnglish
Article number9133061
Pages (from-to)121929-121941
Number of pages13
JournalIEEE Access
Volume8
DOIs
Publication statusPublished - 2020

Keywords

  • Brain-computer interface (BCI)
  • deep convolutional neural network
  • electroencephalogram (EEG)
  • mental states

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint Dive into the research topics of 'Continuous EEG Decoding of Pilots' Mental States Using Multiple Feature Block-Based Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this