Brain-Controlled Robotic Arm System Based on Multi-Directional CNN-BiLSTM Network Using EEG Signals

Ji Hoon Jeong, Kyung Hwan Shim, Dong Joo Kim, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

105 Citations (Scopus)


Brain-machine interfaces (BMIs) can be used to decode brain activity into commands to control external devices. This paper presents the decoding of intuitive upper extremity imagery for multi-directional arm reaching tasks in three-dimensional (3D) environments. We designed and implemented an experimental environment in which electroencephalogram (EEG) signals can be acquired for movement execution and imagery. Fifteen subjects participated in our experiments. We proposed a multi-directional convolution neural network-bidirectional long short-term memory network (MDCBN)-based deep learning framework. The decoding performances for six directions in 3D space were measured by the correlation coefficient (CC) and the normalized root mean square error (NRMSE) between predicted and baseline velocity profiles. The grand-averaged CCs of multi-direction were 0.47 and 0.45 for the execution and imagery sessions, respectively, across all subjects. The NRMSE values were below 0.2 for both sessions. Furthermore, in this study, the proposed MDCBN was evaluated by two online experiments for real-time robotic arm control, and the grand-averaged success rates were approximately 0.60 (±0.14) and 0.43 (±0.09), respectively. Hence, we demonstrate the feasibility of intuitive robotic arm control based on EEG signals for real-world environments.

Original languageEnglish
Article number9040397
Pages (from-to)1226-1238
Number of pages13
JournalIEEE Transactions on Neural Systems and Rehabilitation Engineering
Issue number5
Publication statusPublished - 2020 May


  • Brain-machine interface (BMI)
  • deep learning
  • electroencephalogram (EEG)
  • intuitive robotic arm control
  • motor imagery

ASJC Scopus subject areas

  • Internal Medicine
  • Neuroscience(all)
  • Biomedical Engineering
  • Rehabilitation


Dive into the research topics of 'Brain-Controlled Robotic Arm System Based on Multi-Directional CNN-BiLSTM Network Using EEG Signals'. Together they form a unique fingerprint.

Cite this