Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis

Suk Min Lee, Jeong Woo Kim, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Driving assistance system has been recently studied to prevent emergency braking situations by combining external information on radar or camera devices and internal information on driver's intention. Electroencephalography (EEG) is an effective method to read user's intention with high temporal resolution. Our proposed system is mainly contributed to detecting driver's braking intention prior to stepping on the brake pedal in the emergency situation. We investigated early event-related potential (ERP) curves evoked by visual sensory process in emergency situation by using recurrent convolutional neural networks (RCNN) model. RCNN model has advantages to capture contextual and spatial patterns of brain signal. RCNN model is composed of a convolutional layer, two recurrent convolutional layers (RCLs), and a softmax layer. Fourteen participants drove for 120 minutes with two types of emergency situations and a normal driving situation in a virtual driving environment. In this article, early ERP showed a potential to be used for classifying the driver's braking intention. The classification performances based on RCNN and regularized linear discriminant analysis (RLDA) at 200 ms post-stimulus time were 0.86 AUC score and 0.61 AUC score respectively. Following the results, braking intention was recognized at 380 ms earlier based on early ERP patterns using RCNN model than the brake pedal. Our system could be applied to other brain-computer interface (BCI) system for minimizing detection time by capturing early ERP curves based on RCNN model.

Original languageEnglish
Title of host publicationProceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages846-851
Number of pages6
ISBN (Electronic)9781538633540
DOIs
Publication statusPublished - 2018 Dec 13
Event4th Asian Conference on Pattern Recognition, ACPR 2017 - Nanjing, China
Duration: 2017 Nov 262017 Nov 29

Other

Other4th Asian Conference on Pattern Recognition, ACPR 2017
CountryChina
CityNanjing
Period17/11/2617/11/29

Fingerprint

Braking
Electroencephalography
Neural networks
Brakes
Brain computer interface
Discriminant analysis
Brain
Radar
Cameras

Keywords

  • Brain-Computer Interface (BCI)
  • Electroencephalography (EEG)
  • Emergency Braking
  • Event-Related Potential (ERP)
  • Recurrent Convolutional Neural Networks (RCNN)

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Lee, S. M., Kim, J. W., & Lee, S. W. (2018). Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis. In Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017 (pp. 846-851). [8575932] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACPR.2017.86

Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis. / Lee, Suk Min; Kim, Jeong Woo; Lee, Seong Whan.

Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017. Institute of Electrical and Electronics Engineers Inc., 2018. p. 846-851 8575932.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Lee, SM, Kim, JW & Lee, SW 2018, Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis. in Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017., 8575932, Institute of Electrical and Electronics Engineers Inc., pp. 846-851, 4th Asian Conference on Pattern Recognition, ACPR 2017, Nanjing, China, 17/11/26. https://doi.org/10.1109/ACPR.2017.86
Lee SM, Kim JW, Lee SW. Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis. In Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017. Institute of Electrical and Electronics Engineers Inc. 2018. p. 846-851. 8575932 https://doi.org/10.1109/ACPR.2017.86
Lee, Suk Min ; Kim, Jeong Woo ; Lee, Seong Whan. / Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis. Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 846-851
@inproceedings{c36bc6db672348d49c74f194f15c55e7,
title = "Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis",
abstract = "Driving assistance system has been recently studied to prevent emergency braking situations by combining external information on radar or camera devices and internal information on driver's intention. Electroencephalography (EEG) is an effective method to read user's intention with high temporal resolution. Our proposed system is mainly contributed to detecting driver's braking intention prior to stepping on the brake pedal in the emergency situation. We investigated early event-related potential (ERP) curves evoked by visual sensory process in emergency situation by using recurrent convolutional neural networks (RCNN) model. RCNN model has advantages to capture contextual and spatial patterns of brain signal. RCNN model is composed of a convolutional layer, two recurrent convolutional layers (RCLs), and a softmax layer. Fourteen participants drove for 120 minutes with two types of emergency situations and a normal driving situation in a virtual driving environment. In this article, early ERP showed a potential to be used for classifying the driver's braking intention. The classification performances based on RCNN and regularized linear discriminant analysis (RLDA) at 200 ms post-stimulus time were 0.86 AUC score and 0.61 AUC score respectively. Following the results, braking intention was recognized at 380 ms earlier based on early ERP patterns using RCNN model than the brake pedal. Our system could be applied to other brain-computer interface (BCI) system for minimizing detection time by capturing early ERP curves based on RCNN model.",
keywords = "Brain-Computer Interface (BCI), Electroencephalography (EEG), Emergency Braking, Event-Related Potential (ERP), Recurrent Convolutional Neural Networks (RCNN)",
author = "Lee, {Suk Min} and Kim, {Jeong Woo} and Lee, {Seong Whan}",
year = "2018",
month = "12",
day = "13",
doi = "10.1109/ACPR.2017.86",
language = "English",
pages = "846--851",
booktitle = "Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Detecting driver's braking intention using recurrent convolutional neural networks based EEG Analysis

AU - Lee, Suk Min

AU - Kim, Jeong Woo

AU - Lee, Seong Whan

PY - 2018/12/13

Y1 - 2018/12/13

N2 - Driving assistance system has been recently studied to prevent emergency braking situations by combining external information on radar or camera devices and internal information on driver's intention. Electroencephalography (EEG) is an effective method to read user's intention with high temporal resolution. Our proposed system is mainly contributed to detecting driver's braking intention prior to stepping on the brake pedal in the emergency situation. We investigated early event-related potential (ERP) curves evoked by visual sensory process in emergency situation by using recurrent convolutional neural networks (RCNN) model. RCNN model has advantages to capture contextual and spatial patterns of brain signal. RCNN model is composed of a convolutional layer, two recurrent convolutional layers (RCLs), and a softmax layer. Fourteen participants drove for 120 minutes with two types of emergency situations and a normal driving situation in a virtual driving environment. In this article, early ERP showed a potential to be used for classifying the driver's braking intention. The classification performances based on RCNN and regularized linear discriminant analysis (RLDA) at 200 ms post-stimulus time were 0.86 AUC score and 0.61 AUC score respectively. Following the results, braking intention was recognized at 380 ms earlier based on early ERP patterns using RCNN model than the brake pedal. Our system could be applied to other brain-computer interface (BCI) system for minimizing detection time by capturing early ERP curves based on RCNN model.

AB - Driving assistance system has been recently studied to prevent emergency braking situations by combining external information on radar or camera devices and internal information on driver's intention. Electroencephalography (EEG) is an effective method to read user's intention with high temporal resolution. Our proposed system is mainly contributed to detecting driver's braking intention prior to stepping on the brake pedal in the emergency situation. We investigated early event-related potential (ERP) curves evoked by visual sensory process in emergency situation by using recurrent convolutional neural networks (RCNN) model. RCNN model has advantages to capture contextual and spatial patterns of brain signal. RCNN model is composed of a convolutional layer, two recurrent convolutional layers (RCLs), and a softmax layer. Fourteen participants drove for 120 minutes with two types of emergency situations and a normal driving situation in a virtual driving environment. In this article, early ERP showed a potential to be used for classifying the driver's braking intention. The classification performances based on RCNN and regularized linear discriminant analysis (RLDA) at 200 ms post-stimulus time were 0.86 AUC score and 0.61 AUC score respectively. Following the results, braking intention was recognized at 380 ms earlier based on early ERP patterns using RCNN model than the brake pedal. Our system could be applied to other brain-computer interface (BCI) system for minimizing detection time by capturing early ERP curves based on RCNN model.

KW - Brain-Computer Interface (BCI)

KW - Electroencephalography (EEG)

KW - Emergency Braking

KW - Event-Related Potential (ERP)

KW - Recurrent Convolutional Neural Networks (RCNN)

UR - http://www.scopus.com/inward/record.url?scp=85060549606&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060549606&partnerID=8YFLogxK

U2 - 10.1109/ACPR.2017.86

DO - 10.1109/ACPR.2017.86

M3 - Conference contribution

SP - 846

EP - 851

BT - Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -