Decoding EEG by visual-guided deep neural networks

Zhicheng Jiao, Haoxuan You, Fan Yang, Xin Li, Han Zhang, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Decoding visual stimuli from brain activities is an interdisciplinary study of neuroscience and computer vision. With the emerging of Human-AI Collaboration, Human-Computer Interaction, and the development of advanced machine learning models, brain decoding based on deep learning attracts more attention. Electroencephalogram (EEG) is a widely used neurophysiology tool. Inspired by the success of deep learning on image representation and neural decoding, we proposed a visual-guided EEG decoding method that contains a decoding stage and a generation stage. In the classification stage, we designed a visual-guided convolutional neural network (CNN) to obtain more discriminative representations from EEG, which are applied to achieve the classification results. In the generation stage, the visual-guided EEG features are input to our improved deep generative model with a visual consistence module to generate corresponding visual stimuli. With the help of our visual-guided strategies, the proposed method outperforms traditional machine learning methods and deep learning models in the EEG decoding task.

Original languageEnglish
Title of host publicationProceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019
EditorsSarit Kraus
PublisherInternational Joint Conferences on Artificial Intelligence
Pages1387-1393
Number of pages7
ISBN (Electronic)9780999241141
Publication statusPublished - 2019 Jan 1
Externally publishedYes
Event28th International Joint Conference on Artificial Intelligence, IJCAI 2019 - Macao, China
Duration: 2019 Aug 102019 Aug 16

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
Volume2019-August
ISSN (Print)1045-0823

Conference

Conference28th International Joint Conference on Artificial Intelligence, IJCAI 2019
CountryChina
CityMacao
Period19/8/1019/8/16

Fingerprint

Electroencephalography
Decoding
Learning systems
Brain models
Neurophysiology
Human computer interaction
Computer vision
Deep neural networks
Brain
Neural networks
Deep learning

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Jiao, Z., You, H., Yang, F., Li, X., Zhang, H., & Shen, D. (2019). Decoding EEG by visual-guided deep neural networks. In S. Kraus (Ed.), Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019 (pp. 1387-1393). (IJCAI International Joint Conference on Artificial Intelligence; Vol. 2019-August). International Joint Conferences on Artificial Intelligence.

Decoding EEG by visual-guided deep neural networks. / Jiao, Zhicheng; You, Haoxuan; Yang, Fan; Li, Xin; Zhang, Han; Shen, Dinggang.

Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. ed. / Sarit Kraus. International Joint Conferences on Artificial Intelligence, 2019. p. 1387-1393 (IJCAI International Joint Conference on Artificial Intelligence; Vol. 2019-August).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jiao, Z, You, H, Yang, F, Li, X, Zhang, H & Shen, D 2019, Decoding EEG by visual-guided deep neural networks. in S Kraus (ed.), Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. IJCAI International Joint Conference on Artificial Intelligence, vol. 2019-August, International Joint Conferences on Artificial Intelligence, pp. 1387-1393, 28th International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, 19/8/10.
Jiao Z, You H, Yang F, Li X, Zhang H, Shen D. Decoding EEG by visual-guided deep neural networks. In Kraus S, editor, Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. International Joint Conferences on Artificial Intelligence. 2019. p. 1387-1393. (IJCAI International Joint Conference on Artificial Intelligence).
Jiao, Zhicheng ; You, Haoxuan ; Yang, Fan ; Li, Xin ; Zhang, Han ; Shen, Dinggang. / Decoding EEG by visual-guided deep neural networks. Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. editor / Sarit Kraus. International Joint Conferences on Artificial Intelligence, 2019. pp. 1387-1393 (IJCAI International Joint Conference on Artificial Intelligence).
@inproceedings{19cb735bb5e045168b6aa77d3db1d5cd,
title = "Decoding EEG by visual-guided deep neural networks",
abstract = "Decoding visual stimuli from brain activities is an interdisciplinary study of neuroscience and computer vision. With the emerging of Human-AI Collaboration, Human-Computer Interaction, and the development of advanced machine learning models, brain decoding based on deep learning attracts more attention. Electroencephalogram (EEG) is a widely used neurophysiology tool. Inspired by the success of deep learning on image representation and neural decoding, we proposed a visual-guided EEG decoding method that contains a decoding stage and a generation stage. In the classification stage, we designed a visual-guided convolutional neural network (CNN) to obtain more discriminative representations from EEG, which are applied to achieve the classification results. In the generation stage, the visual-guided EEG features are input to our improved deep generative model with a visual consistence module to generate corresponding visual stimuli. With the help of our visual-guided strategies, the proposed method outperforms traditional machine learning methods and deep learning models in the EEG decoding task.",
author = "Zhicheng Jiao and Haoxuan You and Fan Yang and Xin Li and Han Zhang and Dinggang Shen",
year = "2019",
month = "1",
day = "1",
language = "English",
series = "IJCAI International Joint Conference on Artificial Intelligence",
publisher = "International Joint Conferences on Artificial Intelligence",
pages = "1387--1393",
editor = "Sarit Kraus",
booktitle = "Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019",

}

TY - GEN

T1 - Decoding EEG by visual-guided deep neural networks

AU - Jiao, Zhicheng

AU - You, Haoxuan

AU - Yang, Fan

AU - Li, Xin

AU - Zhang, Han

AU - Shen, Dinggang

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Decoding visual stimuli from brain activities is an interdisciplinary study of neuroscience and computer vision. With the emerging of Human-AI Collaboration, Human-Computer Interaction, and the development of advanced machine learning models, brain decoding based on deep learning attracts more attention. Electroencephalogram (EEG) is a widely used neurophysiology tool. Inspired by the success of deep learning on image representation and neural decoding, we proposed a visual-guided EEG decoding method that contains a decoding stage and a generation stage. In the classification stage, we designed a visual-guided convolutional neural network (CNN) to obtain more discriminative representations from EEG, which are applied to achieve the classification results. In the generation stage, the visual-guided EEG features are input to our improved deep generative model with a visual consistence module to generate corresponding visual stimuli. With the help of our visual-guided strategies, the proposed method outperforms traditional machine learning methods and deep learning models in the EEG decoding task.

AB - Decoding visual stimuli from brain activities is an interdisciplinary study of neuroscience and computer vision. With the emerging of Human-AI Collaboration, Human-Computer Interaction, and the development of advanced machine learning models, brain decoding based on deep learning attracts more attention. Electroencephalogram (EEG) is a widely used neurophysiology tool. Inspired by the success of deep learning on image representation and neural decoding, we proposed a visual-guided EEG decoding method that contains a decoding stage and a generation stage. In the classification stage, we designed a visual-guided convolutional neural network (CNN) to obtain more discriminative representations from EEG, which are applied to achieve the classification results. In the generation stage, the visual-guided EEG features are input to our improved deep generative model with a visual consistence module to generate corresponding visual stimuli. With the help of our visual-guided strategies, the proposed method outperforms traditional machine learning methods and deep learning models in the EEG decoding task.

UR - http://www.scopus.com/inward/record.url?scp=85074915375&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074915375&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85074915375

T3 - IJCAI International Joint Conference on Artificial Intelligence

SP - 1387

EP - 1393

BT - Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019

A2 - Kraus, Sarit

PB - International Joint Conferences on Artificial Intelligence

ER -