Toward a direct measure of video quality perception using EEG

Simon Scholler, Sebastian Bosse, Matthias Sebastian Treder, Benjamin Blankertz, Gabriel Curio, Klaus Muller, Thomas Wiegand

Research output: Contribution to journalArticle

87 Citations (Scopus)

Abstract

An approach to the direct measurement of perception of video quality change using electroencephalography (EEG) is presented. Subjects viewed 8-s video clips while their brain activity was registered using EEG. The video signal was either uncompressed at full length or changed from uncompressed to a lower quality level at a random time point. The distortions were introduced by a hybrid video codec. Subjects had to indicate whether they had perceived a quality change. In response to a quality change, a positive voltage change in EEG (the so-called P3 component) was observed at latency of about 400-600 ms for all subjects. The voltage change positively correlated with the magnitude of the video quality change, substantiating the P3 component as a graded neural index of the perception of video quality change within the presented paradigm. By applying machine learning techniques, we could classify on a single-trial basis whether a subject perceived a quality change. Interestingly, some video clips wherein changes were missed (i.e., not reported) by the subject were classified as quality changes, suggesting that the brain detected a change, although the subject did not press a button. In conclusion, abrupt changes of video quality give rise to specific components in the EEG that can be detected on a single-trial basis. Potentially, a neurotechnological approach to video assessment could lead to a more objective quantification of quality change detection, overcoming the limitations of subjective approaches (such as subjective bias and the requirement of an overt response). Furthermore, it allows for real-time applications wherein the brain response to a video clip is monitored while it is being viewed.

Original languageEnglish
Article number6151827
Pages (from-to)2619-2629
Number of pages11
JournalIEEE Transactions on Image Processing
Volume21
Issue number5
DOIs
Publication statusPublished - 2012 May 1

Fingerprint

Electroencephalography
Surgical Instruments
Brain
Electric potential
Learning systems

Keywords

  • Electroencephalography (EEG)
  • perception
  • video coding
  • video quality

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software
  • Medicine(all)

Cite this

Scholler, S., Bosse, S., Treder, M. S., Blankertz, B., Curio, G., Muller, K., & Wiegand, T. (2012). Toward a direct measure of video quality perception using EEG. IEEE Transactions on Image Processing, 21(5), 2619-2629. [6151827]. https://doi.org/10.1109/TIP.2012.2187672

Toward a direct measure of video quality perception using EEG. / Scholler, Simon; Bosse, Sebastian; Treder, Matthias Sebastian; Blankertz, Benjamin; Curio, Gabriel; Muller, Klaus; Wiegand, Thomas.

In: IEEE Transactions on Image Processing, Vol. 21, No. 5, 6151827, 01.05.2012, p. 2619-2629.

Research output: Contribution to journalArticle

Scholler, S, Bosse, S, Treder, MS, Blankertz, B, Curio, G, Muller, K & Wiegand, T 2012, 'Toward a direct measure of video quality perception using EEG', IEEE Transactions on Image Processing, vol. 21, no. 5, 6151827, pp. 2619-2629. https://doi.org/10.1109/TIP.2012.2187672
Scholler S, Bosse S, Treder MS, Blankertz B, Curio G, Muller K et al. Toward a direct measure of video quality perception using EEG. IEEE Transactions on Image Processing. 2012 May 1;21(5):2619-2629. 6151827. https://doi.org/10.1109/TIP.2012.2187672
Scholler, Simon ; Bosse, Sebastian ; Treder, Matthias Sebastian ; Blankertz, Benjamin ; Curio, Gabriel ; Muller, Klaus ; Wiegand, Thomas. / Toward a direct measure of video quality perception using EEG. In: IEEE Transactions on Image Processing. 2012 ; Vol. 21, No. 5. pp. 2619-2629.
@article{3269aa209a2541c3a58b91a875ddb479,
title = "Toward a direct measure of video quality perception using EEG",
abstract = "An approach to the direct measurement of perception of video quality change using electroencephalography (EEG) is presented. Subjects viewed 8-s video clips while their brain activity was registered using EEG. The video signal was either uncompressed at full length or changed from uncompressed to a lower quality level at a random time point. The distortions were introduced by a hybrid video codec. Subjects had to indicate whether they had perceived a quality change. In response to a quality change, a positive voltage change in EEG (the so-called P3 component) was observed at latency of about 400-600 ms for all subjects. The voltage change positively correlated with the magnitude of the video quality change, substantiating the P3 component as a graded neural index of the perception of video quality change within the presented paradigm. By applying machine learning techniques, we could classify on a single-trial basis whether a subject perceived a quality change. Interestingly, some video clips wherein changes were missed (i.e., not reported) by the subject were classified as quality changes, suggesting that the brain detected a change, although the subject did not press a button. In conclusion, abrupt changes of video quality give rise to specific components in the EEG that can be detected on a single-trial basis. Potentially, a neurotechnological approach to video assessment could lead to a more objective quantification of quality change detection, overcoming the limitations of subjective approaches (such as subjective bias and the requirement of an overt response). Furthermore, it allows for real-time applications wherein the brain response to a video clip is monitored while it is being viewed.",
keywords = "Electroencephalography (EEG), perception, video coding, video quality",
author = "Simon Scholler and Sebastian Bosse and Treder, {Matthias Sebastian} and Benjamin Blankertz and Gabriel Curio and Klaus Muller and Thomas Wiegand",
year = "2012",
month = "5",
day = "1",
doi = "10.1109/TIP.2012.2187672",
language = "English",
volume = "21",
pages = "2619--2629",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

TY - JOUR

T1 - Toward a direct measure of video quality perception using EEG

AU - Scholler, Simon

AU - Bosse, Sebastian

AU - Treder, Matthias Sebastian

AU - Blankertz, Benjamin

AU - Curio, Gabriel

AU - Muller, Klaus

AU - Wiegand, Thomas

PY - 2012/5/1

Y1 - 2012/5/1

N2 - An approach to the direct measurement of perception of video quality change using electroencephalography (EEG) is presented. Subjects viewed 8-s video clips while their brain activity was registered using EEG. The video signal was either uncompressed at full length or changed from uncompressed to a lower quality level at a random time point. The distortions were introduced by a hybrid video codec. Subjects had to indicate whether they had perceived a quality change. In response to a quality change, a positive voltage change in EEG (the so-called P3 component) was observed at latency of about 400-600 ms for all subjects. The voltage change positively correlated with the magnitude of the video quality change, substantiating the P3 component as a graded neural index of the perception of video quality change within the presented paradigm. By applying machine learning techniques, we could classify on a single-trial basis whether a subject perceived a quality change. Interestingly, some video clips wherein changes were missed (i.e., not reported) by the subject were classified as quality changes, suggesting that the brain detected a change, although the subject did not press a button. In conclusion, abrupt changes of video quality give rise to specific components in the EEG that can be detected on a single-trial basis. Potentially, a neurotechnological approach to video assessment could lead to a more objective quantification of quality change detection, overcoming the limitations of subjective approaches (such as subjective bias and the requirement of an overt response). Furthermore, it allows for real-time applications wherein the brain response to a video clip is monitored while it is being viewed.

AB - An approach to the direct measurement of perception of video quality change using electroencephalography (EEG) is presented. Subjects viewed 8-s video clips while their brain activity was registered using EEG. The video signal was either uncompressed at full length or changed from uncompressed to a lower quality level at a random time point. The distortions were introduced by a hybrid video codec. Subjects had to indicate whether they had perceived a quality change. In response to a quality change, a positive voltage change in EEG (the so-called P3 component) was observed at latency of about 400-600 ms for all subjects. The voltage change positively correlated with the magnitude of the video quality change, substantiating the P3 component as a graded neural index of the perception of video quality change within the presented paradigm. By applying machine learning techniques, we could classify on a single-trial basis whether a subject perceived a quality change. Interestingly, some video clips wherein changes were missed (i.e., not reported) by the subject were classified as quality changes, suggesting that the brain detected a change, although the subject did not press a button. In conclusion, abrupt changes of video quality give rise to specific components in the EEG that can be detected on a single-trial basis. Potentially, a neurotechnological approach to video assessment could lead to a more objective quantification of quality change detection, overcoming the limitations of subjective approaches (such as subjective bias and the requirement of an overt response). Furthermore, it allows for real-time applications wherein the brain response to a video clip is monitored while it is being viewed.

KW - Electroencephalography (EEG)

KW - perception

KW - video coding

KW - video quality

UR - http://www.scopus.com/inward/record.url?scp=84860112691&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84860112691&partnerID=8YFLogxK

U2 - 10.1109/TIP.2012.2187672

DO - 10.1109/TIP.2012.2187672

M3 - Article

C2 - 22345537

AN - SCOPUS:84860112691

VL - 21

SP - 2619

EP - 2629

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 5

M1 - 6151827

ER -