Kernel-based nonlinear blind source separation

Stefan Harmeling, Andreas Ziehe, Motoaki Kawanabe, Klaus Muller

Research output: Contribution to journalArticle

90 Citations (Scopus)

Abstract

We propose kTDSEP, a kernel-based algorithm for nonlinear blind source separation (BSS). It combines complementary research fields: kernel feature spaces and BSS using temporal information. This yields an efficient algorithm for nonlinear BSS with invertible nonlinearity. Key assumptions are that the kernel feature space is chosen rich enough to approximate the nonlinearity and that signals of interest contain temporal information. Both assumptions are fulfilled for a wide set of real-world applications. The algorithm works as follows: First, the data are (implicitly) mapped to a high (possibly infinite)-dimensional kernel feature space. In practice, however, the data form a smaller submanifold in feature space - even smaller than the number of training data points - a fact that has already been used by, for example, reduced set techniques for support vector machines. We propose to adapt to this effective dimension as a preprocessing step and to construct an orthonormal basis of this submanifold. The latter dimension-reduction step is essential for making the subsequent application of BSS methods computationally and numerically tractable. In the reduced space, we use a BSS algorithm that is based on second-order temporal decorrelation. Finally, we propose a selection procedure to obtain the original sources from the extracted nonlinear components automatically. Experiments demonstrate the excellent performance and efficiency of our kTDSEP algorithm for several problems of nonlinear BSS and for more than two sources.

Original languageEnglish
Pages (from-to)1089-1124
Number of pages36
JournalNeural Computation
Volume15
Issue number5
DOIs
Publication statusPublished - 2003 May 1
Externally publishedYes

Fingerprint

Blind source separation
Support vector machines
Kernel
Research
Experiments

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Artificial Intelligence
  • Neuroscience(all)

Cite this

Harmeling, S., Ziehe, A., Kawanabe, M., & Muller, K. (2003). Kernel-based nonlinear blind source separation. Neural Computation, 15(5), 1089-1124. https://doi.org/10.1162/089976603765202677

Kernel-based nonlinear blind source separation. / Harmeling, Stefan; Ziehe, Andreas; Kawanabe, Motoaki; Muller, Klaus.

In: Neural Computation, Vol. 15, No. 5, 01.05.2003, p. 1089-1124.

Research output: Contribution to journalArticle

Harmeling, S, Ziehe, A, Kawanabe, M & Muller, K 2003, 'Kernel-based nonlinear blind source separation', Neural Computation, vol. 15, no. 5, pp. 1089-1124. https://doi.org/10.1162/089976603765202677
Harmeling S, Ziehe A, Kawanabe M, Muller K. Kernel-based nonlinear blind source separation. Neural Computation. 2003 May 1;15(5):1089-1124. https://doi.org/10.1162/089976603765202677
Harmeling, Stefan ; Ziehe, Andreas ; Kawanabe, Motoaki ; Muller, Klaus. / Kernel-based nonlinear blind source separation. In: Neural Computation. 2003 ; Vol. 15, No. 5. pp. 1089-1124.
@article{52e62872db764a9891c60546d2e6f125,
title = "Kernel-based nonlinear blind source separation",
abstract = "We propose kTDSEP, a kernel-based algorithm for nonlinear blind source separation (BSS). It combines complementary research fields: kernel feature spaces and BSS using temporal information. This yields an efficient algorithm for nonlinear BSS with invertible nonlinearity. Key assumptions are that the kernel feature space is chosen rich enough to approximate the nonlinearity and that signals of interest contain temporal information. Both assumptions are fulfilled for a wide set of real-world applications. The algorithm works as follows: First, the data are (implicitly) mapped to a high (possibly infinite)-dimensional kernel feature space. In practice, however, the data form a smaller submanifold in feature space - even smaller than the number of training data points - a fact that has already been used by, for example, reduced set techniques for support vector machines. We propose to adapt to this effective dimension as a preprocessing step and to construct an orthonormal basis of this submanifold. The latter dimension-reduction step is essential for making the subsequent application of BSS methods computationally and numerically tractable. In the reduced space, we use a BSS algorithm that is based on second-order temporal decorrelation. Finally, we propose a selection procedure to obtain the original sources from the extracted nonlinear components automatically. Experiments demonstrate the excellent performance and efficiency of our kTDSEP algorithm for several problems of nonlinear BSS and for more than two sources.",
author = "Stefan Harmeling and Andreas Ziehe and Motoaki Kawanabe and Klaus Muller",
year = "2003",
month = "5",
day = "1",
doi = "10.1162/089976603765202677",
language = "English",
volume = "15",
pages = "1089--1124",
journal = "Neural Computation",
issn = "0899-7667",
publisher = "MIT Press Journals",
number = "5",

}

TY - JOUR

T1 - Kernel-based nonlinear blind source separation

AU - Harmeling, Stefan

AU - Ziehe, Andreas

AU - Kawanabe, Motoaki

AU - Muller, Klaus

PY - 2003/5/1

Y1 - 2003/5/1

N2 - We propose kTDSEP, a kernel-based algorithm for nonlinear blind source separation (BSS). It combines complementary research fields: kernel feature spaces and BSS using temporal information. This yields an efficient algorithm for nonlinear BSS with invertible nonlinearity. Key assumptions are that the kernel feature space is chosen rich enough to approximate the nonlinearity and that signals of interest contain temporal information. Both assumptions are fulfilled for a wide set of real-world applications. The algorithm works as follows: First, the data are (implicitly) mapped to a high (possibly infinite)-dimensional kernel feature space. In practice, however, the data form a smaller submanifold in feature space - even smaller than the number of training data points - a fact that has already been used by, for example, reduced set techniques for support vector machines. We propose to adapt to this effective dimension as a preprocessing step and to construct an orthonormal basis of this submanifold. The latter dimension-reduction step is essential for making the subsequent application of BSS methods computationally and numerically tractable. In the reduced space, we use a BSS algorithm that is based on second-order temporal decorrelation. Finally, we propose a selection procedure to obtain the original sources from the extracted nonlinear components automatically. Experiments demonstrate the excellent performance and efficiency of our kTDSEP algorithm for several problems of nonlinear BSS and for more than two sources.

AB - We propose kTDSEP, a kernel-based algorithm for nonlinear blind source separation (BSS). It combines complementary research fields: kernel feature spaces and BSS using temporal information. This yields an efficient algorithm for nonlinear BSS with invertible nonlinearity. Key assumptions are that the kernel feature space is chosen rich enough to approximate the nonlinearity and that signals of interest contain temporal information. Both assumptions are fulfilled for a wide set of real-world applications. The algorithm works as follows: First, the data are (implicitly) mapped to a high (possibly infinite)-dimensional kernel feature space. In practice, however, the data form a smaller submanifold in feature space - even smaller than the number of training data points - a fact that has already been used by, for example, reduced set techniques for support vector machines. We propose to adapt to this effective dimension as a preprocessing step and to construct an orthonormal basis of this submanifold. The latter dimension-reduction step is essential for making the subsequent application of BSS methods computationally and numerically tractable. In the reduced space, we use a BSS algorithm that is based on second-order temporal decorrelation. Finally, we propose a selection procedure to obtain the original sources from the extracted nonlinear components automatically. Experiments demonstrate the excellent performance and efficiency of our kTDSEP algorithm for several problems of nonlinear BSS and for more than two sources.

UR - http://www.scopus.com/inward/record.url?scp=0041376445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0041376445&partnerID=8YFLogxK

U2 - 10.1162/089976603765202677

DO - 10.1162/089976603765202677

M3 - Article

AN - SCOPUS:0041376445

VL - 15

SP - 1089

EP - 1124

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 5

ER -