Support Vector Data Descriptions and k-Means Clustering

One Class?

Nico Gornitz, Luiz Alberto Lima, Klaus Muller, Marius Kloft, Shinichi Nakajima

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.

Original languageEnglish
JournalIEEE Transactions on Neural Networks and Learning Systems
DOIs
Publication statusAccepted/In press - 2017 Sep 27

Fingerprint

Data description
Cluster Analysis
Learning
Boidae
Software packages
Software

Keywords

  • Anomaly detection
  • Anomaly detection
  • clustering
  • Clustering algorithms
  • k-means
  • Kernel
  • Learning systems
  • Level set
  • one-class classification
  • Optimization
  • support vector data description (SVDD)
  • Support vector machines

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Support Vector Data Descriptions and k-Means Clustering : One Class? / Gornitz, Nico; Lima, Luiz Alberto; Muller, Klaus; Kloft, Marius; Nakajima, Shinichi.

In: IEEE Transactions on Neural Networks and Learning Systems, 27.09.2017.

Research output: Contribution to journalArticle

Gornitz, Nico ; Lima, Luiz Alberto ; Muller, Klaus ; Kloft, Marius ; Nakajima, Shinichi. / Support Vector Data Descriptions and k-Means Clustering : One Class?. In: IEEE Transactions on Neural Networks and Learning Systems. 2017.
@article{ecd0ebd17fb841798bee5c8993314b32,
title = "Support Vector Data Descriptions and k-Means Clustering: One Class?",
abstract = "We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.",
keywords = "Anomaly detection, Anomaly detection, clustering, Clustering algorithms, k-means, Kernel, Learning systems, Level set, one-class classification, Optimization, support vector data description (SVDD), Support vector machines",
author = "Nico Gornitz and Lima, {Luiz Alberto} and Klaus Muller and Marius Kloft and Shinichi Nakajima",
year = "2017",
month = "9",
day = "27",
doi = "10.1109/TNNLS.2017.2737941",
language = "English",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",

}

TY - JOUR

T1 - Support Vector Data Descriptions and k-Means Clustering

T2 - One Class?

AU - Gornitz, Nico

AU - Lima, Luiz Alberto

AU - Muller, Klaus

AU - Kloft, Marius

AU - Nakajima, Shinichi

PY - 2017/9/27

Y1 - 2017/9/27

N2 - We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.

AB - We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.

KW - Anomaly detection

KW - Anomaly detection

KW - clustering

KW - Clustering algorithms

KW - k-means

KW - Kernel

KW - Learning systems

KW - Level set

KW - one-class classification

KW - Optimization

KW - support vector data description (SVDD)

KW - Support vector machines

UR - http://www.scopus.com/inward/record.url?scp=85030751821&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030751821&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2017.2737941

DO - 10.1109/TNNLS.2017.2737941

M3 - Article

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

ER -