Two-dimensional solution surface for weighted support vector machines

Seung Jun Shin, Yichao Wu, Hao Helen Zhang

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

The support vector machine (SVM) is a popular learning method for binary classification. Standard SVMs treat all the data points equally, but in some practical problems it is more natural to assign different weights to observations from different classes. This leads to a broader class of learning, the so-called weighted SVMs (WSVMs), and one of their important applications is to estimate class probabilities besides learning the classification boundary. There are two parameters associated with theWSVM optimization problem: one is the regularization parameter and the other is the weight parameter. In this article, we first establish that the WSVM solutions are jointly piecewise-linear with respect to both the regularization and weight parameter.We then develop a state-of-theart algorithm that can compute the entire trajectory of the WSVM solutions for every pair of the regularization parameter and the weight parameter at a feasible computational cost. The derived two-dimensional solution surface provides theoretical insight on the behavior of the WSVM solutions. Numerically, the algorithm can greatly facilitate the implementation of the WSVM and automate the selection process of the optimal regularization parameter. We illustrate the new algorithm on various examples. This article has online supplementary materials.

Original languageEnglish
Pages (from-to)383-402
Number of pages20
JournalJournal of Computational and Graphical Statistics
Volume23
Issue number2
DOIs
Publication statusPublished - 2014

Keywords

  • Binary classification
  • Coefficient path algorithm
  • Probability estimation
  • SVM

ASJC Scopus subject areas

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Two-dimensional solution surface for weighted support vector machines'. Together they form a unique fingerprint.

  • Cite this