Accelerating generalized iterative scaling based on staggered aitken method for on-line conditional random fields

Research output: Contribution to journalArticle

Abstract

In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.

Original languageEnglish
Article number1250059
JournalInternational Journal of Wavelets, Multiresolution and Information Processing
Volume10
Issue number6
DOIs
Publication statusPublished - 2012 Nov 1

Fingerprint

Conditional Random Fields
Scaling
Jacobian matrices
Hessian matrix
Incomplete Data
Speed of Convergence
Jacobian matrix
Stochastic Optimization
Stochastic Methods
Alternate
Optimization Methods
Simplicity
Objective function
Computing
Experimental Results
Estimate

Keywords

  • Aitken acceleration
  • On-line conditional random field
  • sequence labeling

ASJC Scopus subject areas

  • Applied Mathematics
  • Information Systems
  • Signal Processing

Cite this

@article{9f402b06e9ab45dea22e180b79cc4dc0,
title = "Accelerating generalized iterative scaling based on staggered aitken method for on-line conditional random fields",
abstract = "In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.",
keywords = "Aitken acceleration, On-line conditional random field, sequence labeling",
author = "Yang, {Hee Deok} and Heung-Il Suk and Lee, {Seong Whan}",
year = "2012",
month = "11",
day = "1",
doi = "10.1142/S0219691312500592",
language = "English",
volume = "10",
journal = "International Journal of Wavelets, Multiresolution and Information Processing",
issn = "0219-6913",
publisher = "World Scientific Publishing Co. Pte Ltd",
number = "6",

}

TY - JOUR

T1 - Accelerating generalized iterative scaling based on staggered aitken method for on-line conditional random fields

AU - Yang, Hee Deok

AU - Suk, Heung-Il

AU - Lee, Seong Whan

PY - 2012/11/1

Y1 - 2012/11/1

N2 - In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.

AB - In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.

KW - Aitken acceleration

KW - On-line conditional random field

KW - sequence labeling

UR - http://www.scopus.com/inward/record.url?scp=84871560525&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84871560525&partnerID=8YFLogxK

U2 - 10.1142/S0219691312500592

DO - 10.1142/S0219691312500592

M3 - Article

AN - SCOPUS:84871560525

VL - 10

JO - International Journal of Wavelets, Multiresolution and Information Processing

JF - International Journal of Wavelets, Multiresolution and Information Processing

SN - 0219-6913

IS - 6

M1 - 1250059

ER -