Accelerating generalized iterative scaling using componentwise extrapolations for on-line conditional random fields

Hee Deok Yang, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, a simple and globally convergent method based on penalized generalized iterative scaling (GIS) with staggered Aitken acceleration is proposed to efficiently estimate the parameters for an on-line conditional random field (CRF). The staggered Aitken acceleration method, which alternates between an acceleration step and a non-acceleration step, provides numerical stability and computational simplicity in analyzing the incompleteness of data. The proposed method is based on stochastic gradient descent (SGD) and it has the following advantages: (1) it can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) it can reduce the computing time by eliminating computation of the inverse of the objective function's Hessian matrix with staggered Aitken acceleration. We show the convergence of penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with that of other stochastic optimization methods, and also illustrate experimental results with public data sets.

Original languageEnglish
Title of host publication2010 International Conference on Machine Learning and Cybernetics, ICMLC 2010
Pages3169-3173
Number of pages5
DOIs
Publication statusPublished - 2010
Event2010 International Conference on Machine Learning and Cybernetics, ICMLC 2010 - Qingdao, China
Duration: 2010 Jul 112010 Jul 14

Publication series

Name2010 International Conference on Machine Learning and Cybernetics, ICMLC 2010
Volume6

Other

Other2010 International Conference on Machine Learning and Cybernetics, ICMLC 2010
CountryChina
CityQingdao
Period10/7/1110/7/14

Keywords

  • Aitken acceleration
  • Conditional random field
  • Incremental learning
  • On-line learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Accelerating generalized iterative scaling using componentwise extrapolations for on-line conditional random fields'. Together they form a unique fingerprint.

Cite this