Abstract
In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.
| Original language | English |
|---|---|
| Article number | 1250059 |
| Journal | International Journal of Wavelets, Multiresolution and Information Processing |
| Volume | 10 |
| Issue number | 6 |
| DOIs | |
| Publication status | Published - 2012 Nov 1 |
Fingerprint
Keywords
- Aitken acceleration
- On-line conditional random field
- sequence labeling
ASJC Scopus subject areas
- Applied Mathematics
- Information Systems
- Signal Processing
Cite this
Accelerating generalized iterative scaling based on staggered aitken method for on-line conditional random fields. / Yang, Hee Deok; Suk, Heung-Il; Lee, Seong Whan.
In: International Journal of Wavelets, Multiresolution and Information Processing, Vol. 10, No. 6, 1250059, 01.11.2012.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Accelerating generalized iterative scaling based on staggered aitken method for on-line conditional random fields
AU - Yang, Hee Deok
AU - Suk, Heung-Il
AU - Lee, Seong Whan
PY - 2012/11/1
Y1 - 2012/11/1
N2 - In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.
AB - In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method has the following advantages: (1) It can approximate parameters close to the empirical optimum in a single pass through the training examples; (2) It can reduce the computing time by approximating the Jacobian matrix of the mapping function and estimating the relation between the Jacobian and Hessian in order to replace the inverse of the objective function's Hessian matrix. We show the convergence of the penalized GIS based on the staggered Aitken acceleration method, compare its speed of convergence with those of other stochastic optimization methods, and illustrate experimental results with two public datasets.
KW - Aitken acceleration
KW - On-line conditional random field
KW - sequence labeling
UR - http://www.scopus.com/inward/record.url?scp=84871560525&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84871560525&partnerID=8YFLogxK
U2 - 10.1142/S0219691312500592
DO - 10.1142/S0219691312500592
M3 - Article
AN - SCOPUS:84871560525
VL - 10
JO - International Journal of Wavelets, Multiresolution and Information Processing
JF - International Journal of Wavelets, Multiresolution and Information Processing
SN - 0219-6913
IS - 6
M1 - 1250059
ER -