Abstract
Missing values cause critical problems on training a prediction model. Various missing data imputation methods have been introduced to settle down the problem. However, the imputation accuracy obtained by the methods is insufficient to validate performance of prediction models. Thus, in this study, we compare (1) imputation accuracy from various imputation methods as well as (2) the effects of imputation methods on prediction accuracy, investigating a relationship between imputation accuracy and prediction accuracy. For the comparison, we use water quality data composed of the latest actual observational multi-sensor data from Daecheong Lake. We conduct several experiments to compare seven imputation methods including a state of the art method, and their effects on three distinct prediction models. Through quantitative comparison and analysis, we proved that it is necessary to consider both imputation accuracy and model prediction accuracy when choosing an imputation method.
| Original language | English |
|---|---|
| Title of host publication | 21st International Conference on Advanced Communication Technology |
| Subtitle of host publication | ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 592-599 |
| Number of pages | 8 |
| ISBN (Electronic) | 9791188428021 |
| DOIs | |
| Publication status | Published - 2019 Apr 29 |
| Event | 21st International Conference on Advanced Communication Technology, ICACT 2019 - Pyeongchang, Korea, Republic of Duration: 2019 Feb 17 → 2019 Feb 20 |
Publication series
| Name | International Conference on Advanced Communication Technology, ICACT |
|---|---|
| Volume | 2019-February |
| ISSN (Print) | 1738-9445 |
Conference
| Conference | 21st International Conference on Advanced Communication Technology, ICACT 2019 |
|---|---|
| Country | Korea, Republic of |
| City | Pyeongchang |
| Period | 19/2/17 → 19/2/20 |
Fingerprint
Keywords
- amelia imputation
- imputation methods
- incomplete data
- knn imputation
- linear interpolation
- mean imputation
- mice imputation
- missing data
- missing values
- model performance
- randomforest imputation
- SVD imputation
ASJC Scopus subject areas
- Electrical and Electronic Engineering
Cite this
A Comparison of the Effects of Data Imputation Methods on Model Performance. / Kim, Wooyoung; Cho, Wonwoong; Choi, Jangho; Kim, Jiyong; Park, Cheonbok; Choo, Jaegul.
21st International Conference on Advanced Communication Technology: ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding. Institute of Electrical and Electronics Engineers Inc., 2019. p. 592-599 8702000 (International Conference on Advanced Communication Technology, ICACT; Vol. 2019-February).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - A Comparison of the Effects of Data Imputation Methods on Model Performance
AU - Kim, Wooyoung
AU - Cho, Wonwoong
AU - Choi, Jangho
AU - Kim, Jiyong
AU - Park, Cheonbok
AU - Choo, Jaegul
PY - 2019/4/29
Y1 - 2019/4/29
N2 - Missing values cause critical problems on training a prediction model. Various missing data imputation methods have been introduced to settle down the problem. However, the imputation accuracy obtained by the methods is insufficient to validate performance of prediction models. Thus, in this study, we compare (1) imputation accuracy from various imputation methods as well as (2) the effects of imputation methods on prediction accuracy, investigating a relationship between imputation accuracy and prediction accuracy. For the comparison, we use water quality data composed of the latest actual observational multi-sensor data from Daecheong Lake. We conduct several experiments to compare seven imputation methods including a state of the art method, and their effects on three distinct prediction models. Through quantitative comparison and analysis, we proved that it is necessary to consider both imputation accuracy and model prediction accuracy when choosing an imputation method.
AB - Missing values cause critical problems on training a prediction model. Various missing data imputation methods have been introduced to settle down the problem. However, the imputation accuracy obtained by the methods is insufficient to validate performance of prediction models. Thus, in this study, we compare (1) imputation accuracy from various imputation methods as well as (2) the effects of imputation methods on prediction accuracy, investigating a relationship between imputation accuracy and prediction accuracy. For the comparison, we use water quality data composed of the latest actual observational multi-sensor data from Daecheong Lake. We conduct several experiments to compare seven imputation methods including a state of the art method, and their effects on three distinct prediction models. Through quantitative comparison and analysis, we proved that it is necessary to consider both imputation accuracy and model prediction accuracy when choosing an imputation method.
KW - amelia imputation
KW - imputation methods
KW - incomplete data
KW - knn imputation
KW - linear interpolation
KW - mean imputation
KW - mice imputation
KW - missing data
KW - missing values
KW - model performance
KW - randomforest imputation
KW - SVD imputation
UR - http://www.scopus.com/inward/record.url?scp=85065674700&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065674700&partnerID=8YFLogxK
U2 - 10.23919/ICACT.2019.8702000
DO - 10.23919/ICACT.2019.8702000
M3 - Conference contribution
AN - SCOPUS:85065674700
T3 - International Conference on Advanced Communication Technology, ICACT
SP - 592
EP - 599
BT - 21st International Conference on Advanced Communication Technology
PB - Institute of Electrical and Electronics Engineers Inc.
ER -