Une nouvelle mesure de l'efficacité des modéles hydrologiques de prévision pilotés par les données

Translated title of the contribution: A new measure for assessing the efficiency of hydrological data-driven forecasting models

Seok Hwan Hwang, Dae Heon Ham, Joong Hoon Kim

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the naïve model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966-0.713 at correlation coefficients of 0.977-0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943-0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1% to 37.8% according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to naïve models in data-driven forecasting.Editor D. Koutsoyiannis, Associate editor D. YangCitation Hwang, S.H., Ham, D.H., and Kim, J.H., 2012. A new measure for assessing the efficiency of hydrological data-driven forecasting models. Hydrological Sciences Journal, 57 (7), 1257-1274.

Original languageFrench
Pages (from-to)1257-1274
Number of pages18
JournalHydrological Sciences Journal
Volume57
Issue number7
DOIs
Publication statusPublished - 2012 Oct 1

Fingerprint

persistence
autocorrelation
simulation
evaluation method
evaluation
science
index
method

Keywords

  • autocorrelation
  • forecasting efficiency
  • persistence
  • support vector machine

ASJC Scopus subject areas

  • Water Science and Technology

Cite this

Une nouvelle mesure de l'efficacité des modéles hydrologiques de prévision pilotés par les données. / Hwang, Seok Hwan; Ham, Dae Heon; Kim, Joong Hoon.

In: Hydrological Sciences Journal, Vol. 57, No. 7, 01.10.2012, p. 1257-1274.

Research output: Contribution to journalArticle

@article{a452aa44519f43298b46dcae295dda90,
title = "Une nouvelle mesure de l'efficacit{\'e} des mod{\'e}les hydrologiques de pr{\'e}vision pilot{\'e}s par les donn{\'e}es",
abstract = "There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the na{\"i}ve model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966-0.713 at correlation coefficients of 0.977-0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943-0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1{\%} to 37.8{\%} according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to na{\"i}ve models in data-driven forecasting.Editor D. Koutsoyiannis, Associate editor D. YangCitation Hwang, S.H., Ham, D.H., and Kim, J.H., 2012. A new measure for assessing the efficiency of hydrological data-driven forecasting models. Hydrological Sciences Journal, 57 (7), 1257-1274.",
keywords = "autocorrelation, forecasting efficiency, persistence, support vector machine",
author = "Hwang, {Seok Hwan} and Ham, {Dae Heon} and Kim, {Joong Hoon}",
year = "2012",
month = "10",
day = "1",
doi = "10.1080/02626667.2012.710335",
language = "French",
volume = "57",
pages = "1257--1274",
journal = "Hydrological Sciences Journal",
issn = "0262-6667",
publisher = "Taylor and Francis Ltd.",
number = "7",

}

TY - JOUR

T1 - Une nouvelle mesure de l'efficacité des modéles hydrologiques de prévision pilotés par les données

AU - Hwang, Seok Hwan

AU - Ham, Dae Heon

AU - Kim, Joong Hoon

PY - 2012/10/1

Y1 - 2012/10/1

N2 - There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the naïve model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966-0.713 at correlation coefficients of 0.977-0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943-0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1% to 37.8% according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to naïve models in data-driven forecasting.Editor D. Koutsoyiannis, Associate editor D. YangCitation Hwang, S.H., Ham, D.H., and Kim, J.H., 2012. A new measure for assessing the efficiency of hydrological data-driven forecasting models. Hydrological Sciences Journal, 57 (7), 1257-1274.

AB - There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the naïve model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966-0.713 at correlation coefficients of 0.977-0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943-0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1% to 37.8% according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to naïve models in data-driven forecasting.Editor D. Koutsoyiannis, Associate editor D. YangCitation Hwang, S.H., Ham, D.H., and Kim, J.H., 2012. A new measure for assessing the efficiency of hydrological data-driven forecasting models. Hydrological Sciences Journal, 57 (7), 1257-1274.

KW - autocorrelation

KW - forecasting efficiency

KW - persistence

KW - support vector machine

UR - http://www.scopus.com/inward/record.url?scp=84867272604&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867272604&partnerID=8YFLogxK

U2 - 10.1080/02626667.2012.710335

DO - 10.1080/02626667.2012.710335

M3 - Article

VL - 57

SP - 1257

EP - 1274

JO - Hydrological Sciences Journal

JF - Hydrological Sciences Journal

SN - 0262-6667

IS - 7

ER -