Can a freakish event in a long series be blamed for apparent model failure ? An analysis of the sensitivity of continuous evaluation criteria
Résumé
Criteria based on the root mean square error are commonly used to evaluate the performance of hydrological models. Due to their quadratic form, these criteria put more emphasis on the largest model errors. These mostly occur during flood events as hydrological models tend to produce heteroscedastic errors. Hydrologists interested in those events could put up with such a fact if they are not looking for the most likely simulation but for the best performance over some determined events. But this behaviour may hamper the interpretation of performance criteria calculated over long periods due to the role of specific events in the total model error. Using such criteria is difficult to justify if one cannot ensure that the performance criterion converges when the length of the data series increases to infinity. This non convergence may happen if the error magnitude increases too fast with the event magnitude, in comparison with the increase of the intensity of events with the return period. Since a longer data period is more likely to contain a rare event, the performance criterion may be driven by this extreme event. This problem is investigated in the case of flow forecasting. In forecasting applications, the use of data assimilation techniques considerably modifies the distribution of model errors compared with those obtained in simulation mode. The main part of the total errors is generally concentrated on the time steps where large flow variations are observed. This is especially true for short lead times for which the forecasted discharge is mainly driven by the assimilated data. The poster discusses the role of major events in the calculation of continuous criteria and the problem of criteria interpretation in this context.