Making and Evaluating Point Forecasts - Mathematics > Statistics TheoryReport as inadecuate




Making and Evaluating Point Forecasts - Mathematics > Statistics Theory - Download this document for free, or read online. Document in PDF available to download.

Abstract: Typically, point forecasting methods are compared and assessed by means of anerror measure or scoring function, such as the absolute error or the squarederror. The individual scores are then averaged over forecast cases, to resultin a summary measure of the predictive performance, such as the mean absoluteerror or the root mean squared error. I demonstrate that this common practicecan lead to grossly misguided inferences, unless the scoring function and theforecasting task are carefully matched.Effective point forecasting requires that the scoring function be specifiedex ante, or that the forecaster receives a directive in the form of astatistical functional, such as the mean or a quantile of the predictivedistribution. If the scoring function is specified ex ante, the forecaster canissue the optimal point forecast, namely, the Bayes rule. If the forecasterreceives a directive in the form of a functional, it is critical that thescoring function be consistent for it, in the sense that the expected score isminimized when following the directive.A functional is elicitable if there exists a scoring function that isstrictly consistent for it. Expectations, ratios of expectations and quantilesare elicitable. For example, a scoring function is consistent for the meanfunctional if and only if it is a Bregman function. It is consistent for aquantile if and only if it is generalized piecewise linear. Similarcharacterizations apply to ratios of expectations and to expectiles. Weightedscoring functions are consistent for functionals that adapt to the weighting inpeculiar ways. Not all functionals are elicitable; for instance, conditionalvalue-at-risk is not, despite its popularity in quantitative finance.



Author: Tilmann Gneiting

Source: https://arxiv.org/







Related documents