Evaluating Rank Histograms Using Decompositions of the Chi-Square Test Statistic.

Jolliffe, Ian T.; Primo, Cristina

Rank histograms are often plotted to evaluate the forecasts produced by an ensemble forecasting system—an ideal rank histogram is “flat” or uniform. It has been noted previously that the obvious test of “flatness,” the well-known χ2 goodness-of-fit test, spreads its power thinly and hence is not good at detecting specific alternatives to flatness, such as bias or over- or underdispersion. Members of the Cramér–von Mises family of tests do much better in this respect. An alternative to using the Cramér–von Mises family is to decompose the χ2 test statistic into components that correspond to specific alternatives. This approach is described in the present paper. It is arguably easier to use and more flexible than the Cramér–von Mises family of tests, and does at least as well as it in detecting alternatives corresponding to bias and over- or underdispersion.

ORDINAL measurement; FUTURES studies; FLATNESS measurement; CHI-squared test; GOODNESS-of-fit tests; DECOMPOSITION method; STATISTICAL hypothesis testing; DISPERSION (Chemistry)

Monthly Weather Review, 2008, Vol 136, Issue 6, p2133

0027-0644

Academic Journal

10.1175/2007MWR2219.1