Mean and median
Feb 27, 2007
In the comments of the last post on on-line weather forecasts, Hadley raised the evergreen statistical question of mean vs median. In this context, median error is unaffected by particular days in which the forecaster makes extreme errors while mean error takes into account the magnitude of every forecasting error in the sample.
Which one to use depends on the situation. Brandon, who did the original analysis, was motivated by planning a trip to a unfamiliar location. In this case, he might be better served by lower mean error, which would imply few extremely bad forecasts.
On the other hand, if I am interested in my local weather, then I'd likely be less concerned about a few extremely bad forecasts, and more concerned that the forecast is on the money on most days. Then perhaps the median error would come into play.
It turns out it doesn't much matter for our weather forecast data. In this new chart, I superimposed the mean error data (in black). The scatter of points was exactly as it was for median error (in red). (MSN had a particularly bad forecast for a low temperature one day, which pulled its location to the left.)
This shows further that the difference between CNN, Intellicast and The Weather Channel is negligible.