[1]In general, there is a complex relationship between the quality and value of forecasts (Murphy 1993), but analysis of the quality is a reasonable place to start. We plan to carry out an experiment using a model of electrical utility load forecasting to consider the value for at least one user.
[2]We have no way of knowing how independent the different forecast sources are. Presumbably, the media sources use the NWS forecast as an input into their forecasting process or, at the very least, look at the same numerical guidance products that are available to NWS forecasters, but we cannot know that for certain. We wrote letters to each of the media sources asking questions about their procedures and forecast descriptors, but received only one reply. Therefore, we have had to make interpretations of some aspect of the forecasts, particularly the meaning of Probability of Precipitation (PoP) in the media forecasts. When no answer was received, we assumed they used the same definition as the National Weather Service (NWS), although the phrasing of the forecasts implies that a different time period (24 hours for the media and 12 hours for the NWS) is used in the forecasts. Based on the characteristics of the forecast PoP, we do not believe this decision has a significant impact on the interpretation of the forecasts. Nevertheless, the appearance of undefined terms represents a dilemma for forecast users.
3For the protection of all concerned, the four media sources have been assigned numbers 1-4 randomly.
[4]This comparison uses the long-term (1961-1990) climatology for each day. The sample climatology for these forecasts was 20.9%, while the long-term average across all days for Oklahoma City is 23.7%
[5]Doing this reduces the dimensionality of the verification problem, as discussed by Murphy (1991) and Brooks and Doswell (1996).
[6]The newspaper forecast is created by a private company under contract to the paper. We do not know for certain when the forecast is made. NWS forecast information may or may not be available to them.
[7]Significance testing was done using a Monte Carlo technique, using 100,000 trials of flipping a simulated coin the number of times that a forecast source disagreed with the NWS and counting how frequently the number of "heads" occurred by chance.
[8]Due to slightly different timings of the evening news presentations, the forecast portion of the television weather presentations frequently start at different times and it is possible for someone changing channels rapidly to see all the television forecasts from newscasts nominally at the same time.