Norman Temperature Verification TAbles

	PER	LFM	NGM	CON	NWSFO
ME	.15	.15	-.62	-.22	.49
MAE	6.62 (57.72)	3.35 (16.48)	3.38 (17.28)	3.04 (7.81)	2.80
RMSE	9.16 (58.48)	4.48 (15.19)	4.37 (13.01)	4.04 (5.76)	3.80

Table 1: Mean error (ME), mean absolute error (MAE), and root mean square error (RMSE) in šF for five different high temperature forecast systems valid during the 12-24 hour period for Oklahoma City. Forecast systems are persistence (PER), MOS guidance from the LFM (LFM), MOS guidance from the NGM (NGM), an average of the LFM and NGM MOS (CON), and the forecast from the NWS Forecast Office in Norman, Oklahoma (NWS). Numbers in parentheses indicate percentage improvement by NWS forecast over the forecast for the given measure.

Range	PER	LFM	NGM	CON	NWSFO
36-40	1	0	0	0	0
31-35	4	0	0	0	0
26-30	4	0	0	0	0
21-25	11	0	0	0	0
16-20	16	0	1	0	0
11-15	33	6	2	2	0
6-10	52	40	53	36	27
1-5	114	223	276	268	198
0	39	66	57	58	87
1 5	173	192	153	182	230
6 10	84	49	37	34	40
11 15	42	12	10	8	7
16 20	11	0	1	2	1
21 25	4	2	0	0	0
26 30	1	0	0	0	0
31 35	1	0	0	0	0
36 40	0	0	0	0	0

Errors ¾ 5 šF	326 (55.3)	481 (81.5)	486 (82.4)	508 (86.1)	515 (87.3)
Errors ¾ 10 šF	462 (78.3)	570 (96.6)	576 (97.6)	578 (98.0)	582 (98.6)

Table 2: Errors in forecasts (forecast - observation) by 5 šF bins (except for perfect forecast) and number (percentage) of forecasts within 5 šF and 10 šF.

Observations

a)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25	N	p(f )
	¾-25	5	1	0	0	0	0	0	0	0	0	0	6	1.0
	-20	3	6	0	0	0	0	0	0	0	0	0	9	1.5
	-15	4	4	9	3	0	0	0	0	0	0	0	20	3.4
	-10	2	3	9	18	6	4	0	0	0	0	0	42	7.1
	-5	0	1	2	15	36	21	3	1	0	0	0	79	13.4
Forecasts	0	0	0	1	7	29	122	49	8	0	0	0	216	36.6
	5	0	1	1	0	3	40	61	26	2	1	0	135	22.9
	10	0	0	0	0	0	3	18	28	10	1	0	60	10.2
	15	0	0	0	0	0	0	1	2	10	2	2	17	2.9
	20	0	0	0	0	0	0	0	0	0	3	2	5	0.8
	„25	0	0	0	0	0	0	0	0	0	1	0	1	0.2
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7		298

Observations

b)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25	N	p(f )
	¾-25	6	4	0	0	0	0	0	0	0	0	0	10	1.7
	-20	5	6	1	0	0	0	0	0	0	0	0	12	2.0
	-15	2	2	11	3	0	1	0	0	0	0	0	19	3.2
	-10	1	1	5	23	16	3	0	0	0	0	0	49	8.3
	-5	0	2	4	13	35	71	9	0	0	0	0	134	22.7
Forecasts	0	0	1	1	3	16	87	51	7	0	0	0	166	28.1
	5	0	0	0	1	7	24	52	26	2	0	0	112	19.0
	10	0	0	0	0	0	3	19	25	7	0	0	54	9.2
	15	0	0	0	0	0	1	1	7	9	3	1	22	3.7
	20	0	0	0	0	0	0	0	0	4	4	2	10	1.7
	„25	0	0	0	0	0	0	0	0	0	1	1	2	0.3
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7		259

Table 3: Joint distribution of observed temperature changes and forecasts. N is total number of forecasts or observations in that bin. p(f ) [p(x )] is the marginal probability of that forecast [observation] in per cent. Number at lower right is number of forecasts in the same bin as the observed temperature change. a) LFM MOS. b) NGM MOS. c) CON. d) NWSFO

Observations

c)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25	N	p(f )
	¾-25	4	2	0	0	0	0	0	0	0	0	0	6	1.0
	-20	6	4	0	0	0	0	0	0	0	0	0	10	1.7
	-15	3	6	10	3	0	0	0	0	0	0	0	22	3.7
	-10	1	2	8	20	5	2	0	0	0	0	0	38	6.4
	-5	0	1	2	16	42	43	3	0	0	0	0	107	18.1
Forecasts	0	0	1	2	3	23	119	54	8	0	0	0	210	35.6
	5	0	0	0	1	4	24	59	21	2	0	0	111	18.8
	10	0	0	0	0	0	2	16	34	10	2	0	64	10.8
	15	0	0	0	0	0	0	0	2	9	1	2	14	2.4
	20	0	0	0	0	0	0	0	0	1	4	1	6	1.0
	„25	0	0	0	0	0	0	0	0	0	1	1	2	0.3
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7		306

Observations

d)		<-25	-20	-15	-10	-5	0	5	10	15	20	>25	N	p(f )
	¾-25	8	3	0	0	0	0	0	0	0	0	0	11	1.9
	-20	4	6	0	0	0	0	0	0	0	0	0	10	1.7
	-15	1	3	9	1	0	0	0	0	0	0	0	14	2.4
	-10	1	3	10	20	5	0	0	0	0	0	0	39	6.6
	-5	0	1	2	14	39	16	2	0	0	0	0	74	12.5
Forecasts	0	0	0	1	7	24	139	43	4	0	0	0	218	36.9
	5	0	0	0	1	4	30	65	22	2	0	0	124	21.0
	10	0	0	0	0	2	5	22	32	6	2	0	69	11.7
	15	0	0	0	0	0	0	0	7	12	2	1	22	3.7
	20	0	0	0	0	0	0	0	0	2	4	2	8	1.4
	„25	0	0	0	0	0	0	0	0	0	0	1	1	0.2
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7		335

a)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25	N	p(f)
	¾-25	83.3	16.7	.0	.0	.0	.0	.0	.0	.0	.0	.0	6	1.0
	-20	33.3	66.7	.0	.0	.0	.0	.0	.0	.0	.0	.0	9	1.5
	-15	20.0	20.0	45.0	15.0	.0	.0	.0	.0	.0	.0	.0	20	3.4
	-10	4.8	7.1	21.4	42.9	14.3	9.5	.0	.0	.0	.0	.0	42	7.1
	-5	.0	1.3	2.5	19.0	45.6	26.6	3.8	1.3	.0	.0	.0	79	13.4
Forecasts	0	.0	.0	.5	3.2	13.4	56.5	22.7	3.7	.0	.0	.0	216	36.6
	5	.0	.7	.7	.0	2.2	29.6	45.2	19.3	1.5	.7	.0	135	22.9
	10	.0	.0	.0	.0	.0	5.0	30.0	46.7	16.7	1.7	.0	60	10.2
	15	.0	.0	.0	.0	.0	.0	5.9	11.8	58.8	11.8	11.8	17	2.9
	20	.0	.0	.0	.0	.0	.0	.0	.0	.0	60.0	40.0	5	0.8
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	100.0	.0	1	0.2

Observations

b)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25	N	p(f)
	¾-25	60.0	40.0	.0	.0	.0	.0	.0	.0	.0	.0	.0	10	1.7
	-20	41.7	50.0	8.3	.0	.0	.0	.0	.0	.0	.0	.0	12	2.0
	-15	10.5	10.5	57.9	15.8	.0	5.3	.0	.0	.0	.0	.0	19	3.2
	-10	2.0	2.0	10.2	46.9	32.7	6.1	.0	.0	.0	.0	.0	49	8.3
	-5	.0	1.5	3.0	9.7	26.1	53.0	6.7	.0	.0	.0	.0	134	22.7
Forecasts	0	.0	.6	.6	1.8	9.6	52.4	30.7	4.2	.0	.0	.0	166	28.1
	5	.0	.0	.0	.9	6.3	21.4	46.4	23.2	1.8	.0	.0	112	19.0
	10	.0	.0	.0	.0	.0	5.6	35.2	46.3	13.0	.0	.0	54	9.2
	15	.0	.0	.0	.0	.0	4.5	4.5	31.8	40.9	13.6	4.5	22	3.7
	20	.0	.0	.0	.0	.0	.0	.0	.0	40.0	40.0	20.0	10	1.7
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	50.0	50.0	2	.3

Table 4: Conditional probability of observations given forecasts and marginal distribution of forecasts. Column and rows are 5 šF temperature bins centered on number in heading. N [p(f)] is total number (marginal probability) of cases in respective forecast bin. a) LFM MOS. b) NGM MOS. c) CON. d) NWSFO.

Observations

c)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25	N	p(f)
	¾-25	66.7	33.3	.0	.0	.0	.0	.0	.0	.0	.0	.0	6	1.0
	-20	60.0	40.0	.0	.0	.0	.0	.0	.0	.0	.0	.0	10	1.7
	-15	13.6	27.3	45.5	13.6	.0	.0	.0	.0	.0	.0	.0	22	3.7
	-10	2.6	5.3	21.1	52.6	13.2	5.3	.0	.0	.0	.0	.0	38	6.4
	-5	.0	.9	1.9	15.0	39.3	40.2	2.8	.0	.0	.0	.0	107	18.1
Forecasts	0	.0	.5	1.0	1.4	11.0	56.7	25.7	3.8	.0	.0	.0	210	35.6
	5	.0	.0	.0	.9	3.6	21.6	53.2	18.9	1.8	.0	.0	111	18.8
	10	.0	.0	.0	.0	.0	3.1	25.0	53.1	15.6	3.1	.0	64	10.8
	15	.0	.0	.0	.0	.0	.0	.0	14.3	64.3	7.1	14.3	14	2.4
	20	.0	.0	.0	.0	.0	.0	.0	.0	16.7	66.7	16.7	6	1.0
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	50.0	50.0	2	0.3

Observations

d)		<-25	-20	-15	-10	-5	0	5	10	15	20	>25	N	p(f)
	¾-25	72.7	27.3	.0	.0	.0	.0	.0	.0	.0	.0	.0	11	1.9
	-20	40.0	60.0	.0	.0	.0	.0	.0	.0	.0	.0	.0	10	1.7
	-15	7.1	21.4	64.3	7.1	.0	.0	.0	.0	.0	.0	.0	14	2.4
	-10	2.6	7.7	25.6	51.3	12.8	.0	.0	.0	.0	.0	.0	39	6.6
	-5	.0	1.4	2.7	18.9	52.7	21.6	2.7	.0	.0	.0	.0	74	12.5
Forecasts	0	.0	.0	.5	3.2	11.0	63.8	19.7	1.8	.0	.0	.0	218	37.0
	5	.0	.0	.0	.8	3.2	24.2	52.4	17.7	1.6	.0	.0	124	21.0
	10	.0	.0	.0	.0	2.9	7.2	31.9	46.4	8.7	2.9	.0	69	11.7
	15	.0	.0	.0	.0	.0	.0	.0	31.8	54.5	9.1	4.5	22	3.7
	20	.0	.0	.0	.0	.0	.0	.0	.0	25.0	50.0	25.0	8	1.4
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	.0	100.0	1	0.2

Observations

a)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25
	¾-25	35.7	6.3	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-20	21.4	37.5	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-15	28.6	25.0	40.9	7.0	.0	.0	.0	.0	.0	.0	.0
	-10	14.3	18.8	40.9	41.9	8.1	2.1	.0	.0	.0	.0	.0
	-5	.0	6.3	9.1	34.9	48.6	11.1	2.3	1.5	.0	.0	.0
Forecasts	0	.0	.0	4.5	16.3	39.2	64.2	37.1	12.3	.0	.0	.0
	5	.0	6.3	4.5	.0	4.1	21.1	46.2	40.0	9.1	12.5	.0
	10	.0	.0	.0	.0	.0	1.6	13.6	43.1	45.5	12.5	.0
	15	.0	.0	.0	.0	.0	.0	.8	3.1	45.5	25.0	50.0
	20	.0	.0	.0	.0	.0	.0	.0	.0	.0	37.5	50.0
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	12.5	.0
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7

Observations

b)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25
	¾-25	42.9	25.0	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-20	35.7	37.5	4.5	.0	.0	.0	.0	.0	.0	.0	.0
	-15	14.3	12.5	50.0	7.0	.0	.5	.0	.0	.0	.0	.0
	-10	7.1	6.3	22.7	53.5	21.6	1.6	.0	.0	.0	.0	.0
	-5	.0	12.5	18.2	30.2	47.3	37.4	6.8	.0	.0	.0	.0
Forecasts	0	.0	6.3	4.5	7.0	21.6	45.8	38.6	10.8	.0	.0	.0
	5	.0	.0	.0	2.3	9.5	12.6	39.4	40.0	9.1	.0	.0
	10	.0	.0	.0	.0	.0	1.6	14.4	38.5	31.8	.0	.0
	15	.0	.0	.0	.0	.0	.5	.8	10.8	40.9	37.5	25.0
	20	.0	.0	.0	.0	.0	.0	.0	.0	18.2	50.0	50.0
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	12.5	25.0
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7

Table 5: Conditional probability of forecasts given observations. Column and rows are 5 šF temperature bins centered on number in heading. N [p(x)] is total number (marginal distribution) of observations in each respective bin. a) LFM MOS. b) NGM MOS. c) CON. d) NWSFO.
Observations

c)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25
	¾-25	28.6	12.5	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-20	42.9	25.0	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-15	21.4	37.5	45.5	7.0	.0	.0	.0	.0	.0	.0	.0
	-10	7.1	12.5	36.4	46.5	6.8	1.1	.0	.0	.0	.0	.0
	-5	.0	6.3	9.1	37.2	56.8	22.6	2.3	.0	.0	.0	.0
Forecasts	0	.0	6.3	9.1	7.0	31.1	62.6	40.9	12.3	.0	.0	.0
	5	.0	.0	.0	2.3	5.4	12.6	44.7	32.3	9.1	.0	.0
	10	.0	.0	.0	.0	.0	1.1	12.1	52.3	45.5	25.0	.0
	15	.0	.0	.0	.0	.0	.0	.0	3.1	40.9	12.5	50.0
	20	.0	.0	.0	.0	.0	.0	.0	.0	4.5	50.0	25.0
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	12.5	25.0
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7

Observations

d)		¾-25	-20	-15	-10	-5	0	5	10	15	20	„25
	¾-25	57.1	18.8	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-20	28.6	37.5	.0	.0	.0	.0	.0	.0	.0	.0	.0
	-15	7.1	18.8	40.9	2.3	.0	.0	.0	.0	.0	.0	.0
	-10	7.1	18.8	45.5	46.5	6.8	.0	.0	.0	.0	.0	.0
	-5	.0	6.3	9.1	32.6	52.7	8.4	1.5	.0	.0	.0	.0
Forecasts	0	.0	.0	4.5	16.3	32.4	73.2	32.6	6.2	.0	.0	.0
	5	.0	.0	.0	2.3	5.4	15.8	49.2	33.8	9.1	.0	.0
	10	.0	.0	.0	.0	2.7	2.6	16.7	49.2	27.3	25.0	.0
	15	.0	.0	.0	.0	.0	.0	.0	10.8	54.5	25.0	25.0
	20	.0	.0	.0	.0	.0	.0	.0	.0	9.1	50.0	50.0
	„25	.0	.0	.0	.0	.0	.0	.0	.0	.0	.0	25.0
	N	14	16	22	43	74	190	132	65	22	8	4
	p(x)	2.4	2.7	3.7	7.3	12.5	32.2	22.4	11.0	3.7	1.4	0.7

	LFM	NGM	CON	NWSFO	OBS
Mean	.30	-.47	-.08	.64	.15
Standard deviation	7.56	8.41	7.60	8.17	9.17
Correlation with observations	.87	.88	.90	.91
a	-.17	.60	.23	-.51
b	1.06	.96	1.09	1.02

Table 6: Parameters associated with linear regression model for expected value of observations given forecast.

		COOLING
N = 139	LFM	NGM	CON	NWSFO
ME	2.45	1.63	2.24	2.78
MAE	3.63 (5.35)	3.30 (-4.14)	3.19 (-7.90)	3.44
RMSE	4.73 (4.18)	4.40 (-3.21)	4.23 (-7.28)	4.54
		WARMING
N = 219	LFM	NGM	CON	NWSFO
ME	-1.95	-1.86	-2.08	-1.15
MAE	3.20 (17.40)	3.31 (20.14)	3.01 (12.27)	2.64
RMSE	3.99 (15.95)	4.09 (17.86)	3.70 (9.28)	3.36

Table 7: Simple statistics as in Table 1, stratified by observed cooling of 3-17 šF and observed warming of 3-17 šF. Numbers in parenthesis are percentage improvement by unadjusted NWSFO forecast over guidance product. Negative values indicate guidance performed better in this parameter. Number of forecasts indicated at upper left.

		COOLING
N = 139	LFM	NGM	CON	NWSFO
ME	2.45	1.63	2.24	2.29
MAE	3.63 (12.24)	3.30 (3.44)	3.19 (-0.05)	3.19
RMSE	4.73 (10.14)	4.40 (3.22)	4.23 (-0.60)	4.25
		WARMING
N = 219	LFM	NGM	CON	NWSFO
ME	-1.95	-1.86	-2.08	-1.64
MAE	3.20 (17.40)	3.31 (20.14)	3.01 (12.27)	2.86
RMSE	3.99 (15.95)	4.09 (17.86)	3.70 (9.28)	3.55

Table 8: Same as Table 7, except for adjusted NWSFO forecast (cooled by 0.49 šF).

		COOLING
N = 30	LFM	NGM	CON	NWSFO
ME	6.83	4.73	6.07	3.47
MAE	7.43 (30.94)	6.13 (16.30)	6.80 (24.51)	5.13
RMSE	9.00 (29.10)	7.88 (19.03)	8.32 (23.28)	6.38
		WARMING
N = 12	LFM	NGM	CON	NWSFO
ME	-5.25	-2.00	-3.83	-3.75
MAE	5.75 (17.39)	3.83 (-23.91)	4.83 (1.72)	4.75
RMSE	7.21 (23.46)	4.30 (-28.22)	5.74 (3.99)	5.52

Table 9: Same as Table 7, except for extreme temperature changes (¾-18 šF and „18 šF).

		Forecast			Error
Date	CON	NWSFO	Observed	CON	NWSFO	NWSFO Improvement
2-Feb-93	-6	5	-4	-2	9	-7
22-Feb-93	-1	5	-2	1	7	-6
4-Feb-94	-11	-5	-11	0	6	-6
19-Feb-94	2	8	-3	5	11	-6
27-Mar-94	0	-6	0	0	-6	-6
10-Mar-94	25	14	22	3	-8	-5
14-Jan-93	4	-2	3	1	-5	-4
30-Dec-93	0	6	1	-1	5	-4
26-Sep-93	3	-3	1	2	-4	-2
5-Nov-93	-28	-36	-31	3	-5	-2
9-Dec-93	8	16	11	-3	5	-2
6-Jan-94	-16	-27	-22	6	-5	1
31-Jan-93	9	15	13	-4	2	2
2-Jul-94	-9	-3	-5	-4	2	2
9-Jan-93	-2	5	3	-5	2	3
24-Nov-93	-31	-39	-37	6	-2	4
9-Jan-94	2	10	8	-6	2	4
24-Feb-94	24	33	31	-7	2	5
15-Feb-93	-6	-12	-13	7	1	6
16-Feb-93	-3	-9	-12	9	3	6
29-Oct-93	-27	-21	-20	-7	-1	6
21-Feb-94	-3	-11	-10	7	-1	6
9-Mar-94	5	-1	-1	6	0	6
27-Dec-93	-16	-9	-9	-7	0	7
25-Feb-94	-15	-25	-30	15	5	10
8-Feb-94	2	-9	-18	20	9	11

Table 10: CON and NWSFO forecasts and errors for 26 days on which NWSFO and CON disagreed by more than 5 šF. "NWSFO Improvement" is the difference in the error of the two statistics, with the sign convention such that positive values indicate lower error for NWSFO.