15th AMS Conference on Weather Analysis and Forecasting
Norfolk, Virginia, 19-23 August 1996, in press
18.3
There are three premises that underlay our discussion. They have driven our thought processes and we believe, in many cases, that they lead to certain inescapable conclusions. These premises are:
1. Technology will usurp the traditional role of weather forecasters.
2. Quality and value [in the sense of Murphy (1993)] of weather forecasts should form the basis for the evaluation of the forecast process.
3 Strong economic pressures will reshape weather forecasting; e.g., funding for public weather services will not increase significantly relative to inflation.
From these basic assumptions, we believe it is possible to construct a weather forecasting system that best serves the needs of the public, and makes possible future improvement.
Public weather services are most appropriately suited to delivering two kinds of weather products. The first is a day to-day forecast of weather elements, e.g., minimum and maximum temperature, probability of precipitation (PoP) and precipitation type. This service provides the general public with enough information to plan the majority of daily activities in exchange for support, via taxation, of the observing and forecasting systems. It is not intended to provide detailed, very precise forecasts for extremely weather-sensitive users. For example, power-generating utilities have found that forecast errors as small as 2šF may cost them tens of thousands of dollars per day (Maddox, personal communication). Obviously, with economic risks this large, user needs are best served by the private sector.
Improvements in automated techniques for the forecasting of basic weather elements, such as Model Output Statistics (MOS), have led to a situation where human forecasters are adding little practical value to the numerical forecast product in most situations (Vislocky and Fritsch 1995, Roebber and Bosart 1996). Linear regression on the trends of accuracy of MOS and National Weather Service (NWS) forecasts indicate that the difference in root-mean squared error between the NWS and MOS 12-24 hour temperature forecasts has dropped from 4.4šF to 3.2šF over the last two decades (Fig. 1). Similarly, the difference in improvement over climatological PoP forecasts has dropped from 4.4% to 2.4%. In 1993, MOS temperature forecasts from the Nested Grid Model (NGM) for the 12-24 hour period for the United States were correct within 5šF approximately 85% of the time. Given typical temperature gradients and the relatively large area associated with most current zone forecasts, such as that for metropolitan areas, this probably represents an accuracy that is at the edge of detection for most of the general public.
Most disturbing, however, is that MOS is within 10šF 98% of the time! Moreover, as model resolution increases and the use of ensemble techniques adds richness to the information available from NWP, the difficulty for humans to routinely improve upon the accuracy of automated forecasting techniques for basic weather elements will become economically unfeasible. Within a decade, numerical models will be capable of generating forecasts or weather elements on spatial scales of several kilometers and temporal scales of several minutes. Indeed, it is already possible for anyone with access to the Internet to get more detailed forecasts of temperature and PoP from MOS than that delivered in forecasts from NWS forecast offices. It will be impossible for a human forecasters to improve on automated forecasts at a meaningful number of locations and times, unless the forecast user has very specific needs or on those rare occasions when the numerical models are in gross error. It is important to note, though, that although the number of large MOS errors is small, errors larger than 30šF occasionally do occur. Thus, one primary role of human forecasters in the public weather service of the future will be in the area of "quality control", i.e., rescuing the numerical models on days of extremely large errors.
The second type of public weather product, and by far the most important one, stems from the primary mission of the NWS: protection of life and property. Forecasts and warnings must be issued for a wide variety of threats to public safety and property. It is important to note that protection of life and property is, in some sense, the hardest forecast. It includes forecasts of rare events and cases in which the consequences of a bad forecast have the greatest impact on the general public. Thus, the requirements for forecasters must be the most stringent of all.
As human forecasters in the public sector move primarily into the area of dealing with very difficult, very important forecasts, the staffing structure and the individuals involved will need to be very different from the present time. Although they are closely related, we will deal with these two areas separately.
In order to consider the needs for staffing at individual offices, we need to consider the appropriate area of responsibility for an individual office. Ideally, the area should be large enough so that forecast challenges are sufficiently frequent for forecasters to maintain their skills through constant use. Roebber and Bosart (1996) caution against "the likelihood that human forecaster skills will atrophy with time unless they are used on a regular basis." One way to produce an order-of-magnitude estimate of the area of responsibility necessary for this is to consider the area needed to have a significant chance of doing the NWP "quality control" aspect of forecasting on a regular basis. Based on the figure of 98% for the 12-24 hour NGM MOS forecasts being within 10šF of the observations, it would require 35 MOS sites for a forecaster to average having a 50% chance of having a MOS error or 10šF or more each day, if MOS errors are randomly distributed in space and time. Since they are not, this puts a lower bound on the number of sites necessary to have a 50% chance of large MOS errors. Still, it represents approximately 10% of all MOS sites in the contiguous United States. As a first guess, therefore, we believe that an appropriate number of NWS forecast offices would be on the O(10), not the O(100) currently planned in the NWS modernization.
On the other hand, we believe that the number of meteorologists on duty at any one time at a forecast office should be much larger than in today's offices. An analogy for the staffing situation is found in fire stations around the country. Fire stations are staffed in such a way that there are enough trained people to handle emergency situations. This results in a situation where the staff may spend much of a particular shift doing support and training activities. Similarly, in our vision of the public weather service of the future, there will be a "critical mass" of well-trained, well-equipped meteorologists in an office to handle the emergency situations that arise and are the primary concern of the forecasters at the office. This size of this group would reduce the problem of "tunnel vision", in which a single forecaster may not anticipate a significant, rare event or may become preoccupied with one threat, such as severe thunderstorms or tornadoes, and ignore another important threat, such as flash flooding. Again, an order of magnitude estimate of the number of trained meteorologists on duty at any one time in any office would be around 10.
Electronic communications have reached the state where a forecaster does not need to be located near the warning radar in order to get information from it in a timely fashion. Having multiple platforms from which to access radar data, for example, will allow a group of forecasters to survey the entire range of threats, even in the case where the thunderstorms are confined to a small area. The question of communicating with spotter groups is one that needs to be addressed, but does not represent a major stumbling block.
There will continue to be a need for two approaches to the general forecast problem, one based on the phenomena of interest and the other based on the area of threat. In the current structure of the NWS, we see the phenomenonological emphasis as being associated with the national centers, e.g., the Storm Prediction Center (SPC), Hydrometeorological Prediction Center (HPC), and Tropical Prediction Center (TPC), while the areal emphasis is associated with the local forecast offices. By decreasing the number of local offices, it should be easier for the lead forecasters from the national centers and local offices to communicate and discuss the forecast problem(s) of the day.
The kind of forecaster needed in the NWS structure that we envision will be much different from that which exists now. Furthermore, the institutional support will also have to change dramatically. We believe that a "Top Gun" approach to the selection, training, and monitoring of forecasters is absolutely essential. A system must be developed to identify the very best forecasters, train them, and monitor their progress. Such systems already exist in the public sector in areas such as the air traffic controllers of the Federal Aviation Administration. In many respects, the situation of air traffic controllers always on the alert for dangerous conditions and the operational meteorologists forecasting rare, severe events are similar. Although automated techniques can handle a large part of their duties, the human element remains essential, especially in situations in which failure to make a correct decision can result in significant danger to public safety.
Substantive training is a critical step in the process. Since one effect of implementing our proposal would be a reduction in the number of meteorologists in the NWS, significantly more effort could be applied to the task of training entering meteorologists. Little effort has been made to date at identifying the traits that make a good weather forecaster. In fact, to our knowledge, no studies documenting how human beings make weather forecasts exist to show what skills, if any, good forecasters have in common. This seems to be an obvious place where effort should be expended if we are to expect human forecasters to make a difference in the quality and value of forecasts.
During the initial training course, substantial time must be spent testing and evaluating a candidate's skill in forecasting. Trainees must be tested rigorously and repeatedly on the real problems they would face on a daily basis. It should be anticipated that a relatively large fraction of the trainees will fail and not become forecasters in this system. This aspect of the process is important and serves at least two purposes. The most obvious is that it prevents people who cannot do the job from having that responsibility. The less obvious benefit is that it gives the successful candidates the psychological assurance that they can do a job that few others can do. This boost to their confidence may be an important part of enabling them to do their jobs well. A training program that leads to certification of forecasters is needed. It would boost public confidence in the NWS and serve as a goal for aspiring forecasters.
Training in the application of meteorological knowledge to the task of operational forecasting is a non-trivial task, even for well-educated meteorologists. It is hard to see how such a training course could be shorter than six months - - - more likely on the order of 12 to 18 months. Once forecasters pass through the course and go on duty as forecasters, it is absolutely critical to monitor their performance. A distributions-based verification program (Murphy and Winkler 1987, Brooks and Doswell 1996) needs to be adopted that will identify strengths and weaknesses of forecasters and forecasting systems. It will help show areas where resources can be applied effectively to improve forecasts. Forecasters whose skills diminish, for whatever reason, will have to go through refresher training or lose their certification.
Weather forecasting in the public sector, even in a country such as the United States where competition with the private sector is extremely limited, sets the tone for the remainder of the field. There are definite implications for the rest of meteorology for a public sector such as we have described.
Limiting the kinds of forecasts made by the public sector clearly increases opportunities for the private sector. Given that the public sector will continue to produce forecasts of basic weather elements, based almost entirely on the NWP output, the private sector will be able to package that output and provide detailed, tailored forecasts for specific users that either improve upon the NWP products or make it easier for users to interpret. We believe that the private sector will be increasingly driven by economic pressures, both in the kinds of forecasts that are made and in the evaluation by clients of those forecasts. Value, in the sense of Murphy (1993), will be an increasingly important factor for the private sector. Sales of forecasts will depend heavily upon the ability to demonstrate economic impact of the forecasts. As a result, verification of weather forecasts will be a more important issue.
Education of forecasters will need to focus more on the solving of special forecasting problems, with connections between theoretical and practical concepts made clear. Techniques for the verification of weather forecasts will need to be a part of the educational process, as both public and private sector forecasters will need to show that they are making a difference in the forecast process if they are going to continue to be a part of the process. Methods such as cost/benefit analysis will be necessary for almost all successful meteorologists.
It is possible that highly specialized forecasters may find niches for their skills, particularly in the private sector, but a broad meteorological education will be essential for success in most areas. This will be especially true for those forecasters who desire to be the "Top Guns" in the public sector. Holes in their scientific education will have serious consequences for their ability to succeed in training and on the job.
Researchers will find their agendas dominated by the major themes of the public and private sector forecasting, weather that threatens public safety and weather that threatens economic well-being. Initiatives for research will need to be focused tightly, rather than merely efforts to understand a particular phenomenon better. Researchers are likely to find themselves being asked to do research that has direct applicability to forecast problems. Efforts to tap the private sector for funding of research will also be more important, even for public sector researchers. For example, insurance and reinsurance companies are concerned about economic vulnerability due to increasing threats to life and property from weather hazards. Since they have an economic stake in better forecasting, it seems logical that they should provide support.
The field of weather forecasting is in the midst of a revolution. The slow response of institutions to revolutionary change makes it difficult for them to focus on the ultimate goal of improved services. We have presented a vision of the future in which human beings continue to have a major role in weather forecasting. If that role is to be more than as caretakers and developers of technology, radical change in the institutional structures of forecasting are needed. If such changes do not occur, the Reconstruction Era following the revolution will be one in which forecasters have no control over their destiny and find themselves increasingly irrelevant to society.
In particular, we want to thank Bob Maddox (NSSL), with whom we all have had discussions about the nature of weather forecasting. Discussions over the years with Allan Murphy about the human element in forecasting and adding value to numerical products have also been stimulating. Many other people have contributed to the ideas that have been developed here, some of them without even realizing it or agreeing with them! We thank and apologize to all of them, as appropriate.
Brooks, H. E., and C. A. Doswell III, 1996: A comparison of measures-oriented and distributions-oriented approaches to forecast verification. Wea. Forecasting, 11, in press.
Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281-293.
_____, and R. L. Winkler, 1987: A general framework for forecast verification. Mon. Wea. Rev., 115, 1330 1338.
Roebber, P. J., and L. F. Bosart, 1996: The contributions of education and experience to forecast skill. Wea. Forecasting, 11, 21-40.
Vislocky, R. L., and J. M. Fritsch, 1995: Improved model output statistics forecasts through model consensus. Bull. Amer. Meteor. Soc., 76, 1157-1164.