The FSL 1995 Real-Time Exercise (RT'95), conducted during the months of August, October, and November, evaluated the WFO-Advanced system (MacDonald and Wakefield, 1996), developed primarily at FSL. The major goals for the exercise were to provide feedback to developers on the various components and subcomponents of the WFO-Advanced system when used to support a pseudo-operational environment, and to determine whether components of the WFO-Advanced can support a full range of WFO activities.
The RT'95 exercise was the first rigorous test of nearly all components of WFO-Advanced. The primary components include an interactive two-dimensional display and text system (D2D), the Advanced Weather Interactive Processing System (AWIPS) Forecast Preparation System (AFPS), FSL's Local Analysis and Prediction System (LAPS) (Schultz, 1996), a three-dimensional visualization system (D3D), and the FSL Experimental Dissemination System (Subramaniam, 1996). The evaluation efforts will be directed toward gathering information on a broad range of topics. These topics include system reliability and performance, basic usability and utility of components, user interface, data presentation, screen resource management, training and documentation. Evaluation feedback will also be collected on other workspace applications such as the General Meteorological Software Package (GEMPAK) used to customize and manipulate data, and hydrologic applications.
Forecast shifts during the August segment of the RT'95 Exercise were scheduled for Monday-through-Friday operations. A single (day) shift approach was sufficient to test the system in August, since convective activity in Colorado typically begins by midday, peaks in the early afternoon, and moves eastward out of the forecast area during the early evening hours. If severe convective conditions did not occur for a period of time, modifications to the warning criteria were made to reduce warning thresholds. For example, warning thresholds were lowered from three-quarter inch to one-quarter inch for hail on some days. A determination was made at the start of each day whether to use the standard or modified warning criteria so forecasters could prepare products accordingly. Additionally, simulated spotter and public reports were occasionally phoned in.
Nearly all of the participants for the August exercise were from the Boulder/Denver area. Most of participants were actually FSL staff members, and a few NWS forecasters from Denver and Norman also participated. Each participant was scheduled for about one week of forecast shifts which were divided between the short-term and long-term desks. The August participants will be paired with NWS forecasters in the October/November exercise, which has not occurred at this writing.
Each forecaster received workstation training and a WFO-Advanced User's Guide before they participated in the exercise. The User's Guide contained self-paced training modules for forecasters to perform most workstation functions. Forecasters were strongly encouraged to complete these modules before their individual training sessions, which occurred shortly before their first scheduled shift.
Shift responsibilities were divided between one short-term forecaster and one long-term forecaster. In general, the short-term forecaster was responsible for current weather, warnings, and forecasts out to 12 hours. The long-term forecaster was responsible for forecasts beyond 12 hours. With this arrangement of responsibilities, both forecasters were responsible for parts of nearly all forecast products. For example, the long-term forecaster would compose all but the first period for the zone package then pass it to the short-term forecaster who would add the first period then transmit the final product. Forecast products and transmission times were based on the schedule from the Denver WSFO. Routine aviation and public products were composed using text word processing and data displayed two dimensionally (i.e., D2D). Warnings were composed using the WARNGEN application program, which uses a graphical user interface to output an initial textual product that the forecaster can modify prior to transmission. Forecast and warning products were not actually transmitted to the public but were printed and saved for review.
Forecasting will begin on October 12 and continue for a period of 5 and one-half weeks, ending November 18. Nearly continuous operations (6AM local time to 12PM local time, seven days per week) over this period is required for several reasons, including expected multi-day weather patterns, invited participants, and AFPS use. Nearly continuous operations enable forecasters using AFPS to initialize from previously prepared forecast grids. Further, this will allow testing of the broad range of WFO operational responsibilities.
Forecasters from each NWS region and Taiwan's Central Weather Bureau have been invited to participate and give their feedback on the various components of the WFO-Advanced. As mentioned earlier, invited forecasters and Boulder participants will be paired during the exercise. Each participant will be scheduled for forecast shifts over a two to three-week period. All forecasters will be rotated through both long-term and short-term desks and all three shifts (early, midday, and late).
The primary components to be tested are D2D and the other workspace applications. AFPS will be employed during part of the October/November exercise. In particular, forecasters will use AFPS on a limited basis for about one week then use it for most forecast preparations for the rest of their participation period (Terminal Area Forecast preparation will not be supported by AFPS during the exercise). One week of limited AFPS use is expected to help mitigate the effects of a very steep learning curve for forecasters who are unfamiliar with the system's components.
Each of these methods has various advantages and disadvantages in terms of cost to the evaluation team, cost to the forecasters and other staff, obtrusiveness in the operational environment, and data quality. The complementary nature of the methods help overcome the deficiencies inherent in each individual method, and as a whole provide a more complete picture. All of these methods have been used successfully in past evaluation activities. One new feature was an on-line evaluation window (E-window) which was incorporated to help facilitate participant's assessment of the WFO-Advanced.
Initially, we asked the forecasters to provide comments on the user interface of D2D, which included the Menu Bar layout, the various pull-down menus, dialogue boxes, and display controls. Several sections dealt with the visual presentation of the numerous datasets, from satellite and radar imagery, to Surface Aviation Observation plots, and profilers. Model fields, for example, can be displayed as images using graduated color tables, in addition to the standard contour charts. (The ability to display model information as images was unanimously applauded by the forecasters.) The general 5-pane layout and display manipulation were addressed in another section. We also asked the forecasters to rate and provide comments on other capabilities, such as: animation, stepping, fading, zooming, panning, toggling, magnifying labels, changing the density of the data, and changing color tables.
We obtained a number of comments on the Warning Generation capability, which allows the forecaster to identify a storm, estimate its motion (speed and direction), create and edit a warning box, and generate and edit the textual message. Each of these components was covered in some detail in the questionnaire.
Another section was devoted to the Text Workstation, in which we asked the forecasters to rate and/or comment on the layout, menus, and text data retrieval, as well as the WordPerfect editor and macros (designed at FSL). Last, we asked for feedback on training, documentation, and the RT'95 Exercise as a whole.
Forecasters provided a large amount of feedback on all aspects of the available workstation components in August in addition to preparing their scheduled forecast products. This information is currently being compiled by the FSL Evaluation team and will be fed back into the development process. Early feedback collected in the first few weeks of the exercise has already been addressed by the development staff. In some cases software bugs were identified and fixed. In other cases design changes were needed or features added. Developers also addressed these issues whenever feasible. This process continues in preparation for the second phase of the RT'95 exercise. A summary of the feedback from the entire exercise will be presented at the conference.