Preprints, Twelfth International Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, 28 January - 2 February 1996, Atlanta, GA, 180-185.
Susan Williams* and Darien Davis
NOAA Forecast Systems Laboratory
Boulder, Colorado
*contract with the Cooperative Institute for the Environmental Sciences
Table of Contents
During the past two years, the Forecast Systems Laboratory (FSL) has been developing a UNIX-based workstation incorporating a two-dimensional integrated display of hydrometeorological data into a system with editing tools, forecast preparation, and three-dimensional displays. This ubiquitous workstation, known as the WFO-Advanced (MacDonald and Wakefield, 1996), has matured to a level useful in a forecasting exercise. One component of this system is the two-dimensional display, D2D, used for integrating several types of model, point, and image data for forecaster use. This paper addresses the development of this component, from a project manager/team leader perspective.
The development of a customized workstation from the ground level represents many challenges. Although FSL has a rich history developing such workstations for the National Weather Service and other collaborative agencies, many characteristics of the system are new: FDDI network, Hewlett Packard-9000 hardware running HP-UX OS, data acquisition and X-based display.
After assigning experienced programmers to the FX-Advanced project, the requirements phase began. The workstation would not degrade any of the services currently available at the Denver WSFO and would add many enhancements. The AWIPS site workstation was used as a building block for this system. Although not mimicking the AWIPS workstation, the major functionality of the AWIPS system was incorporated.
The requirements for D2D had been identified during previous efforts of development. The requirements identified display capabilities and data ingest functionality, as well as system support functions.
The spiral development method was used as a guideline for development. An overall general design addressing all requirements had to be done first otherwise early work might not be useful for later cycles. Cyclic phases included identifying requirements, design, code development, unit testing, integration, and evaluation (refer to Section 5.1). The results of the evaluation were then fed into the next cycle of development and affected the next requirements phase. Five such cycles of D2D were needed in preparation for the August 1995 exercise.
Coordination of requirements for display and data acquisition was carefully planned. It is very difficult to develop display capabilities for unavailable data. In fact, the data decoding/access functions were needed prior to starting the display development.
As the integration and testing was being completed in the cycle, efforts were underway to identify the next phase of requirements. Much care was taken to eliminate any possible bottlenecks in the development cycle.
Software engineers were assigned to the workstation display development or data management. However, additional personnel resources have played an important role in the success of developing this rapid-prototype workstation.
A programmer was assigned to develop an application layer on the interprocess communications mechanism. This gave us the flexibility of using a set of Application Programming Interfaces (API) and loosely coupling the implementation of the IPC.
A programmer was assigned to develop a sophisticated data and system monitoring mechanism. Once the system is deployed operationally, the WSFO in Denver will be able to identify system and display problems on-site.
Quality assurance is an important characteristic of this system. A person assumed this role de facto, and has been a tremendous asset to the success of this system. This job entails setting up a test plan after each development cycle, testing the requirements, regression testing, bug tracking, and leading evaluation efforts of the workstation.
The completion of the major goals for the workstation development hinged on three deliverables: an internal forecasting exercise using the new workstation in August 1995 (Roberts, et al. 1996), an October forecasting exercise with the six regions of the NWS, and the installation as an operational system in the Denver WSFO in the spring of 1996.
During September 1994, the requirements for the workstation display were identified and scheduled for completion -- August, October, or February 1996. The majority of the work was required by the first exercise. After the display requirements were identified, the support data were identified.
As mentioned, the initial development was a five-phase effort. In September 1994, the first set of requirements were identified.
The first cycle of development was Backbone-1 and a strong internal skeleton of the system was the first major goal. The definition of the database, utility libraries and routines, the interprocess communication mechanism, and the User Interface (UI) library modules were implemented. The requirements were simple -- display one satellite image, gridded model data (height, temperature, etc.) and a surface plot. Although the resulting workstation was very primitive, the internal structure was strong and easy to build on. The subsequent cycles added the interactive display capabilities that are customarily used for the weather workstation -- zooming, panning, animation, overlays, etc.
Although the interactive display portion of the workstation is the most visible part of the system, the areas of data acquisition, decoding, storage, and access (collectively referred to as "data management") are a larger coding effort than any of the display developments. Data acquisition development was largely delayed until after the October 1995 exercise due to insufficient resources. Since all data were available from another source, emphasis on decoding, storage, and access became the primary effort. The system was carefully designed so that it could plug in different acquisition streams in the future with minimal development.
The WFO-Advanced D2D workstation hardware configuration is based on a Hewlett-Packard (HP) platform consisting of HP-9000 Series 700 workstations. The system consists of four basic hardware components: a Data Acquisition Server, Database Server, Workstation Display and an X-Terminal for text data. The basic network is illustrated in Figure 1.
The system is networked using the Fiber Distributed Data Interface (FDDI), and disks are Network File System (NFS) mounted to the applicable machines (Grote and Kahn, 1996). For example, the display software accesses the decoded and stored data on the database server. Likewise, the data acquisition is NFS mounted to the database server so that text product arrival notifications are relayed.
Three environments were identified and configured: development, integration, and exercise systems. These hardware configurations allowed software engineers the freedom of rapid development, quality assurance to test software in a stable environment, and the user community to evaluate D2D for extended periods in a somewhat static environment.
The hardware configuration for the development environment consists of HP 755 workstations for the data acquisition and the database server. A mixture of HP workstations were used by software engineers for workstation display, data acquisition, and data management software development. These workstations include HP 710s, 715s, 735s, 750s and X-terminals.
The integration environment is configured to stage routinely built software at intervals driven by the software fixes or new features exhibited in the display, data acquisition or data management software. Ideally, the data management team ran stable decoders and storage in the integration environment. This was the dataset used for display development. This configuration consists of the four components of the basic network configuration exhibited in Figure 1. Typically this hardware configuration was used for quality assurance (testing) and demonstrations. The integration hardware was isolated from the development and exercise environment.
Prior to workstation deployment, an internal exercise simulating operations at a Weather Forecast Office was staged to evaluate system performance, reliability and solicit suggestions from the user community (Roberts, et al., 1996). The hardware configuration for this exercise includes all the components of the basic network with six display workstations and three X-terminals. Again this environment was isolated from the development and integration configurations.
The software development process followed the spiral method of overlapping requirement identification, design and implementation phases as illustrated in Figure 2.
Using this method of requirement identification, design, and implementation phases allowed D2D to adhere to the fast track which development milestones were achieved. During this process, two areas of software development were identified, the workstation display software and the acquisition of real-time data software.
The primary objective of the workstation display software development was to enable the user to access a highly effective and responsive workstation under dynamic conditions, i.e. access to real-time data. The workstation display software development centered around the following areas:
- Interactive Graphics Capabilities - the ability to manipulate displayed data
- User Interface (UI) - the ability to select menu choices, etc.
- Depictable Generation - the ability to scale and display data to predetermined limitations
- Application Generation - the ability to create a customized product via the use of tools
- Interactive Text Capabilities - the ability to create, edit and save text products, i.e. issuance of warnings
- Application Programmers Interface - the ability to create customized meteorological applications
- Monitoring and Logging - the ability to monitor the system through usage logs.
The principal objective of the data acquisition/data management software development focused on timely provision of data on the national and local scales in a readable format (Edwards, 1996). With this objective in mind, the data acquisition system was designed modularly by function. The major components of the data acquisition software development are:
- Communications Servers - provide interfaces between external data sources and D2D
- Communication Routers - receive data passively from the communication server
- Data Controllers - routes data from the communications routers to the appropriate decoders
- Data Decoders - decode data and store in a consistent format.
Commercial off-the-Shelf (COTS) products were purchased to allow the software development staff to produce quality code in a rapid prototype environment. These products include:
- Pure Software's Purify - checks for memory problems, i.e. memory leaks
- Pure Software's Quantify - locates performance bottlenecks in the code
- OpenWare's Object Interface - automatic layout of on screen widgets
- Digital Equipment Corporation's DecMessageQue - interprocess communication software package
- Local Data Manager (LDM) - system of software and protocols for data distribution and processing (UCAR, 1992)
- Informix - relational database for text product storage.
The software configuration control (SCC) methods for the D2D portion of the WFO-Advanced utilize two software configuration packages. Software configuration control was enforced to protect and monitor source code during the course of rapid software development. The two software configuration control packages used are based on Revision Control System (RCS).
REvision COntrol Unifying Related Source Environments (RECOURSE), a FSL-developed SCC package based on RCS, was initially utilized on D2D for source code control. The RECOURSE source code controls system supports the use of source trees and provides centralized control over master project files as well as tools for managing these files.
During the course of software development and integration of the D2D workstation with the other components of the WFO-Advanced, the decision was made to transition to the Baseline Configuration System (BCS). The BCS was the software configuration control method of choice for the AWIPS Forecast Preparation System (AFPS) because of its reliability and performance.
The BCS is a software package (free software) which provides a stable, potentially read-only software baseline and has multiple staging areas in which individual developers can work. Aliases were established for RECOURSE commands, defining the applicable BCS command which helped make the transition transparent to the user. The added advantage of BCS over RECOURSE was that the user could automatically see updates before software changes were released into the baseline.
Quality assurance procedures were implemented to insure the integrity of the WFO-Advanced D2D Workstation. These procedures include developing a test plan that directly correlates to the milestone requirements, extensive unit testing performed by software engineers prior to system integration (including the use of COTS evaluation tools), and tracking problems reported during the milestone testing phases.
Every design milestone is accompanied by a detailed set of requirements. From these requirements, a test plan is derived with test procedures written to verify that the requirements were satisfactorily met and any deficiencies were recorded and reported to management and staff. Prior to system integration and execution of the test plan, unit testing was performed to ensure that the function complies with the requirements set forth. Good communications between software engineers was paramount during the development phase to reduce the amount of recoding during integration and testing.
Nightly software builds were automatically executed via the UNIX utility crontab. A clean build prior to testing is essential. Once a clean build has been "pushed" to the integration hardware configuration, testing commenced. Dedicated networked hardware provided an isolated testing environment in which the test procedures mentioned were executed. Testing personnel consisted of the test coordinator and a meteorologist not associated with software development. Project software engineers were available for question or comments arising from testing and were present on an as needed basis. Each test procedure was signed off by testing personnel and observations and deficiencies noted.
The Bug Tracking and status accounting system implemented for D2D uses the Informix relational database. As a result of testing, bugs are entered into the database with an OPEN status and classified as to the severity of the problem. A status accounting report of OPEN bugs is issued to management and bug assignments are made to appropriate staff. Once the bug has been assigned, the bug is in a PENDING state. When a satisfactory fix has been applied, the software is released, rebuilt in the nightly master build, pushed to the test configuration and tested by testing personnel. If testing results are satisfactory, the status of the bug is deemed CLOSED. If the testing of the bug is unsatisfactory, the bug returns to an OPEN or PENDING state and the problem is given further consideration.
Several status accounting reports are available to management and staff. These reports include the following:
- Status of all the bugs
- Detailed information regarding one bug
- Status of bug in a particular state, i.e. OPEN, PENDING or CLOSED
- Status of bugs of a particular classification.
Although the software for the system was developed within the D2D group, services from electronic technicians, network managers, and system administrators were needed for staging a major workstation system. FSL's Facilities Division is dedicated to supporting the laboratory's networking needs.
The change from Ethernet to FDDI was led by a team of networking specialists and the system administrators. In addition, workstation moves and hardware add-ons are accomplished with the expertise from this division.
Two system administrators handle the functions necessary for upgrades, installations, and trouble-shooting of the operating system.
The support facility also has services providing data. A relay of data from the Telecommunications Gateway in Denver to the WFO-Advanced system was accomplished via a data management subsystem maintained within this division. This has been instrumental in the success of the August exercise, allowing the actual data acquisition software development to be delayed.
Several strategies became apparent during the development of the rapid prototype of the FX-Advanced workstation. They are:
- Use COTS software whenever possible. The cost of developing similar software is a very expensive solution and should be avoided whenever possible.
- The data must be available before the display code development can proceed. Make sure that the data are a cycle ahead of their display counterpart.
- Design is very important. Although managers can be somewhat apprehensive when only a paper trail exists, a proper design can severely reduce the amount of development time.
- Although not practiced by the D2D team, milestones should be at least four-month intervals. This gives proper time for design and design walkthroughs as well as code walkthroughs.
The rapid prototype development of the D2D posed many challenges to the project manager/team leaders. However, despite these challenges, the D2D milestones were met in the time frame outlined in the project plan and satisfactorily met the requirements set forth.
Edwards, G.J., 1996: Implementation of the data acquisition component of the WFO-Advanced 2-D Display (D2D), Preprints, Twelfth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography and Hydrology, January 28 - February 2, 1996, Atlanta, Georgia, Amer. Meteor. Soc.
Grote, U.H. and R. Kahn, 1996: Use of RISC-based workstation architecture and distributed network design in meteorological workstations, Preprints, Twelfth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography and Hydrology, January 28 - February 2, 1996, Atlanta, Georgia, Amer. Meteor. Soc.
MacDonald, A., and J. Wakefield, 1996: WFO-Advanced: An AWIPS-like prototype forecaster workstation. Preprints, Twelfth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography and Hydrology, January 28 - February 2, 1996, Atlanta, Georgia, Amer. Meteor. Soc.
Roberts, W.F., P. Kucera, C. Lusk, D. Walker and L. Johnson, 1996: 1995 Real-Time forecast exercise for WFO-Advanced, Preprints, Twelfth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, January 28 - February 2, 1996, Atlanta, Georgia, Amer. Meteor. Soc.
University Corporation for Atmospheric Research (UCAR), 1992: Site Manager's Guide for the Unidata LDM, v4. Unidata Program Center, Boulder, Colorado.
This document is maintained by Joe Wakefield.
Last updated 15 Feb 96