Over the years, the DTC has put in place several mechanisms to facilitate the use of operational models by the general community, mostly by supporting operational codes (for data assimilation, forecasting, postprocessing etc.) and organizing workshops and tutorials.
By stimulating the use of operational codes by the research community, composed of universities, NCAR, and government laboratories, several new NWP developments have been transitioned to NCEP operations. However, in spite of the relative success of the DTC, there are still significant gaps in the collaboration between the research and operational groups. The NITE project focuses on infrastructure design elements that can be used to facilitate this collaborative environment.
During the past year, the DTC received funding from NOAA to create a design for an infrastructure to facilitate development of NCEP numerical models by scientists both within and outside of EMC. Requirements for NITE are based on a survey of potential users and developers of NCEP models, information obtained during site visits to the NOAA Environmental Modeling Center, the UK Meteorological Office, and the European Centre for Medium-Range Weather Forecasting, discussions with focus groups, and reviews of various existing model development systems.
The NITE design has been developed with the following goals in mind:
- modeling experiments easier to run;
- a single system available to NCEP and collaborators;
- results relevant for R2O;
- reproducibility and records of experiments; and
- general to any NCEP modeling suite.
The following elements are included in the system design:
Data management and experiment database Scientists need access to input datasets (model and observations), a mechanism for storing selected output from all experiments, and tools for browsing, interrogating, subsetting, and easily retrieving data. To facilitate sharing information, key aspects of the experiment setup, such as provenance of source code and scripts, configuration files, and namelist parameters, need to be recorded in a searchable database.
Source code management and build systems Source code repositories for all workflow components need to be available and accessible to the community. Fast, parallel build systems should be implemented to efficiently build all workflow components of a suite before experiments are conducted.
Suite definition and configuration tools All configurable aspects of a suite are abstracted to files that can be edited to create the experiments. Predefined suites are provided as a starting point for creating experiments, with scientists also having the option to compose their own suites.
Scripts The scripting is such that each workflow component (e.g., data assimilation) is associated with a single script, regardless of which suite is being run.
Workflow automation system The workflow automation system handles all job submission activity. Hence, the scripts used to run workflow components do not contain job submission commands.
Documentation and training Documentation and training on all workflow components and suites are readily available through electronic means.
In addition to the elements above, standardized tools for data visualization and forecast verification need to be available to all scientists.
Next steps for NITE: Modernization of the modeling infrastructure at NCEP is very important for community involvement with all NCEP suites, and with the Next Generation Global Prediction System (NGGPS) in particular. The recommended implementation approach for NITE includes several phases, to minimize disruption to operational systems, and limit implementation costs, while providing useful, incremental capabilities that will encourage collaboration. Ongoing discussions between EMC and DTC, especially in the context of NGGPS infrastructure modernization, will likely lead to NITE implementation in the coming years.
