The U.S. Air Force’s partnership with the DTC dates back to 2003 and one of the main areas of collaboration has been the advancement of scientific and technical research into a community-backed set of tools for model verification known as Model Evaluation Tools (MET) and its subsequent evolution into the METplus suite of tools. The USAF has provided financial as well as in-kind participation in governance to the DTC and in return benefits from the expansion of this tool set across many operational numerical weather centers from the UK Met Office and its partners, NOAA/NWS, and others.
The USAF operates both global and regional atmospheric models, including deterministic and probabilistic solutions, leveraging external collaborations and state-of-the-science solutions. The products that are produced must be verified and the results communicated effectively to Air Force leadership and to the downstream warfighters generating actionable insights daily. Over the last few years, the 557th Weather Wing (the Air Force’s operational modeling center) has moved its modeling operations to a new high performance computing environment at Oak Ridge National Lab and is now leveraging the METplus framework on that system. Previous instantiations of operational model verification involved a series of scheduled calls to older versions of MET via many scripts resulting in a cluttered repository and hard to trace orchestration. In the new environment, a Development Security Operations approach using a Continuous Integration and Continuous Deployment (CI/CD) model has been employed for all software development, including these new Model Verification Tools (MVTK).
The Cylc workflow engine used by the 557 WW, UK Met Office and their partners is an orchestration tool for modeling which can initiate jobs based on events or at specific times. Using Cylc as the backbone for MVTK allows for routine triggering of the necessary observation and modeling pre-processing METplus jobs upon file receipt. Subsequently, the main statistics modules contained in MET are run and the resultant output aggregate files are sent to a shared file repository. Figures are generated using home-grown python scripts and are scheduled in Cylc at desired intervals, including near real-time and monthly/seasonal aggregation. In future operations, containerized versions of METplus will become the standard for upgrading our model verification software stack across multiple computing platforms and a streamlined end-to-end approach leveraging METplus analysis tools and METviewer. The USAF is looking forward to continuing to work with DTC to benefit from enhancements to METplus.
Our previous partnerships with DTC can be viewed as three pronged: support to 557 WW users and their requests for capabilities (described above), cybersecurity, and veracity testing. Creating a cybersecure METplus that can be run in the Air Force computing environments has proven to be a big lift, but we hope that the hard work of DTC personnel will benefit all users of METplus with a more secure codebase. While the 557 WW continuously verifies operationally running models, our acquisition corps, planners, and leadership often want to know what benefit new innovations are bringing to the Air Force Weather Enterprise – this is where veracity testing by DTC plays a role. In the past we have exercised options to evaluate new methods for forecasting dust and flooding. We are very excited to include all three activities from our previous support to DTC in our next contract, covering fiscal years 2024-26.
