NCAR and NOAA are each adopting a unified approach to coupled environmental modeling, where success for both efforts is critically dependent on community contributions.  At the end of January 2019, NCAR and NOAA signed a Memorandum of Agreement (MOA) to develop a shared infrastructure that encourages the broader community to engage in improving the Nation’s weather and climate modeling capabilities. Collaborating on the development of a common infrastructure will reduce duplication of effort and create common community code repositories through which future research advances can more easily benefit the operational community.  NOAA will also be able to leverage NCAR experience to provide community access and support for NOAA’s operational models and tools. The MOA focuses on seven key elements of common infrastructure for the NOAA Unified Forecast System (UFS) and the NCAR Unified Community Model (UCM).

Coupling between Components - NCAR and NOAA have already developed an initial design for a new framework for coupling component models. This common mediator framework will ultimately accommodate evolving model coupling strategies, facilitating community research contributions and accelerating the transition of research into operations.

Coupling within a Component - A flexible framework for a common interface that allows interoperability / integration of physics packages within component models offers many near term and longer term benefits.  NCAR and NOAA are initially focusing on implementing such an interface for the community atmospheric models. The Common Community Physics Package (CCPP) developed by the DTC’s Global Model Test Bed (GMTB) through NGGPS funding, which is being implemented in NOAA’s atmospheric models, has paved the way for a collaborative approach with NCAR’s Community Physics Framework (CPF).

Workflow - Workflow in this context refers to all the infrastructure, code and datasets needed to configure, build and run an end-to-end forecast system utilizing a coupled model for a specific application.  NCAR’s well-documented and user-friendly workflow infrastructure known as CIME (Common Infrastructure for Modeling the Earth) and its Case Control System (CCS) makes coupled Earth System Modeling easily accessible in the face of increasing complexity. NCAR and NOAA have already begun exploring a common workflow infrastructure using a CIME/CCS based approach to provide a portable workflow for the larger community.

Quality Assurance Testing - Testing is critical to ensuring software quality and that code performs as expected.  Given CIME already contains elements of quality assurance testing, NCAR and NOAA are planning to adapt CIME and the CCS to provide a common testing framework for both research and operations.

Forecast Verification - Using the same toolkits in research and operations, as well as weather and climate, reduces duplication of work and accelerates the transition of innovations from research to operations. Supporting sustained improvement of coupled models will require tools that provide approaches more relevant to research, as well as considering output relevant to coupled processes.  The Model Evaluation Tools (MET), a community-supported software package, is a comprehensive set of tools for diagnostic evaluation of atmospheric models that can be expanded to address environmental component models beyond weather such as ocean, waves, and sea-ice. This expansion will take advantage of existing evaluation packages for other component models.

Software Repository Management - Effective community software development requires open access repositories.  All infrastructure and supporting code developed under this MOA will reside in open-access GitHub repositories. The management of these repositories will enable collaborative development across the wider research community and include governance, quality assurance, and workflow tools.

User and Developer Support - A robust infrastructure for providing user and developer support is key to engaging the broader community in advancing the capabilities of both NCAR’s UCM and NOAA’s UFS. This infrastructure will leverage existing practices and protocols developed for the NCAR’s community models (CESM, WRF and MPAS), as well as support efforts provided by the DTC for NOAA’s community codes (GSI/EnKF, HWRF, UPP and CCPP) and MET, to provide active and passive user support for the UFS and UCM.

The result of this MOA will be a state-of-the-art, well-documented and easy-to-use modeling system. Coordinating existing and ongoing investments and governance between NOAA and NCAR ensures alignment with unified coupled community modeling, sets joint priorities and leverages resources.

https://www.noaa.gov/media-release/noaa-and-ncar-partner-on-new-state-of-art-us-modeling-framework

https://www.weather.gov/media/sti/nggps/18-064553_SignedMOU.pdf

We stand at the dawn of the operational FV3 era, representing the first step in the implementation of NOAA's Next-Generation Global Prediction System (NGGPS). With this new era comes many challenges, but also many opportunities to work collaboratively to improve the operational modeling system. How can NOAA best work with the community to leverage these opportunities and develop a truly world-class weather and climate prediction system? A few ideas:

Develop a decadal vision for world-class numerical weather prediction.

As FV3 becomes operational, now is the time for NOAA and the community to look forward, collaboratively, through the 2020s. Already, the field is advancing toward global convection-allowing weather prediction and ensemble data assimilation, fully coupled atmosphere-ocean modeling, and unified physical parameterizations. These are all likely to be ready for operations by 2030. Getting there, however, requires collaboration across the weather enterprise. I encourage NOAA to identify the visionaries within the field who can bring the community together to develop a vision for the operational modeling system of 2030, establish infrastructures and promote collaborations in support of this vision, and advocate for the long-term investments necessary to see it to reality.

Support applied research at lower readiness levels that have a high potential for transformative advancements.

Many of NOAA's collaborative funding programs target applied research with intermediate-to-high readiness levels and anticipated pathways to operations within two to five years. The advances resulting from such research are undoubtedly valuable. Yet, outside of limited base funding, there is arguably insufficient support for applied research at lower readiness levels with higher risk but concordantly high potential reward. This poses a significant opportunity cost: the few funded initiatives remain under development for lengthy periods of time, and many worthy endeavors never see daylight because the resources to advance their development do not exist. Regular funding opportunities for early-stage applied research tied toward the decadal vision would help to develop a truly world-class weather and climate prediction system.

Support collaboration across the operational, university, and research communities.

Fortunately, there are many examples of effective collaborations between NOAA and partners/stakeholders across the weather enterprise, with the NGGPS initiative representing one such example - albeit one that started with significant disagreements on what its focus should be, and continues today without perhaps as much buy-in across all interested parties as is needed. Unfortunately, NOAA's ability to support these collaborations is resource-constrained; as new opportunities have come, so too have existing opportunities been lost. Establishing a decadal vision for world-class numerical weather prediction may provide NOAA leadership with a cohesive plan that can be used to advocate for additional support for collaborative research.

To be sure, there are many examples of successful collaborations between NOAA and the broader community toward advancing its operational modeling systems, and the DTC is both an example and a crucial facilitator of such collaborations. With some tweaks, however, I feel that NOAA's collaborations can become even more effective. As a community, let's begin to have the needed conversations to help make this a reality.

Gerard Ketefian tries to “get big computers to tell him if it’s going to be sunny or rainy or snowy tomorrow, and the next day, and the day after that, and…” that’s how he explains his job to a child - and he has two -- a 2-year old boy and 4-year old girl.

Gerard’s background is engineering. He has a B.S. in civil and mechanical engineering and M.S. in environmental engineering with emphasis on environmental fluid mechanics (both air and water). Gerard’s Ph.D. involved computational fluid dynamics applied to atmospheric dynamics.  His first job after graduate school involved modeling of hydrodynamics -- modeling of transport in San Francisco Bay. When the Cooperative Institute for Research in Environmental Sciences in Boulder, Colorado offered Gerard a position in NOAA’s Global Systems Division (GSD), he decided to return to atmospheric applications.  

Gerard currently works on two DTC tasks - the UFS-CAM project (Unified Forecast System - Convection Allowing Model) and the MERIT project (Model Evaluation for Research Innovation Transition). He is currently developing a community workflow for the FV3-Stand-Alone Regional Model (FV3-SAR), and implementing new features such as getting initial and boundary conditions from sources other than the GFS, enabling a new and more uniform grid, and enabling the use of the Community Common Physics Package (CCPP) to swap physics parameterizations easily. And of course, documentation of every step. What is rewarding about his work? Gerard says, “Finally getting a piece of code I've been working on for many days or weeks to do what I intended it to do!”

Gerard looks forward to the days when his kids are old enough to enjoy travel, and he hopes to spend time in Europe and South America. “Travel gives me the opportunity to get out of my daily routine and to put myself in a completely new environment.  It's rejuvenating.” But for now, if he could have a superpower, it would be to be able to slow down time, just for himself.

With funding from the Next Generation Global Prediction System (NGGPS) initiative and broad support from the community, the National Centers for Environmental Prediction (NCEP)/Environmental Modeling Center (EMC) recently replaced the dynamic core in its flagship operational model, the Global Forecast System (GFS). Version 15 of the GFS (GFSv15), implemented in operations on June 12, 2019, includes the Finite-Volume Cubed-Sphere (FV3) non-hydrostatic dynamical core in place of the long-running spectral hydrostatic core. This modeling system provides a fundamental early building block for the emerging Unified Forecast System (UFS) that is envisioned to be a full community-based Earth-System model.

The next major upgrade of the GFS, scheduled for 2021, is expected to include significant changes in model physics, posing the UFS community with a variety of challenges. Individual physical parameterizations need to be upgraded or replaced to produce superior forecast performance. Additionally, the physics suite needs to be well-integrated so that information is correctly transferred among parameterizations. Finally, the suite needs to run within the time available in the operational computing platform.

To address these challenges, three suites were identified as possible replacements for the GFS v15 suite (Suite 1). Suite 2 is the most similar to the operational suite, containing a single parameterization replacement, the planetary boundary layer (PBL) scheme. Suite 3 contains two parameterization replacements (the convective and microphysics schemes), harnessing development conducted at multiple research centers and universities, including Colorado State, Utah, NASA, NCAR, and EMC. Suite 4 contains five parameterization replacements, as it is derived from the operational RAP/HRRR modeling system, which was developed by NOAA Global Systems Division from years of community contributions through the WRF community modeling system for mesoscale applications.

In addition to the differences in physics listed in the Table above, it should be noted that the forecasts by the various configurations differ in a few other aspects, including dynamics settings and computational platforms. Additionally, Suite 4 uses the Common Community Physics Package (CCPP) as a demonstration of the UFS’s new paradigm for integrating physics and dynamics.

Runs were conducted between December 2018 and February 2019. Each suite was applied in a total of 163 model initializations, and performance for 10-day forecasts were compared objectively and subjectively. The initializations included 16 high-impact events selected by EMC’s Model Evaluation Group (MEG) along with an additional 147 dates from all seasons in 2016 and 2017. The DTC’s Global Model Test Bed (GMTB) and EMC collaboratively conducted the model runs and the output was analyzed using EMC’s verification statistics database (VSDB - the basis for all model upgrade decisions), the Model Evaluation Tools (MET) package, and a comprehensive MEG evaluation. Additionally, GMTB produced diagnostic analyses focusing on tropical cyclones, precipitation characteristics, spectral decomposition, and boundary layer properties. These diagnostic and statistical summaries were examined by an impartial panel of experts to inform their formal recommendation for next steps to EMC. Consistent with the panel’s recommendation, EMC’s final decision was to use suite 2 as the basis for developing a prototype configuration for the next GFS implementation (GFSv16). Specifically, EMC has configured this prototype with the PBL parameterization that distinguishes suite 2, along with already planned upgrades to parameterizations for gravity wave drag, land, and atmospheric radiation - and a doubling of vertical levels with extension of the model upper boundary to the top of the mesosphere. Optimization and development of this prototype in a fully cycled system will proceed in coming months, in anticipation of an early 2021 operational implementation.

For more information about this test, visit the DTC website at https://dtcenter.org/eval/gmtb/phytest2019/.

Jacob Radford, a Ph.D. student at North Carolina State University and visitor to the DTC during June of 2018, investigated the forecast skill of the High-Resolution Rapid Refresh (HRRR) model for banded snowfall events. In particular, he evaluated the HRRR’s ability to capture the location, areal extent, orientation, and aspect ratio of these locally enhanced regions of reflectivity. In theory, snowbands should be adequately resolved by the HRRR thanks to its fine grid-spacing, but model skill has not yet been assessed quantitatively.

Snowbands, or narrow regions of intense snowfall, present hazardous travel conditions due to rapid onset, high precipitation rates, and reduced visibility. Though there has been no quantification of the societal or economic impacts associated with snowbands, in particular, the economic costs of heavy snow events are estimated to be in the billions. Furthermore, mesoscale precipitation bands account for a significant portion of annual precipitation and were found to occur in a majority of cold-season precipitation events in the Northeast and Central U.S. Because these snowbands are small in scale, even the mere occurrence of snowbands is difficult to predict, not to mention the timing, location, and intensity. Thus, there is great incentive to improve understanding of the environmental conditions, physical processes, climatologies, and predictability of snowbands.

Jacob utilized the DTC’s Method for Object-based Diagnostic Evaluation (MODE) to match snowbands in the HRRR’s 1000-m reflectivity field to bands in national mosaicked base reflectivity fields. However, snowbands were defined based on local reflectivity heterogeneity rather than a set reflectivity threshold, a capability not possible in the current iteration of MODE. Thus, the primary goal of Jacob’s visit was to enable greater flexibility in MODE object identification. Jacob worked with Model Evaluation Tools (MET) developers Tara Jenson, Jamie Wolff, John Gotway, and Randy Bullock to implement these changes in MODE. The team quickly determined that the most practical way to accomplish this feat was to build a Python interface for MODE, allowing users to define objects in a Python script however they see fit and then input these objects into MODE via Numpy arrays or xarrays. This added functionality is a significant stepping stone for MODE user flexibility and is available as of MET v8.0.

With MET v8.0, Jacob could then define snowbands as a narrow region of reflectivity at least 1.25 standard deviations above the local reflectivity background, identified in HRRR-simulated and observed reflectivity, and matched with MODE based on similarity in location, area, orientation angle, and aspect ratio. The distributions of interest scores, or measures of the observation/forecast similarities based on these four properties, are shown in Fig. 1. The median interest score of 0.66 indicates that while the HRRR demonstrates some ability to match observed snowband cases, there are significant errors in at least two of the four interest parameters. Applying a cutoff of 0.70 to correspond to a reasonably well-forecasted case, only 30% of cases were well-forecasted by the HRRR. Ultimately, while the HRRR may be helpful in identifying areas of heightened snowband risk, it lacks forecast precision in location and timing. The next step in this work will be to apply a similar verification procedure to the HRRR ensemble to evaluate probabilistic snowband forecast skill.

Jacob found collaboration with the DTC to be an extremely valuable experience vital to the completion of his Master’s research and a step towards his Ph.D. Everyone at the DTC was extremely friendly, accommodating, and knowledgeable about forecast verification. While Jacob wasn’t at the DTC, he spent his time exploring the Flatirons and Rocky Mountain National Park.

The Model Evaluation for Research Innovation Transition (MERIT) project provides a critical framework for physics developers to test innovations within their schemes using selected meteorological cases that have been analyzed in depth.  Comparing their results to baseline MERIT simulations will allow developers to determine whether their innovations address model shortcomings and improve operational numerical weather prediction.

For the DTC’s AOP 2018, three high-impact global FV3 baseline cases were selected for in-depth analysis: the Mid-Atlantic blizzard of January 2016, Hurricane Matthew, and the May 2017 severe weather outbreak in the Southern Plains.  These cases were chosen after consultation with the Model Evaluation Group (MEG) at NOAA’s Environmental Modeling Center, as each case exhibits known deficiencies in the global configuration of the Finite-Volume Cubed-Sphere (FV3) model.  Multiple-day simulations were run using an end-to-end workflow developed to handle the pre-processing of initial conditions, the integration of the model, post-processing with the Unified Post Processor (UPP), and verification with the Model Evaluation Tools (MET).  Also, in collaboration with the MEG, the MERIT team has been working on developing and applying unique verification techniques and metrics that will help assess the impact that physics innovations may have on these known FV3 biases. In particular, the progression of certain meteorological features will be assessed through the MET Method for Object-Based Diagnostic Evaluation (MODE) time-domain/storm-relative feature analyses.

The initial focus of MERIT continues to be on existing capabilities available in the global model framework. However, this activity is expected to include high-resolution/convection-allowing modeling as the Stand Alone Regional (SAR)-FV3 becomes available. Providing the research and operational communities with an end-to-end framework will streamline the testing process, leading to more effective and efficient physics development.  In addition, it will also encourage community engagement and provide an infrastructure that supports R2O and O2R. 

See https://dtcenter.org/eval/meso_mod/merit/.

Model Evaluation Tools – Tropical Cyclone (MET-TC) was first developed in 2012 by the Developmental Testbed Center (DTC) as part of the Hurricane Forecast Improvement Project (HFIP) to provide a set of tools to aid in tropical cyclone forecast evaluation and verification. The primary goal of MET-TC was to replicate the functionality of the National Hurricane Center (NHC) verification software while utilizing the modular tools available within the MET software framework. The MET-TC software was first released with METv4.1, making the tools available to all users and enabling consistent forecast evaluation studies to be undertaken across the tropical cyclone modeling community. The initial tools provided a standard set of verification metrics and comprehensive output statistics with flexibility for user-defined stratification, which can be used for homogeneous comparisons of operational and experimental track and intensity forecasts. Graphical capabilities are also included with MET-TC, allowing easy visualization of verification statistics. Since the initial release, the tools have expanded through several DTC and non-DTC projects. Notable additions include:

• Rapid Intensification (RI) verification capabilities with user-defined thresholds (rirw)
• Ability to read Automated Tropical Cyclone Forecast (ATCF) Probability/Error Data Files (e-decks) and apply probabilistic methods to RI probabilities (-edeck, probrirw)
• Masking according to track and a circle of configurable radius around one or more lat/lon locations (gen_vx_mask)
• Application to shift gridded data field by source and destination lon/lat values (shift_data_plane)
• Feature-relative evaluation (METplus application)

New development is on the horizon to further enhance the tropical cyclone verification capabilities within the METplus system, these include:

• A new tool to convert model output to storm-relative coordinates and allow users to normalize by radius of maximum winds, compute azimuthal averages, derive tangential and radial wind (tc-rmw)
• A new tool to verify deterministic and probabilistic tropical cyclone genesis forecasts (tc-gen)
• Support for reading Statistical Hurricane Intensity Prediction Scheme (SHIPS) diagnostics files