With the advent of the Unified Forecast System (UFS), the operational model development at the National Weather Service (NWS) is increasingly becoming a collective effort that includes contributions from multiple NOAA labs, national agencies, and university research groups. The NWS forecasters are the primary stakeholders of the UFS’ operational forecast products, and therefore, continued engagement with them is important for staying abreast of the real-world performance of the model and its utility in society. Feedback from forecasters is another a key driver that informs the NOAA funding offices on model development priorities. For the purposes of these modeling-oriented workshops, the NWS Center and Regional Offices served to represent the broader state, local, and private forecast community.

Figure: Engaging stakeholders in the UFS: Feedback from forecasters and other stakeholders help us better understand model performance and errors, and thereby define development priorities. For a successful community model development and transition to operations, close coordination between (1) the UFS research and development efforts, (2) the operational model development and implementation organizations (NCEP’s Environmental Modeling Center and Central Operations), and  (3) the forecasting community is essential. 

The NWS Office of Science and Technology Integration (OSTI) Modeling Program Division conducted three workshops for forecasters during 2020-21. The key objective was to identify forecasters’ top priorities and modeling gaps, particularly focusing on the UFS Medium Range Weather/Subseasonal to Seasonal (MRW/S2S) global and Short Range Weather (SRW) regional Applications (See more on UFS Applications here).

The first workshop (held on November 16, 2020) solicited forecasters’ concerns, which were organized under a set of topics including convection, winds and terrain issues, precipitation, floods and hydrology, tropical cyclones, visibility, marine/coastal issues, temperature and air quality, and space weather. These concerns were later synthesized and shared with the UFS model developers and evaluators. The follow-up workshops held on January 29, 2021 and February 11, 2021 further addressed these concerns as relevant to the  evolution of the UFS Medium Range Weather (MRW) and Short Range Weather (SRW) Applications, respectively. These two follow-up workshops, led by the UFS Application leads, along with operational model developers and evaluators at the NCEP EMC, focused on connecting the forecasters’ concerns with model-specific issues and known biases.

The key outcome of the workshop is a finalized list of forecasters' requests for the UFS MRW and SRW Applications that consists of a total of 23 issues classified under 7 major topics, which are 1) Surface temperature and moisture, 2) Precipitation, 3) Convection, 4) Winds, 5) Tropical cyclones, 6) Marine waves, winds, and sea ice, and 7) Space weather. 

Overall, the workshops elevated the UFS community to a new level of synergetic coordination that strengthened the link between model development and forecasting challenges. Discussions between the forecasters and the modelers proved fruitful in that many forecast issues that appeared as independent disconnected items at the beginning of the meeting could later be tied to a few underlying model issues. Some of the development priorities that emerged are Boundary Layer (BL) over the land and ocean, air-sea coupling, land-surface processes, land initialization, microphysics and sea ice, and marine winds and waves. The Model Evaluation Group (MEG) at the NCEP EMC, played a pivotal role in bringing some of the key underlying issues to the forefront.

While the above list is a starting point, the UFS community and NOAA Program Offices are committed to continuing the conversation and facilitating opportunities for ongoing engagement between the modeling and the forecaster communities.

The contributing author team includes
Hendrik Tolman, NOAA / OSTI,
Deepthi Achuthavarier, NOAA / OSTI,
Geoffrey Manikin NOAA / EMC,
Jason Levit, NOAA/ EMC,
Linden Wolf, NOAA / OSTI, and
Farida Adimi, NOAA / OSTI.

The DTC is at an interesting pivot point in its 19-year history as it has the opportunity to redirect  its efforts back to its original imperative  of advanced numerical weather prediction (NWP) testing and evaluation. This opportunity is due to the establishment of the Earth Prediction Innovation Center (EPIC). As stated in the DTC charter, “The DTC was initiated in 2003 as a means for the NWP community to test and evaluate new models, technologies, and techniques for use in research and operations and serves as a bridge between research and operations to facilitate the activities of both halves of the NWP community in pursuit of their own objectives…”. Likewise, the goal of EPIC is to “Accelerate scientific research and modeling contributions through continuous and sustained community engagement to produce the most accurate and reliable operational modeling systems in the world.” The DTC and EPIC include a development environment, use of high-performance computing, code repository, observations and tools, and delivery of data with the latest codes, utilities, documentation, and user support.

From a distance, it appears that there is now some overlap between the DTC and EPIC, given that the DTC has been performing similar services as those envisioned for a fully established EPIC. In fact, the DTC’s role over several years expanded to include a larger fraction of user support and code management. So with these activities now shifting to EPIC, the DTC can rebalance its portfolio to focus its unique expertise on the critical areas of hierarchical testing and evaluation of community contributed model enhancements.    

The DTC leadership team has been working hard over the last two years to ensure that while EPIC is ramped up, the tasks previously performed by the DTC and planned for EPIC would transition to EPIC. Given the DTC’s successful history and extensive experience in user support, community engagement, code management, and documentation among other topics, it is imperative that EPIC take advantage of this experience. As this transition of tasks takes place, it is critical that the DTC and EPIC collaborate closely to define their respective roles in the advancement of NWP capabilities and clearly articulate these roles to the user community. Advancing the nation’s weather prediction capabilities and improving model performance skill requires that the DTC and EPIC work hand-in-glove in a tightly coordinated fashion. Annual work plans need to be synchronized to the greatest extent possible to optimize the roles of both centers. The leadership and governing bodies of both the DTC and EPIC need to consider tasking that is complementary. As stated by Maoyi Huang, EPIC Program Manager, Weather Program Office, NOAA, “DTC’s experience and expertise in code management, and user and developer support for the UFS weather code base and applications are indispensable assets to the community.” The community’s ability to effectively utilize the expertise and capabilities of both the DTC and EPIC is dependent on both centers working in unison and in a complementary manner.

Bill Mahoney is the Director of the Research Applications Laboratory at the National Center for Atmospheric Research. He serves as an Executive Committee Member of the DTC.  

Julie and her older brother grew up in South Florida. She worked her way through high school and part of college as a cashier at Publix Super Markets. She earned her college degree in accounting from the University of Central Florida (UCF). During her college years, she started working for the accounting department at the UCF Foundation, and by her senior year, she had earned a prestigious internship with Coopers and Lybrand.

In her freshman year at UCF, she met her future husband, Paul Prestopnik.  When Paul was accepted to CU’s graduate program, they both decided to make the move together to Longmont, where Julie took a job as the Assistant Vice President of FirstBank of Longmont.  The long hours and organizational frustrations of working at a bank soon set in. But, as is often the case, what seems like trouble at first can also turn into possibility.

Studying computer science became an attractive career choice, but it was a big risk to completely restart her career. Before doing anything drastic, Paul gave her some "homework assignments" to try out her aptitude for computer science. She enjoyed the challenge, and seeing how fulfilling Paul’s work was, she decided to make the change.

After getting married to Paul, in 2004, she took on a student assistant position at NCAR’s Research Applications Laboratory (RAL) working on the Dynamic Integrated Forecast (DICast®) System. A year later with a Bachelor’s Degree in Computer Science, she was hired as a Software Engineer I working in the Aviation Applications Program (AAP) on the InSitu Turbulence Project.

Since then, Julie has provided software engineering expertise and support to the Maintenance Decision Support System (MDSS) project, the Pikalert project, the Graphical Turbulence Guidance (GTG) project, and the Graphical Turbulence Guidance Nowcast (GTGN) project.  She started working in the Joint Numerical Testbed program, a collaborative facility within RAL that is connected to an international network of collaborators, on the Model Evaluation Tools (MET) project in the fall of 2012 and has been working solely with the METplus for about two years. 

Julie has led several RAL technology transfer efforts to external partners including the first successful transfer to NOAA's Weather and Climate Operational Supercomputing System (WCOSS) with the GTG code. Her software support of METplus with system installation, infrastructure, and maintenance, requires close collaboration with EMC at NOAA to install and maintain unofficial and official releases on various NCAR machines, NOAA machines, including WCOSS and WCOSS2, and community machines.   She feels fortunate to work with all of the seven METplus components (METplus wrappers, MET, METviewer, METplotpy, METcalcpy, METdatadb, and METexpress) as the documentation lead for the team.

Julie is deeply committed to RAL’s culture because she feels the organization has been such an excellent match for her work style and work/life balance. To reciprocate these cultural benefits, she recently joined the RAL Communications Committee, and is a member of the UFS Workflow team, and an NCAR level-recognition committee.  She attends the Technology Affinity Group (TAG) meetings, Forging Allies and Connections for Equity in STEM (FACES) Meet-Ups, the True Colors meetings, and the Diversity, Equity, and Inclusion (DEI) Book Club meetings when she can to deepen her understanding of RAL’s diverse and inspiring staff.

Spending time with family (12-year-old daughter), walking, running, hiking, reading, listening to her daughter play music (piano and trumpet), traveling, volunteering, going to the theater, sleeping, watching movies, playing games all replenish her reserves. As if her job wasn’t enough of a commitment, she also gives back in her personal life. Julie serves on the Board of Directors for her daughter’s middle school and co-leads her daughter’s Girl Scout troop.  She hopes the family can travel to Peru on a school trip, which would make three continents they’ve visited so far. Only four more to go!

Traditional radar product integrity suffers from gaps in coverage over large areas. While some geographical areas are fortunate to have ground- and/or satellite-based radar coverage, most areas around the globe do not have complete coverage from reliable radar networks. The use of radar-based products is advantageous for understanding current weather conditions and how weather systems may evolve over time. Convection and precipitating weather systems often impact critical missions for the United States Air Force (USAF); therefore, access to products that go beyond traditional radar output would be beneficial for mission planning and execution. To address this, the Massachusetts Institute of Technology Lincoln Laboratory (MITLL) has created the Global Synthetic Weather Radar (GSWR) product, which provides near-global coverage of radar-based outputs using advanced machine-learning techniques. To aid the USAF in making evidenced-based decisions when using GSWR products, the Developmental Testbed Center (DTC) performed rigorous veracity testing to compare GSWR products against several ground-based radar outputs to ensure the products were developed properly. The evaluation was performed using the enhanced Model Evaluation Tools (METplus) and was largely conducted on the cloud via Amazon Web Services (AWS). Both traditional evaluation approaches, as well as more advanced spatial approaches, including the application of MET’s Method for Object-based Diagnostic Evaluation (MODE), were employed.

The DTC’s evaluation was based on GSWR analysis data spanning from 15 May - 31 August 2020. These products included composite reflectivity, echo top, and vertically integrated liquid. GSWR products were compared against several observation and analysis datasets, including Multi-Radar/Multi-Sensor (MRMS) products, Next Generation Weather Radar (NEXRAD) products, Operational Programme for the Exchange of Weather Radar Information (OPERA) products.

For all three variables evaluated, GSWR analyses showed the highest skill at the lowest thresholds evaluated and the lowest skill at the highest thresholds evaluated; VIL typically showed lower skill than composite reflectivity and echo tops. Object-based evaluation provided value-added information beyond traditional measures. Key results from the object-based evaluation established that GSWR often produced fewer, larger composite reflectivity and echo top MODE objects when compared to the “truth” datasets and GSWR MODE objects typically exhibited weaker intensities and a south-west displacement. The results from this robust veracity testing activity were shared with the USAF and MITLL with the goal of providing information to make informed operational decisions as well as aid in the future development of GSWR products.

The Hurricane WRF (HWRF) has been the US operational hurricane forecast model since 2007. The Hurricane Forecast and Analysis System (HAFS), based on the GFDL Finite­-Volume Cubed-Sphere Dynamical Core (FV3), a scalable and flexible dynamical core, is the likely candidate to replace the operational HWRF. HAFS is the focus of a Unified Forecast System - Research to Operations (UFS-R2O) hurricane modeling project and the next generation hurricane forecast system. For the DTC Visitor Project, my study used satellite imagery to evaluate the HWRF and HAFS model forecasts. The tropical cyclone’s (TC) track and max wind of the models have been extensively evaluated, so to go beyond the track and max wind, we conducted a physics process-based evaluation of the hurricane vortex structure and size using the Geostationary Operational Environmental Satellites—R Series (GOES-R) data, which has global coverage, and visible and infrared bands for hurricane observing. The goal was to compare the model-generated synthetic satellite imagery to the observed satellite imagery to see how realistically they can be simulated and what systematic biases exist, thereby providing insight for model developers to further improve the model performance.

This past year, the DTC team evaluated and tested HAFS with two physics parameterization suites: the NCEP Global Forecast System (GFS) suite and the HWRF suite. The original goal of this DTC Visitor Project was  to compare the testing results to observations to determine which physics suite, GFS or HWRF, produces more realistic hurricane structure forecasts. The model synthetic imagery was created by the UFS Unified Post-Process (UPP). Thousands of images were compared, and the results revealed that when the HWRF physics suite was used in HAFS, the model generated exceptionally large hurricane vortices. See the figure below with an example of hurricane Dorian in 2019, located along the US East coast.

The HAFS is being actively developed at NCEP Environmental Modeling Center (EMC). A seasonal test of HAFS (using a highly modified version of the GFS physics suite plus other advancements such as coupling with an ocean model) was performed at EMC in the summer of 2021. EMC suggested that it would be beneficial to the HAFS model development if we could broaden the scope of the DTC visitor project  to include the evaluation mentioned above. To accomplish this, we had to develop a standalone software tool to read the model's archived atmosphere profile columns and surface features to create the model synthetic imagery, because the native output files from these tests, which are required for UPP to generate synthetic model imagery, had not been archived. We used Community Radiative Transfer Model (CRTM) v2.3 to  transform the model atmosphere properties and the hydrometeor profiles into synthetic model satellite imagery. This time, we focused on comparing the TC inner core (9 deg x 9 deg) against observations. We included the operational HWRF forecast in these comparisons. 

Tropical cyclones Dorian (2019), Laura (2020), and Teddy (2020) were compared.  As an example, Dorian's observed and model synthetic imageries from 18Z 2019-08-30 to 18Z 2019-09-06 are combined to form composite imageries; Figure 2 depicts the results. It appears that the HAFS model produced synthetic satellite imagery that was more accurate in terms of size and temperature than the HWRF model. This finding is also clearly demonstrated in the probability density function (PDF), where you can see that the PDF from HAFS and OBS matched well, but the PDF from HWRF result showed a large discrepancy from the other two, primarily because HWRF has a very large fraction in the cold end of the spectrum around 220 K, indicating that the HWRF model simulated a larger and colder vortex than OBS; the inner-core simulated HAFS, on the other hand, is slightly warmer than OBS.  These findings demonstrated that the HAFS improved the simulated vortex structure with more realistic size, structure, and brightness temperature.

As a former DTC team member, I've always enjoyed working with DTC colleagues to learn innovative ideas and techniques. Unfortunately, due to Covid-19, this year's visit was virtual, and I missed meeting the DTC team in person. Now, as an associate professor at the Coastal Carolina University's department of coastal and marine systems science, I teach coastal meteorology, hydrodynamics, and oceanography. My research focuses on the air-sea-coupled physical processes that occur in coastal areas during extreme weather events. The DTC software, such as the METplus, UPP, UFS, and CCPP, have been, and I believe will always be, a valuable tool that I use extensively in my research and teaching. 

Supporting the development of the Unified Forecast System (UFS) has been a central focus of the DTC for several years now. Many staff have concentrated on support, development, and training of this new state-of-the-art system that bridges the gap between the research and operational communities. In September 2021, we took another significant step in this process when the DTC hosted the first training event for the UFS Short-Range Weather (SRW) Application.

The UFS SRW Application Training Event was held 20–24 September 2021 and was attended virtually by more than 100 students, scientists, software engineers, and other professionals from more than a dozen time zones around the world. The goal was to teach those in attendance not only how to build, run, and customize the App, but also impart the overall vision of the UFS and invite them to provide code and scientific enhancements to the UFS community.

Over the course of the five-day, Monday through Friday training event, the attendees were treated to a wide suite of lectures from subject-matter experts representing the DTC staff and various NOAA labs and centers, including the Environmental Modeling Center (EMC), Geophysical Fluid Dynamics Laboratory (GFDL), and Global Systems Laboratory (GSL). On the first day, the overall vision of the UFS SRW Application in research and operations was explained, before delving into the basics of installing, configuring, and running the Application. More details about the pre-processing utilities were covered on day 2, along with several in-depth lectures on the atmospheric model and its dynamical core. Day 3 included details about prerequisite libraries, using different physics packages, post-processing and graphics/visualization. More in-depth topics were introduced on day 4, particularly geared towards scientific users and code developers seeking to make changes to the application and potentially contribute those changes back to the community code. Finally, Friday included presentations on future development from operational and research groups in EMC and GSL. 

In addition to lectures, hands-on sessions were conducted each day for a subset of the attendees to practice the installation and use of the application using the NCAR Cheyenne supercomputer. On days 4 and 5, open forums were held for discussion and questions, exploring topics such as computational performance, customization, and visions for future capabilities and adding new components to the system.

The training event may be over, but you can still benefit from the plentiful training materials. Videos of the lectures have been posted online on the agenda page, as well as slides. We invite anyone interested in the UFS SRW Application to take advantage of these materials, as well as other resources such as the Users Guide, Support Forums, and GitHub page.

Despite the challenges of COVID, the DTC Visitor Program is thriving. With the incremental opening of NCAR facilities, some visitors have been working on site, while others have been collaborating with DTC staff remotely.  We invite researchers to apply to the program as an opportunity to collaborate with us on a diverse range of exciting projects. Our program has hosted visitors who have made cutting-edge enhancements to DTC software and tools, model improvements, innovated verification methodologies, and tested and evaluated products. To learn more about our program and how to apply, please read our recently updated Announcement of Opportunity.   

See past Visitor Program articles that describe projects that supported operational weather prediction centers with advances in operational weather forecasting, and those that provided researchers with NWP codes that represent the latest advances in technology, including cloud computing tools and containers.

For more information about the visitor program, please go to: https://dtcenter.org/visitor-program