Building a Community through UFS Medium-Range Weather Application Users’ Training

Winter 2021

The DTC, in cooperation with subject-matter experts from NOAA's Environmental Modeling Center (EMC) and Geophysical Fluid Dynamics Laboratory (GFDL), as well as NCAR’s Climate and Global Dynamics (CGD) Laboratory hosted a live, virtual training session for the Unified Forecast System (UFS) Medium-Range Weather (MRW) Application 4-6 November and 9 November 2020. 

The UFS MRW Application targets predictions of global atmospheric behavior out to about two weeks. This training was designed to teach community users how to set up and run the latest officially released UFS MRW Application (version 1.1) for their own experiments.  The training comprised a wide range of sessions taught by highly experienced experts and developers in the field. Live lectures were presented by experts on the various UFS components, including the CIME-based workflow, the Finite-Volume Cubed-Sphere (FV3) dynamic core and physics suite options, and pre/post-processing. In addition to lectures, live virtual practice sessions were hosted to broaden experience with building, running, and modifying the system to take full advantage of the supported capabilities for research and forecasting.  Throughout the training, participants were able to interact directly with the SMEs to gain a deeper understanding of the system and how to configure it for their purposes.  The final day provided an optional "deeper dive" for developers that covered advanced subjects, including code modification, domain configuration, and repository management protocols.  The slides from these presentations as well as recordings, are available on the DTC website.

A total of 34 participants registered for the event, representing eight different time zones! Upwards of 54 participants took part in some sessions, with the average being 40-45 attendees including instructors. We had enthusiastic participation in all of the lectures, and the instructors were able to provide prompt and detailed answers to questions raised during the practical sessions using Google Meet and Slack.

Although a virtual meeting was not the preferred method for teaching the material, it nevertheless was a great success, according to the participants. Quotes from the feedback included,  "I've really appreciated the practicals and the rapid feedback in the Slack channels." and "I personally enjoyed this training very much and learned a lot. The materials really helped me consolidate some of my knowledge and skill for different components and functionalities in the UFS apps."

'Build diagram' discussed at the Unified Forecast System Medium-Range Weather Application Users’ Training

Introducing Undergraduate Students to NWP by Using Software Containers

Autumn 2020

Jamie Wolff, the lead of the software containers and cloud computing task in the DTC, contacted me in the Spring of 2019 about a unique opportunity to collaborate on a course that would offer interested students at Metropolitan State University an opportunity to run and experiment with an end-to-end numerical weather prediction (NWP) system utilizing cloud computing.

The motivation behind this collaboration was to introduce undergraduates of a bachelor’s degree Meteorology program to NWP in a stress-free environment in which students would not spend unnecessary time configuring, compiling, and optimizing the code and libraries for the NWP system.  The DTC containers would streamline running the NWP system so the students would be able to grasp how beneficial NWP could be in their toolkit. The overarching goal was to empower the students to use and understand NWP models on a fundamental level and prepare them for post-undergraduate positions in the NWP arena.

It was decided that an active learning approach would be the ideal format to present the information and disseminate the material taught as a series of workshops/seminars.

During the 2019 Fall semester, I taught a Forecasting Lab course that met once a week for 2 hours.  The extended class time allowed the DTC team to teach the attendees and demonstrate how to compile and run the Weather Research and Forecasting (WRF) Model on the Amazon Web Service (AWS) Cloud Server. The DTC team met with my students for a total of eight hours (six face-to-face hours and two hours via Google Meet).

The initial design of the WRF Cloud Computing (WCC) course was somewhat challenging because the setup and configuration of the AWS on our UNIX-based workstations faced some computing and financial hurdles. Nevertheless, the DTC team created Docker container images for each component of the end-to-end WRF-based NWP system on the AWS Cloud. Docker makes it easier to create, deploy, and run applications by using “containers.” A container is a software tool that packages code and all of its dependencies, so the application runs quickly and reliably from one computing environment to another (Docker Website). The end-to-end system containers were built and configured by the DTC team ahead of classes. With the accessibility of the components in containers, it was possible to efficiently and effectively create a full end-to-end NWP system teaching toolkit for use by the students.

Next, the DTC team provided step-by-step instructions on how to use those images to access the WRF data and ecosystem to run several case studies through the end-to-end NWP system from pre-processing, to running the model, then post-processing, visualization, and verification.  The DTC team also recommended installing Docker on the MSU Denver Weather Lab network so the students could run the end-to-end NWP container system locally on our in-house desktops as well.

Was the class a success? Well, here are a few student quotes that endorsed its success.

“I thought the NCAR instructors were knowledgeable, patient, and helpful…”

“Was awesome to have the hands-on help”

“I enjoyed the class very much and want to express gratitude to all those from NCAR and Dr Ng for their time and patience with us (or at least me) in learning something foreign and more difficult to grasp.”

Metropolitan State University classroom Forecasting Lab course.

UFS Users’ Workshop

Summer 2020

The first Unified Forecast System (UFS) Users’ Workshop, held on 27-29 July 2020, convened a broad cross-section of the community, despite transitioning from the originally planned in-person format to a virtual format. Organized by a diverse committee from NOAA Global Systems Laboratory (GSL), NOAA Physical Science Laboratory (PSL), NOAA Environmental Modeling Center (EMC), NOAA National Weather Service (NWS), the National Center for Atmospheric Research (NCAR), and George Mason University, the workshop presenters gave one hundred and ten talks to nearly five hundred attendees who registered for the event. While NOAA had a strong presence at the workshop, over 46% of the participants were not affiliated with NOAA, with a strong showing from the academic community (approximately 25%) representing over 44 different academic institutions.  The private sector was the third largest contingent comprising almost 10% of the registrants.  NOAA participants were almost equally split between the NWS and NOAA Research.  The participants and presenters also represented diverse expertise spanning weather, climate, hydrology, oceanography, space weather, modeling and High Performance Computing.  

The workshop kicked off with introductory remarks from Dr. Neil Jacobs, the Assistant Secretary of Commerce for Environmental Observation and Prediction, and an overview of the UFS was presented by the co-leads of the UFS Steering Committee, Dr. Ricky Rood of the University of Michigan and Dr. Hendrik Tolman, the NWS Senior Advisor for Advanced Modeling Systems.  The introductory session was rounded out by presentations by the leads for the six UFS application teams: Medium-Range Weather (MRW) and Subseasonal-to-Seasonal (S2S), Short-Range Weather (SRW), Hurricane, Space Weather, Coastal, and Air Quality.  The remainder of the workshop was a mix of plenary and parallel sessions focused on six topic areas: 

  • UFS Updates, Cloud Computing, Infrastructure, and Computational Performance
  • Model Dynamics, Physics, and Air Quality
  • Data Assimilation, Ensembles, and Predictability
  • Regional Configurations and Extremes: Development and Applications
  • Verification, Evaluation, and Post Processing
  • Earth-System Modeling (Land/Hydrology, Ocean, Sea Ice, Space Weather, Cryosphere)

The participants of the first UFS Users’ Workshop enjoyed lively engagement and discussions, despite the virtual format.  To facilitate exchanges that would normally take place over coffee breaks, lunch, or a reception, the organizing committee set up Slack channels where workshop participants were encouraged to post their questions and comments to the speakers, and dialogue was encouraged to continue on this platform following each presentation.  Each general topic area was issued a Slack channel to aid in navigating this communication forum and foster dialogue during the parallel sessions.  

Find Out More About UFS

Slack, a channel-based messaging platform, used for the first UFS Users’ Workshop to aid in navigating this communication forum and foster dialogue during the parallel sessions.

Communication and Outreach in the Unified Forecast System

Spring 2020

The Unified Forecast System (UFS) is a coupled, comprehensive Earth modeling and data assimilation system that will be used in NOAA operations and by the research community. There are naturally a lot of questions about UFS from potential collaborators, for example: What UFS codes are available to run? How do I find out more information about the project, and how can I get involved?

One of the roles of the UFS Communication and Outreach Working Group (C&O WG, for short) is to communicate the answers to such questions. Members include physical scientists, social scientists, NOAA public affairs representatives, managers, and software developers. The C&O WG also established a “UFS Focus Group,” which is a diverse collection of 50+ people, populated by graduate students, scientists, field officers, and others, who have volunteered their time to review and test UFS products.

The first product generated by the C&O WG was the UFS Portal, at Launched last year following a Focus Group review, the Portal is a one-stop-shop for all things UFS: news, upcoming events, highlights of current activities, documents, and plans. Looking for answers to the questions above? Information about the March 2020 release of the UFS, the  Medium-Range Weather Application 1.0, is available on Portal along with a link to support forums. There is an overview of all UFS applications (e.g. short-range weather, sub-seasonal to seasonal prediction, space weather), a description of the UFS governance structure and working groups, and information about how to get involved.

The role of the C&O WG extends beyond the Portal to more general aspects of UFS communications. Making sure that UFS participants have a space where they can work together easily is critical, which is why the C&O team collaborated with the NOAA Environmental Modeling Center (EMC) and their UFS partners to set up a GitHub organization, repositories, and wikis for UFS applications. Making the code available, testing it, and sharing documentation are key aspects of community participation. The ufs-community organization on GitHub, at, is how UFS releases are being distributed.

The C&O WG is also working with community members to understand and document the usability of UFS code. To do this, the C&O WG has engaged with UFS software developers to prepare and distribute  “Graduate Student Tests” or GSTs. The UFS project defines the GST as the ability of a student to easily get code, run code, change code, test code for correct operation, and compare and evaluate results. The most recent GST was released with the Medium-Range Weather Application, and it is open to all - not just graduate students!  Evaluators get, build, and run an example, change a physics parameter, rerun, and visually compare results, in less than six hours. Then they fill out a questionnaire about their experience. Results will be shared on the Portal and the feedback used to improve future releases. Like other C&O WG activities, the GST encourages communication that will help to advance UFS scientifically and computationally.

Interested in participating in a GST? See

Container & CCPP AMS Short Courses

Autumn 2019

One of the primary goals of the DTC is to provide software and infrastructure that aid in transitions between the research and operational communities.  The American Meteorology Society (AMS) provides an ideal venue for sharing these tools with the community through the AMS short course offerings at the Annual Meeting.  The DTC is looking forward to presenting two short courses at the 100th AMS Annual Meeting in Boston, MA in January 2020. 

“Integrating Numerical Weather Prediction (NWP) System Components Using Container Technology and Cloud Services”

The goal of this course is to raise awareness about tools and facilities available to the community for testing and evaluating Numerical Weather Prediction (NWP) innovations, including the emerging set of software tools in reusable containers and cloud computing resources, through hands-on learning. Containerized software is used to bundle all operating systems, code, library dependencies, and executables needed to both build and run software packages on any computing environment.  The DTC has leveraged this technology to create a portable end-to-end system comprised of various NWP components such as the Weather Research and Forecasting model (WRF), Gridpoint Statistical Interpolation (GSI) data assimilation system, Unified Post Processor (UPP), and Model Evaluation Tools (MET). These can be executed on any platform, including in the cloud, without the typical upfront time and frustration of building the software packages from the ground up. The course will introduce the concept of containerized software, provide an overview of the NWP components available from the DTC, and offer a hands-on tutorial that will allow participants to use the containers to complete case study examples using cloud services.  While this course may appeal to a wide-reaching audience, this information may be particularly useful to undergraduate and graduate students interested in learning more about NWP and to university faculty that may find software containers and cloud computing to be useful teaching tools to add to their course curriculum. The online tutorial for the end-to-end NWP containers is publicly available at:

“Experimentation and Development of Physical Parameterizations for Numerical Weather Prediction Using a Single-Column Model and the Common Community Physics Package (CCPP)”

This half-day course will teach participants how to develop and experiment with physics parameterizations within the CCPP framework. CCPP is the mechanism adopted by NOAA to drive atmospheric physics within Unified Forecast System (UFS) applications. NCAR also plans to use CCPP in their modeling systems (e.g., WRF, MPAS, CESM). A single-column model will be used to demonstrate how the CCPP framework works and to expose participants to physics suites available in the CCPP. The use of prepared, observationally-based cases combined with this tool’s computational simplicity will allow participants to grasp relevant concepts related to the CCPP and to conduct basic experiments. Graduate students, physics developers and researchers, as well as those with a general interest in working within NOAA frameworks could benefit from attending this course.

AMS Short Course

100th AMS Meeting

Model Evaluation for Research Innovation Transition (MERIT)

Spring 2019

The Model Evaluation for Research Innovation Transition (MERIT) project provides a critical framework for physics developers to test innovations within their schemes using selected meteorological cases that have been analyzed in depth.  Comparing their results to baseline MERIT simulations will allow developers to determine whether their innovations address model shortcomings and improve operational numerical weather prediction.

For the DTC’s AOP 2018, three high-impact global FV3 baseline cases were selected for in-depth analysis: the Mid-Atlantic blizzard of January 2016, Hurricane Matthew, and the May 2017 severe weather outbreak in the Southern Plains.  These cases were chosen after consultation with the Model Evaluation Group (MEG) at NOAA’s Environmental Modeling Center, as each case exhibits known deficiencies in the global configuration of the Finite-Volume Cubed-Sphere (FV3) model.  Multiple-day simulations were run using an end-to-end workflow developed to handle the pre-processing of initial conditions, the integration of the model, post-processing with the Unified Post Processor (UPP), and verification with the Model Evaluation Tools (MET).  Also, in collaboration with the MEG, the MERIT team has been working on developing and applying unique verification techniques and metrics that will help assess the impact that physics innovations may have on these known FV3 biases. In particular, the progression of certain meteorological features will be assessed through the MET Method for Object-Based Diagnostic Evaluation (MODE) time-domain/storm-relative feature analyses.

The initial focus of MERIT continues to be on existing capabilities available in the global model framework. However, this activity is expected to include high-resolution/convection-allowing modeling as the Stand Alone Regional (SAR)-FV3 becomes available. Providing the research and operational communities with an end-to-end framework will streamline the testing process, leading to more effective and efficient physics development.  In addition, it will also encourage community engagement and provide an infrastructure that supports R2O and O2R. 


MERIT's three high-impact global FV3 baseline cases.

The DTC helps the research community enhance the GSI/EnKF operational data assimilation system

Winter 2019

Gridpoint Statistical Interpolation (GSI)/Ensemble Kalman Filter (EnKF) are operational data assimilation systems, open to contributions from scientists and software engineers from both operations and research. The development and maintenance of NOAA GSI/EnKF data assimilation systems are coordinated and managed by the Data Assimilation Review Committee (DARC), which incorporates all major GSI/EnKF data assimilation development teams in the United States within a unified community framework. DARC established a code review and transition process for all GSI/EnKF developers, reviews proposals for code commits to the GSI/EnKF repository and ensures that coding standards and tests are being fulfilled. Once DARC approves, the contributed code is committed to the GSI/EnKF code repository and available for operational implementation and public release.

The Developmental Testbed Center (DTC) Data Assimilation (DA) Team serves as a bridge between the research and operational communities by making the operational data assimilation system available as a community resource and by providing a mechanism to commit innovative research to the operational code repository. Prospective code contributors can contact the DTC DA helpdesk to prepare, integrate, and document the expected impact of their code and ensure that any proposed code change meets GSI coding standards. They can also apply to the DTC Visitor Program for their DA research and code transition.

Since the code transition procedures were established, the DTC DA team has helped many researchers contribute code to the repository:

  • NOAA/GSD and NCAR MMM scientists improved chemical initial conditions for WRF-Chem and GO-CART forecasts by using WRF-Chem and GOCART as background to analyze surface measurements of fine particulate matter (PM2.5) and MODerate resolution Imaging Spectroradiometer (MODIS) total Aerosol Optical Depth at a wavelength of 550 nm. These functions are available through the DTC GSI release and have been used by many researchers including Barbara Scherllin-Pirscher from the Central Institute for Meteorology and Geodynamics, Vienna, Austria. Scherllin-Pirscher is a DTC visitor working to further enhance GSI chemical analysis by assimilating vertical light detection and ranging (LIDAR) measurements to improve the vertical aerosol representation in WRF-Chem forecasts.

  • The DTC hosted Mengjuan Liu from the Shanghai Meteorological Service to study how to use GSI to improve the surface data analysis. Liu found the conventional observation operator can introduce large representativeness errors when surface conditions are inhomogeneous, such as on coastlines. An improved forward model for surface observation along the coastline was developed and added to GSI repository. This new forward observation operator was used in the recent operational Rapid Refresh/High-Resolution Rapid Refresh (RAP/HRRR) update.

  • The DTC Visitor Program hosted Ting-Chi Wu from CIRA/CSU to add the capability to assimilate solid-water content path (SWCP) and liquid-water content path (LWCP), which are satellite retrieved hydrometeor observations from Global Precipitation Measurement (GPM) from the Goddard PROFiling algorithm (GPROF).

  • The DTC Visitor Program hosted Karina Apodaca from CIRA/CSU to incorporate two new lightning flash rate observation operators suitable for the Geostationary Operational Environmental Satellite (GOES)/Global Lightning Mapper (GLM) instrument in the GSI variational data assimilation framework.  One operator accounts for coarse resolution and simplified cloud microphysics in the global model to evaluate the impact of lightning observations on the large-scale environment around and prior to storm initiation. Another forward operator for use with non-hydrostatic, cloud-resolving models permits the inclusion of precipitating and non-precipitating hydrometeors as analysis control variables.

The GSI/EnKF code commit procedures established by DARC and the DTC successfully moves innovative contributions into the repository. The DTC and its Visitor Program is a great resource for the research community to introduce new techniques and model components to advance numerical weather prediction technology.

Contributed by Ming Hu and Chunhua Zhou.

2018 DTC Community Unified Forecast System Test Plans and Metrics Workshop

Summer 2018

The 2018 DTC Community Unified Forecast System Test Plan and Metrics Workshop was held at NOAA’s National Center for Weather and Climate Prediction (NCEP) on July 30 - August 1, 2018. The major goal of this workshop was to work towards a community test plan with common validation and verification metrics for the emerging Unified Forecast System (UFS). The plan will serve as a guide for the Numerical Weather Prediction (NWP) community for testing and evaluating new developments for the UFS models and components by comparison of both historical and real-time forecasts using observations and analyses, through standardized hierarchical testing.

The workshop organization was led by the Developmental Testbed Center (DTC), and the organizing committee was representative of various aspects of the NWP verification and validation (V&V) enterprise, including voices from those working on research, development, transitions, and operations. The membership of the organizing committee was:

  • Curtis Alexander - NOAA/Earth Systems Research Laboratory Global Systems Division (GSD)

  • Ligia Bernardet - CU/Cooperative Institute for Research in Environmental Studies (CIRES) at NOAA/GSD and DTC

  • Tara Jensen - National Center for Atmospheric Research (NCAR) and DTC

  • Jim Kinter - George Mason University/Center for Ocean-Land-Atmosphere Studies (COLA)

  • Sherrie Morris (NOAA Office of Science and Technology Integration (OSTI)

  • Jason Levit (NOAA/NCEP/Environmental Modeling Center (EMC)

  • Ryan Torn (State University of New York at Albany)

  • Ivanka Stajner (NOAA OSTI on detail to NOAA/NCEP/EMC)

The workshop was attended by approximately 100 participants, cross-cutting through various sectors of the V&V community, including international (Taiwan Central Weather Bureau and European Center for Medium-range Weather Forecasting (ECMWF), universities, NASA, NOAA National Environmental Satellite, Data, and Information Service (NESDIS), research laboratories, Office of the Federal Coordinator for Meteorological Services and Supporting Research, National Weather Service, testbeds, Navy, U.S. Air Force, and the private sector.

The workshop had a mix of presentations, discussion periods, and working sessions in which participants contributed to the three topic-based breakout sessions: test plans, metrics, and hierarchical testing.  Metrics for all spatial and temporal scales for numerical weather prediction models and emerging topics such as the verification of convective allowing models, coupled earth system models, and ensemble systems were discussed.

The last activity in the workshop was a summary of the working sessions’ discussions by their leads, which was presented to workshop participants and members of the UFS Strategic Implementation Plan (SIP) meeting. A report on the progress made during the workshop will be available in the next few months.

A link to the workshop page is here: 2018 DTC Community Unified Forecast SystemTest Plans and Metrics Workshop


2018 DTC Community Unified Forecast System Test Plans and Metrics Workshop Attendees
2018 DTC Community Unified Forecast System Test Plans and Metrics Workshop Attendees


DTC staff host AMS Short Course on Containers

Spring 2018

A major hurdle for running new software systems is often building and compiling the necessary code on a particular computer platform. In recent years, the concept of using “containers” has been gaining momentum in the numerical weather prediction (NWP) community. This new container technology allows for the complete software system to be bundled (operating system, libraries, code, etc.) and shipped to users in order to reduce the spin-up time, leading to a more efficient setup process. A core mission of the DTC is to assist the research community in efficiently demonstrating the merits of new model innovations.  The development and support of end-to-end NWP containers is in direct support of that mission.

In recent years, a number of NWP software components (including pre-processing, the model itself, post-processing, graphics generation, and statistics computation and visualization) were implemented into Docker containers to better assist the user community. The work conducted by DTC staff leveraged previous efforts of the Big Weather Web (, which initially established software containers for the WRF Pre-Processing System (WPS), Weather Research and Forecasting (WRF) model, and NCAR Command Language (NCL) components. From there, DTC staff expanded the containerized tools to include the Unified Post-Processor (UPP), Model Evaluation Tools (MET), and METViewer. Through this complementary work, a full end-to-end NWP system was established, allowing for verification of the model output and visualization of the statistical output.

Several DTC staff (Kate Fossell, John Halley Gotway, and Tara Jensen, and Jamie Wolff) hosted a short course at the 98th Annual AMS meeting in Austin, TX on 6 January 2018 that offered hands-on experience with the established software containers. In preparation for the short course, an online tutorial was created that can be accessed at: If you are an undergraduate/graduate student, university faculty, or researcher who is interested in these new tools, please check it out! Participants of the 2018 short course were complementary of the day-long tutorial, and the DTC plans to offer it again next year at the Annual meeting in Phoenix, AZ – stay tuned for more information regarding future training opportunities.

DTC staff and trainers

WRF Users' Workshop - June 2017

Autumn 2017

The first Weather Research and Forecasting model (WRF) Users’ Workshop was held in 2000. Since then, eighteen annual workshops have been organized and hosted by the National Center for Atmospheric Research in Boulder, Colorado to provide a platform where developers and users can share new developments, test results, and feedback. This exchange ensures the WRF model continues to progress and remain relevant.

The workshop program has evolved through the years. In 2006, instructional sessions were introduced, with the first focused on the newly developed WRF Pre-processing System (WPS). The number of users has grown since the WRF Version 3 release in 2008, so a lecture series on the fundamentals of physics was introduced in 2010 to train users to better understand and apply the model. Since that time, the series has covered microphysics, planetary boundary layer (PBL) and land surface physics, convection and atmospheric radiation. The series then expanded to address dynamics, modeling system best practices, and computing.

The 18th WRF Users’ Workshop was held June 12 – 16, 2017. The workshop was attended by 180 users from 20 countries, including 57 first time attendees, and 130 papers were presented. The first afternoon of the workshop, four lectures covered the basics of ensemble forecasting, model error, verification and virtualization of ensemble forecast products. The following days included nine sessions on a wide range of WRF model development and applications. On Friday, five mini-tutorials were offered on WRF-Hydro, Model for Prediction Across Scales (MPAS) for WRF Users, Visualization and Analysis Platform for Ocean, Atmosphere, and Solar Researchers (VAPOR), NCAR Common Language (NCL) and WRF-Python. All workshop presentations are available from

The WRF Modeling System Development session included the annual update, plus status reports on WRF Data Assimilation (WRFDA), WRF-Chem, WRF software, Gridpoint Statistical Interpolation (GSI), Hurricane WRF (HWRF) and WRF-Hydro. A hybrid vertical coordinate was introduced in Version 3.9 for the Advanced Research WRF (ARW) that may potentially improve prediction in the upper-air jet streak region. Another notable addition to the model is the predicted particle properties or P3 scheme, a new type of microphysics.

Both data assimilation and model physics were improved in the operational application of WRF in the Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR) models. Advances were made in the Grell-Freitas cumulus scheme, Mellor-Yamada-Nakanishi-Niino (MYNN) PBL scheme and the Rapid Update Cycle (RUC) Land Surface Model (LSM). The HWRF operational upgrade included a scale-aware Simplified Arakawa Schubert (SAS) cumulus scheme, new Ferrier-Aligo microphysics schemes, and improved data assimilation. Other development and applications of WRF were also presented. Notably, the large-eddy simulation (LES) capability has been extended to many real-data applications in recent years.

There were two discussions during the workshop devoted to physics suites and model unification. Two suites of pre-selected physics combinations are now available in V3.9 that are verified to work well together for weather prediction applications. The second discussion was about model unification between WRF and the newer MPAS. While the two models remain independent, both are supported by the community and aspects of their development effort can be shared.

The next WRF Users’ Workshop will in June 2018.

Science Advisory Board Meeting - Sept 14-15

Winter 2017

The DTC hosted its annual DTC Science Advisory Board (SAB) meeting on September 14-15, 2016.  This annual event provides an opportunity for the DTC to present a review of DTC key accomplishments over the past three years to representatives of the research community and to collect input directed at shaping a strategy for the future.

 “The DTC enables and supports a wide variety of research-to-operations and operations-to-research activities,” said SAB Chair Russ Schumacher. “The SAB discussed ways that these activities can be further strengthened, including through model evaluation and maintaining operational model codes.  A topic of particular discussion was NOAA and NCEP’s goal to move toward a unified modeling approach.  This brings with it some great opportunities to connect the research and operational communities to enhance weather prediction, but will also pose challenges in familiarizing researchers with new modeling systems.”

The agenda included an Operational Partners Session, an opportunity for the new Environmental Modeling Center (EMC) Director Dr. Mike Farrar to present a vision for NOAA’s unified modeling, followed by the Air Force outlook for its modeling suite by Dr. John Zapotocny. DTC task area presentations reviewed key accomplishments with a focus on research to operations, and presented thoughts for possible research to operations activities, risks, and challenges within the coming three years. Break-out task area group discussions were productive, followed by the SAB recommendations session. The recommendations are detailed in the DTC SAB meeting summary, but the following are a few highlights.

With NCEP’s transition to a unified modeling system centered around the Finite Volume Cubed Sphere (FV3) dynamic core, the SAB voiced their belief that the community will need the DTC’s leadership in supporting FV3 and the NOAA Environmental Modeling System (NEMS) in future years, and the DTC should build up internal expertise in advance of these transitions. They noted no other organization in the U.S. has a core responsibility to be an unbiased evaluator of Numerical Weather Prediction (NWP) models, and advised the DTC to not lose sight of their unique function. As the NGGPS paradigm emerges, the SAB encourages the DTC leadership to keep close attention on where future DTC funding might be anticipated, and make sure future SAB members have expertise in those funding areas.

The SAB recommended the DTC build an interactive research community to hear from active users with fresh ideas and provide a conduit between research now and possible operations in the future. It would be a good forum for interacting with active scientists in other areas of modeling and would be a promising mechanism for getting users involved in the DTC Visitor Program. They also encouraged further engagement with the global ensemble, convection-allowing ensemble, data assimilation, and verification communities.

To support DTC-supported code and address the DTC staffing problem, the SAB suggested a graduate student model. This could build and enhance capabilities of upcoming researchers relevant to the needs of operational models. It would also benefit the university community by supporting students with a gap in funding -- a win-win situation for both the DTC and the university.

The SAB recommended the DTC to continue their work to put supported codes into Docker containers, an open-source project that automates the deployment of applications inside software containers for community use. This significantly reduces challenges associated with setting up and running code on different platforms, and building the libraries the codes use. They also suggested clarifying the roles of model and code developers, the DTC, and the user community in accessing, supporting, and adding new innovations to reduce possible confusion and redundancy.

Because all DTC task areas require robust verification tools to achieve their objectives, the SAB recommended strengthening and supporting collaborations that already exist between the verification task and the other task areas. This may involve increasing the flexibility of Model Evaluation Tools (MET) to support the needs of both DTC tasks and communities. The SAB also indicated there needs to be clear pathways for those who develop new verification tools or methods to have those tools incorporated into MET. They also encouraged the DTC to have more year-to-year continuity in testing and evaluation activities to increase productivity and yield more fruitful outcomes, and to continue to thoughtfully balance these activities on a task-by-task basis.

The external community Science Advisory Board acts as a sounding board to assist the DTC Director, and provides advice on emerging NWP technologies, strategic computer resource issues, selection of code for testing and evaluation, and selection of candidates for the visiting scientist program. Members are nominated by the Management Board, and the Executive Committee provides final approval of SAB nominations for a 3-year term. Current members of the DTC Science Advisory Board can be found at under governance.

NGGPS Atmospheric Physics Workshop

Summer 2017

On 8-9 November 2016, more than 80 scientists from a broad cross-section of the physics development community gathered for the Next Generation Global Prediction System (NGGPS) Atmospheric Physics Workshop at the NOAA Center for Weather and Climate Prediction. The workshop provided an opportunity for the NGGPS Physics Team to revisit and refine its near- and long-term priorities for advancing the National Centers for Environmental Prediction (NCEP) global physics suite and identify key areas that need attention. Current plans are to deliver the advanced physics suite by October 2018. 

Workshop participants proposed the following approaches to advance NCEP’s global physics suite:

  • Upgrade the radiation code to RRTMGP, a restructured and modern version of RRTMG (Rapid Radiative Transfer Model for Global Climate Models), to allow more interactions between advanced schemes, such as different radiative processes for separate distributions of cloud ice and snow, and to significantly increase the speed of this computationally costly component of the physics.
  • Determine an approach for selecting a microphysics scheme from a list of strong candidates that meets NGGPS priorities/goals and proceed with an open selection process.  This approach should include both testing metrics and an assessment of how the scheme would set up the physics suite for future advancement.  General consensus of the breakout group discussions was that the Thompson scheme is a strong candidate for NGGPS testing.
  • Convection and boundary-layer schemes would follow the idea of having an evolved suite and an advanced suite that can be compared with suitable metrics. It is recognized that decisions regarding these physics components need to be taken in the context of a suite with other candidate components.
  • The land-surface model will include updated data, and will evolve based on process studies for individual components. The final model will likely come from selected Noah-MP options.

The workshop discussions laid the groundwork for a collaborative framework that will allow the research and operational communities to efficiently and effectively accelerate the advancement of NCEP’s global physics suite.  At the center of this framework is the Interoperable Physics Driver (IPD)/Common Community Physics Package (CCPP) concept and a clearly defined and documented systematic process to select and advance innovations that define the composition of future operational physics suites.

While the workshop discussions made great strides towards defining the key aspects of this collaborative framework, many of the details of the decision-making process still remain to be defined and vetted.  As these final details are worked out, it will be imperative to make sure the process is open and transparent, and takes into account the needs of both the research and operational communities.

Sea Ice Modeling Workshop

Spring 2016

A sea ice modeling workshop was convened on 3-4 February 2016 in Boulder, CO for the purpose of informing NOAA on the inclusion and selection of a community-contributed sea ice model into the future Next-Generation Global Prediction System (NGGPS). Another workshop goal was to identify potential research and development opportunities and gaps.

The workshop was hosted at the National Center for Atmospheric Research (NCAR) by the DTC’s Global Model Test Bed, and counted as sponsors the National Weather Service, the NOAA Office of Atmospheric Research Climate Program Office, and the Office of Naval Research. Sixty-five scientists attended the workshop, representing a broad spectrum of research and operational organizations, such as universities, NOAA, Navy, Department of Energy (DoE), the US National Ice Center, National Snow and Ice Data Center, NCAR, NASA, and DTC. International participation included representatives from the UK Meteorological Office, U. Reading, U. Toronto, and Environment Canada.

NOAA’s NGGPS will be a single fully coupled Earth modeling system with application to forecasts from days to seasons, spanning spatial scales from 1 to 25 km. While the sea ice model to be selected for inclusion in NGGPS needs to have good performance for all forecast applications over these time and space scales, this workshop focused on the short- and medium-term sea ice forecast needs. Workshop participants reviewed several state-of-the-art sea ice modeling efforts, along with various Earth modeling systems, such as the NCEP Keeping Ice’S Simplicity (KISS) model, the DoE Community Ice CodE (CICE) model, the NOAA Geophysical Fluid Dynamics Laboratory Sea Ice Simulator (SIS), the NOAA National Centers for Environmental Prediction Climate Forecast System (CFS), the Canadian Regional Ice-Ocean Prediction System (RIOPS), the ESRL Regional Arctic System Model (RASM) and the Navy Research Laboratory Arctic Cap Nowcast/Forecast System (ACNFS).

It was noted that the majority of advanced sea ice models have similar physical parameterizations, and that the differences among advanced models is smaller than the uncertainty due to initial conditions and external forcing. Given that the use of a community-contributed and supported model in NGGPS was raised as a priority for model selection, participants recommended the tentative adoption of the CICE model, pending follow-up testing and addressing concerns raised regarding the CICE model governance and potential for introduction of numerical artifacts due to differences in staggering between the grids used in the ocean models and in CICE.

The framework for follow-up testing was discussed and important points regarding resolution, domain, coupling, verification metrics, and observations were raised. The importance of the ocean model for ice model performance was also discussed, and will need to be a factor in the test design. These issues will be considered by a tiger team formed to design and conduct the tests. A variety of synergistic project efforts in the scientific community were identified and recommendations for future model development were put forth. Test results will be used to prioritize future model development efforts.

For more information on the NGGPS sea ice modeling workshop and its presentations, please visit

Sea Ice Workshop group photo

New HWRF Developers Website: R2O for Hurricane Model Development

Winter 2015

The mission of the DTC is to accelerate the rate of transition of new research and development to operational numerical weather prediction models. To that end, the DTC makes NCEP operational models, such as the Hurricane WRF, available to the general community through yearly releases of stable, well-tested, and well-documented codes, which are supported through a help desk.

While the DTC has hundreds of registered HWRF users, only a small subset of them actually contribute innovations, raising questions about the return on the DTC’s investment.

To address this concern, an additional type of support, targeted to this select group of active developers, has been launched by the DTC with support from the Hurricane Forecast Improvement Project (HFIP). Through the HWRF developers’ website (, scientists external to EMC can request access to the HWRF code repositories, giving them access to retrieve and contribute to experimental codes. They can also obtain information about the HWRF code management process and the steps to get their code made available for consideration by EMC. Finally, scientists can get training on advanced HWRF aspects not made available to the general community, such as the HWRF build system and HWRF automation with the Rocoto Workflow Manager System.

This new DTC service, which goes well beyond what is provided to the general community through public releases, has been extensively used by many HWRF developers, and has been particularly helpful to the principal investigators funded by HFIP.

An HWRF Tutorial in Taiwan

Summer 2014
Tim Brown-DTC, Qingfu Liu-EMC, Yong Kwon-formerly of EMC, Ligia Bernardet-DTC, Vijay Tallapragada-EMC, and Sam Trahan-EMC some of the HWRF instructors, May 2014, Taipei, Taiwan.

The Hurricane Weather Research and Forecasting model (HWRF) is a U.S. operational hurricane prediction model used by the National Hurricane Center for tropical cyclone track and intensity forecasts in its basins of responsibility: North Atlantic and Eastern North Pacific. However, HWRF can be employed in any basin. In 2013 the HWRF real-time runs conducted by the NOAA Environmental Modeling Center (EMC) for the West Pacific basin were found to be very valuable by the Joint Typhoon Warning Center (JTWC). Because of the demonstrated skill of the HWRF model and its advanced capabilities, there has been a strong interest in HWRF from the research community as well as the international weather centers that are responsible for tropical cyclones forecasting. Currently, there are more than 1000 registered users for HWRF. With a goal of encouraging the participation of international research and operational community in the development and applications of HWRF, the NOAA Hurricane Forecast Improvement Project (HFIP) sponsored an HWRF tutorial in Taipei, Taiwan, 22-23 May 2014. The HWRF Tutorial was held immediately following the Workshop on Numerical Prediction of Tropical Cyclones, 20-21 May 2014, which was attended by about 60 scientists from Taiwan, U.S., China, Japan, S. Korea, India, Vietnam, Thailand, the Philippines, and Malaysia. Fred Toepfer, HFIP Program Director, gave a keynote speech at the workshop. The HWRF Tutorial was organized jointly by DTC, EMC, HFIP, Taiwan’s Central Weather Bureau (CWB), and the Taiwan Typhoon Flood Research Inst (TTFRI). Twenty-six students from Malaysia, the UK, Thailand, Vietnam, USA, Singapore, and Taiwan participated. The tutorial instructors included Robert Gall of HFIP, Vijay Tallapragada, Young Kwon, Sam Trahan, Qingfu Liu, and Chanh Kieu of EMC, and Timothy Brown and Ligia Bernardet of DTC. The feedback from the students was overwhelmingly positive, in spite of the torrential rain of 14 inches in 24 hours which fell in Taipei during the event! We anticipate an increased use of HWRF in the West Pacific typhoon community in the years to come, which will lead to valuable collaboration on the continued development on HWRF.

Students in the classroom

Vijay Tallapragada lecturing

Community Software Maintenance and Support

Winter 2014

One function of the DTC has been to archive and maintain important model-related code, and to make it available to operational and research segments of the meteorological community. As Laurie Carson describes it, the code maintenance and support function has important objectives in both O2R and R2O arenas: for the former, providing operational software to the research community, and for the latter, facilitating transfer of research capabilities to operational software packages. DTC’s approach is based on a philosophy that community software is a resource shared with a broad community of (distributed) developers specifically including the capabilities of operational systems. Two keys to its success are periodic public releases that include new capabilities and techniques, and effective user support. The chart summarizes present and planned DTC software support activities in five principal areas: WRF model updates and support, data assimilation (GSI) code releases and support, the end-to-end operational hurricane forecast system (HWRF), verification package maintenance and support (MET), and planning for a future community package of the NOAA Environmental Modeling System (NEMS) that includes the NMMB model. Some community code now supported in this way has derived from DTC visitor projects; an example is the field alignment technique described in the 2012 visitor project of Sai Ravela (summary available at http://www.dtcenter. org/visitors/year_archive/2012/). For further description of the DTC community software efforts, see http://www.


“The DTC software maintenance task has both O2R and R2O objectives.”

As the chart indicates, another community outreach-related DTC activity involves arranging and contributing to workshops and tutorials to facilitate use of these community model and analysis packages. A future issue of Transitions will summarize recent and upcoming events of this kind.

The 2014 GSI Community Tutorial

Autumn 2014

The DTC hosted the 5th Community Gridpoint Statistical Interpolation (GSI) Tutorial on July 14-16 of this summer at the NCAR Foothills Laboratory in Boulder, Colorado. One of several outreach events sponsored recently by the DTC, this tutorial was held in collaboration with other major GSI development teams from around the United States. With an ultimate goal of providing operational capabilities to the research community, this series of tutorials has become a primary training resource whereby both operational and research users can gain knowledge essential to running and further developing GSI.

The tutorial this year was a three-day venture that included both invited lectures and practical hands-on sessions relevant to GSI. Within the program were lectures designed to cover both fundamental (e.g., compilation, execution, and di agnos t i c s ) and advanced (pre-processing, radiance and radar data assimilation, hybrid techniques, and GSI infrastructure) topics.

Lecturers and practical session instructors were invited from major GSI development/support teams, including NCEP/EMC, NASA/GMAO, NOAA/ESRL, and NCAR/MMM, along with DTC members from NOAA/ESRL and NCAR/ RAL. The principal guest speaker from the university community this year was Dr. Milija Zupanski from Colorado State University. Attended by 41 students from the U. S. and other international agencies and universities, the tutorial easily reached maximum capacity.

Tutorial presentations and lectures are posted at http:// For more information about the GSI system itself and its community support, please visit: http://www.dtcenter. org/com-GSI/users/index.php.

On July 17 after the Community Gridpoint Statistical Interpolation (GSI) Community Tutorial (summarized on page 4), the GSI Review Committee also met at the NCAR Foothills Laboratory in Boulder. Established in 2010, this committee continues to coordinate GSI development from both operational and research communities, and is also responsible for reviewing and approving GSI code updates. During general review of ongoing GSI development efforts and future plans for GSI, the committee specifically discussed potential community support of the NOAA Ensemble Kalman Filter (EnKF) system, which is currently a part of the GSI-based hybrid ensemble variational system of the NOAA Global Forecast System (GFS), and a potential candidate for other operational applications. The decision was then made to establish code management for this EnKF system that follows the existing GSI code management protocol. As a consequence, the GSI review committee effectively becomes a joint review committee for both GSI and EnKF, and new membership (NOAA/ESRL, and the University of Maryland as a deputy member) was approved to represent the EnKF development effort. This new DA review committee thus includes members from NCEP/ EMC, NOAA/ESRL, NASA/GMAO, NESDIS, AFWA, NCAR, the University of Maryland, and the DTC.

Object-based Verification at WPC

Summer 2013

The Weather Prediction Center (WPC) at NCEP has been using MODE to supplement its traditional verification techniques since April 2010. The Method for Object-based Diagnostic Evaluation (MODE), a utility that is part of the MET verification package, has been developed with substantial support from the DTC. Both are systematically expanded and maintained for specific DTC tasks and an array of outside users. MODE output is available to WPC forecasters in real time through an internal website that displays graphical verification for forecasts of 24 hr precipitation valid at 1200 UTC (see the figure). Forecasters can select the forecast lead time (Day 1 – 36 hr, Day 2 – 60 hr, or Day 3 – 84 hr) and precipitation threshold (0.50 in, 1.0 in, and 2.0 in), then view the corresponding verification for WPC forecasts and 9 numerical models.

“Two things that seem to resonate with our forecasters the most are the real time aspect of the website and the visual nature of the comparison.”

The graphical nature of the MODE verification allows for a quick comparison of forecasts in a way that goes beyond traditional threat scores and bias values to consider other measures of forecast quality (distance between forecast and observed objects, differences in angle of orientation and object size, etc.). The most recent update to the website attempts to better quantify these qualities by adding statistical comparisons of the interest value and the displacement distance between matched objects to complement the traditional graphical comparisons. Future plans include additional statistical information on the website, including longer term summaries (monthly, annually, etc.), and making the website available to the public.

Running MODE on a national scale at an operational center like WPC can present some unique challenges since MODE must be able to correctly identify precipitation objects from meteorological phenomena as varied as cool season synoptic scale storms to warm season convection. Determining the ideal configuration is still a work in progress, but it is an essential piece of the puzzle in order to build forecaster confidence in the utility of object-based verification.

DTC Science Advisory Board Meeting

Autumn 2013

Given its mission to facilitate the research to operations transition in numerical weather prediction, the DTC has a mandate to stay connected with both the research and operational NWP communities.

As a means to that end, the DTC Science Advisory Board (SAB) was established to provide (i) advice on strategic directions, (ii) recommendations for new code or new NWP innovations for testing and evaluation, and (iii) reviews of DTC Visitor Program proposals and recommendations for selection.

The third meeting of the SAB (the first with the new members announced in an earlier DTC Newsletter) was held recently (25-27 September 2013) in Boulder. To stimulate communication between the research and operational NWP communities, the DTC invited Geoff DiMego and Vijay Tallapragada to present future plans for mesoscale and hurricane modeling at the National Centers for Environmental Prediction’s (NCEP) Environmental Modeling Center (EMC), and Mike Horner to give an Air Force Weather Agency (AFWA) modeling update. Both informative presentations stimulated considerable discussion.

John Murphy, Chair of the DTC Executive Committee (EC), presented NWS’s view on research to operations and the important role of the DTC. Col. John Egentowich, also a member of DTC EC, spoke positively of the contributions of the DTC to Air Force weather prediction modeling. During summary and discussion periods, SAB members provided many valuable recommendations for the DTC to consider during their planning for the new DTC Annual Operating Plan (AOP) for 2014. These recommendations will be detailed in a DTC SAB Meeting Summary. Of these, several were of particular interest.

The SAB recommended that the DTC and EMC develop a community model-testing infrastructure at NCEP/EMC. The goal for such an infrastructure would be to allow visiting scientists easy access to EMC operational models, enabling them to collaborate with EMC scientists to perform model experiments using alternative approaches. Such an infrastructure would hopefully facilitate accelerated R2O transition in NWP.

The SAB voiced their belief that operational centers will have significantly more computing resources in the near future, putting nationwide high-resolution mesoscale ensemble forecasting within reach. Given that possibility, they recommended that DTC should help facilitate transition to cloud-permitting scale ensemble forecasting with multiple physics. The current members of the DTC Science Advisory Board can be found at under governance.