NOAA Embraces Community Modeling to Advance Weather Prediction

Summer 2018

NOAA and the National Weather Service (NWS) are embarking on a new strategy of community engagement for developing the numerical weather prediction models that provide NWS forecasters with the best possible guidance. By engaging the broader numerical modeling community, NWS will leverage the vast modeling expertise that resides therein, since each model developer offers a unique perspective about modeling challenges and possible solutions.


EMC is excited at the prospect of leveraging the modeling expertise in the numerical modeling community to improve NOAA guidance, forecasts, and other products and services.

 

The goal of this community effort is to evolve the Next Generation Global Prediction System (NGGPS) towards a national unified Earth system modeling framework for operations and research, to the mutual benefit of both. A Unified Forecasting System (UFS) will  function on temporal scales from seasonal to sub-seasonal (S2S) on the order of months, down to short-term weather prediction on the order of hours to days. The UFS will also work across spatial scales, from global-scale predictions down to high-resolution, convection-resolving local/regional scales. Operational implementations of the UFS will be guided by the NWS’s National Centers for Environmental Prediction Environmental Modeling Center that leads the integration of research innovations into operational models.

The UFS is being developed by NOAA, other federal partners, and the broader research and academic community to build the best national modeling system possible. The definition of “community” is important, and not all community efforts will be identical. We are learning from prior and ongoing community modeling efforts (such as WRF, CESM, WW3, MOM6, etc.) and are adopting best practices that meet our specific situation.

NOAA recognizes that the UFS must support the needs of both operations and research. Without that linkage, the incentives will not be there for the research community to help make improvements that will benefit operational predictions, nor will operational innovations feed back into the models used for research. Building a community model involves both give and take from the operational and research sides. Lessons learned, such as from the Developmental Testbed Center (DTC), have shown us that the community will expect sufficient training, full support (including help desk), and vetting of scientific advances. Also, through NGGPS, the Joint Technology Transfer Initiative, and other coordinated programs, NOAA has opportunities for partners to engage through recurring Federal Funding Opportunities.

Working groups that span the specific modeling areas needed for the UFS began meeting in Spring 2017 to develop three-year plans that identify key partners and provide a set of milestones to benchmark progress. The collective input of those groups resulted in the publication of the first Strategic Implementation Plan (SIP) in November 2017.  It is a dynamic process, and these working groups are working on updates to the SIP, with version 2 expected later this year.

To effectively coordinate the activities of the community partners, as well as to manage the collaborative projects of those partners described in the SIP, a robust governance structure is being put in place. Our governance approach is based a commitment by core development partners, informed practices, and community values. A UFS Steering Committee, comprised of both NOAA and non-NOAA members, is already working to provide technical guidance to the Working Groups. A Technical Oversight Board is about to be established to provide support for the programmatic elements across the NOAA Line Offices.

EMC is excited at the prospect of leveraging the modeling expertise in the numerical modeling community to improve NOAA guidance, forecasts, and other products and services. Better predictions can come from ensembles of model runs, better models can come from the assembled intellectual might of the entire modeling community. If you’d like to become engaged, please contact Tim Schneider at timothy.schneider@noaa.gov. While a UFS web portal is being developed, up to date information can be found on the NGGPS web pages.


Brian Gross, Acting Director EMC

Director’s Corner - Georg Grell

Spring 2018

As chief of the Model Development Branch (MDB) at NOAA’s Global System Division (GSD), I am honored to work with many of the scientists from the DTC. The DTC has been a vital partner for the development, testing, and evaluation of the Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR) models, and I look forward to continuing this valuable collaboration as the community works together to define and develop the next-generation modeling system.  The work performed in my branch at GSD is strongly linked to the five task areas that the DTC supports: Regional Ensembles, Hurricanes, Data Assimilation, Verification and the Global Model Test Bed (GMTB). Each of these areas will also be an important component of the Next Generation Global Prediction System (NGGPS), currently under development.

The goals of the NWS NGGPS include not only to generate a much-advanced global modeling system with state-of-the-art non-hydrostatic dynamics, physics and data assimilation, but also to foster much broader involvement of other labs and the academic community. This opens an opportunity for a new role for the DTC. While the initial objective for NGGPS was the selection of a dynamic core, other tasks are now of high priority. Among those, the selection of the advanced physics parameterizations and the involvement of the community may be the most challenging. The selection of an advanced physics suite should be completed by the end of 2018, to allow rigorous testing and tuning to be performed with the complete package. It is recognized that all of the physics schemes will likely need further testing, development, and calibration prior to, during, and after implementation.  The DTC is ideally positioned to contribute to this task.

Model physics describe the grid-scale changes in forecast variables due to sub-grid scale diabatic processes, as well as resolved-scale physical processes.  Physical parameterization development has been a critical driver of increased forecast accuracy of global and regional models, as more processes are accounted for with sophistication appropriate for the model resolution and vertical domain.  Key atmospheric processes that are parameterized in current global models include subgrid turbulent mixing in and above the boundary layer, cloud microphysics and ‘macrophysics’ (subgrid cloud variability), cumulus convection, radiative heating, and gravity wave drag.  Parameterizations of surface heat, moisture, and momentum fluxes over both ocean and land, subgrid mixing within the ocean due to top and bottom boundary layers, gravity waves and unresolved eddies, land surface and sea ice properties are also important on weather and seasonal time scales.


A NGGPS modeling workshop and meetings of the Strategic Implementation Plan (SIP) Working Groups were held in August of 2017 at the National Center for Weather and Climate Prediction in College Park, Maryland, to finalize the SIP document that describes projects for the development of the Unified Forecast System (Link to SIP document). The future of physics parameterization development, as described in this report, listed many of the current DTC tasks. A possible DTC priority could be to test (at multiple resolutions), evaluate, and maybe even assist in tuning scale-aware and stochastic parameterizations for processes such as microphysics, cumulus convection and gravity wave drag.

To achieve the goal of involving the broader community in the development, testing, and assessment of physical parameterizations, the GMTB was established as a new task area in the DTC. The GMTB, led by Ligia Bernadet and Grant Firl, is an ambitious project to make model development much more user-friendly, catalyzing partnerships between EMC and research groups in national laboratories and academic institutions. The GMTB aims to implement transparent and community-oriented approaches to software engineering, metrics, documentation, code access, and model testing and evaluation.  

The initial charge to GMTB was the development and community support of the Common Community Physics Package, a software and governance framework to facilitate Research to Operation (R2O) transitions of community contributions. Additionally, the GMTB is tasked with defining a hierarchy of tests (model configuration, initial conditions, etc.), iteratively exercising each candidate physics configuration over the tests, and providing assessments in an open manner – tasks which are also needed for the development of unified metrics. GMTB already was of great help to implement the Grell-Freitas convective parameterization into the GFS physics package, now running in versions of FV3 and GFS.

The second essential project defined by the SIP physics working group is the design of unified metrics (or at least standardized metrics, dependent on application). The development of the Model Evaluation Tools (MET) is another component of DTC work. MET is designed to be a highly-configurable, state-of-the-art suite of verification tools. The development of the Model Evaluation Tools (MET), which is designed to be a highly-configurable, state-of-the-art suite of verification tools, is another component of DTC work. Current MET development efforts are focused on expanding its capabilities to capture the full range of EMC’s multiple verification packages.

With the existing task areas spanning ensemble work, stochastics, verification software development, community support, data assimilation, and GMTB, DTC will be able to play an increased role in assisting R2O transfer with respect to development and improvements in physical parameterizations. Once the advanced physics suite is selected, this will require gaining expertise in the different parameterizations that are chosen. DTC can conduct carefully controlled, rigorous testing, including the generation of objective verification statistics, and provide the operational community with an increased amount of guidance for selecting new NWP technologies with potential value for operational implementation.


NOAA ESRL GSD Georg Grell

Russ S. Schumacher

Autumn 2017

Faculty and graduate students at universities typically conduct basic research to better understand the fundamental workings of their area of interest, which in our field is the atmosphere. Transitioning these findings into practical applications, including operational weather forecasting, is then done by national labs and their cooperative institutes.  Yet in many cases, university researchers are working on problems that are directly relevant to operations, and have the potential (with a little help) to be considered for transition into the operational environment.  How to cross the many hurdles associated with this transition, however, is not a topic that is well understood in the academic lab setting, where the project may be developed by a faculty member and one or two graduate students.


Participants in the Flash Flood and Intense Rainfall experiment, forecast discussion at the Weather Prediction Center and Hydrometeorology Testbed during summer 2017.

I've collaborated closely with forecasters and forecast centers in the past, mainly on what might be called "operationally relevant" research -- work that can inform the forecast process but isn't immediately applicable.  This summer, I had my first real experience as a faculty member in formally testing a product that could be considered for operational transition. With support from NOAA's Joint Technology Transfer Initiative, we tested my Ph.D. student’s heavy rainfall forecasts at the Flash Flood and Intense Rainfall (FFaIR) experiment at the Weather Prediction Center (WPC) and Hydrometeorology Testbed during June and July of 2017.

Preparing for the experiment raised several issues of a scientific and technical nature, that I was not really accustomed to having to think about in an academic setting.  Some were fairly mundane, like "how do we generate files in the proper format for the operational computers to read them?"  But others were more conceptual and philosophical: “How should a forecaster use this product in their forecast process?  What should the relationship be between the forecast probabilities and observed rainfall/flooding?  How can we quantify flooding rainfall in a consistent way to use in evaluating the forecasts?”

So why do I bring all of these experiences up in the “DTC Transitions” newsletter? Because one of the DTC’s key roles is to facilitate these types of research-to-operations activities for the broader community (including universities as well as research labs.)  One particular contribution that the DTC makes to this effort is the Model Evaluation Tools (MET), a robust, standardized set of codes that allow for evaluating numerical model forecasts in a variety of ways.  For new forecast systems or tools to be accepted into operational use, they should demonstrate superior performance over the existing systems, and the only way to establish this is through thorough evaluation of their forecasts.  Careful evaluation can also point to areas for additional research that can lead to further model improvements.  The DTC also sponsors a visitor program that supports university faculty and graduate students to work toward operational implementation of their research findings.

Conducting research-to-operations activities in an academic setting will certainly fall outside the comfort zone of many university researchers.  Furthermore, we should be sure not to lose our focus on basic research, which is often best suited to academia.  But the fruits of that basic research are also often ready to take the next step to direct application and broader use, and I encourage fellow academics to test out taking that step, especially with the support and tools offered by the DTC.


Michael Farrar

Winter 2017

As the new Director of the Environmental Modeling Center (EMC), I have the pleasure of leading nearly two hundred world-class scientists, engineers and other staff in developing, transitioning, improving and maintaining a suite of global and regional environmental models to support the National Weather Service (NWS) mission of the protecting life and property for our country.  To meet the challenges of this mission, EMC has formed strategic partnerships with numerous community organizations over the years to collaboratively manage the existing suite of models, as well as to build the next generation of environmental models.  The DTC has been a vital partner for developmental testing, verification, and community support activities related to large segments of EMC’s modeling suite.  While the genesis of DTC began with WRF and so originally focused on regional models and related applications, the DTC has begun to expand their efforts into global modeling in support of the Next Generation Global Prediction System (NGGPS).  In particular, DTC has formed a Global Model Test Bed (GMTB) to support NGGPS development on a variety of tasks, most notably to assist in the development and testing of Common Community Physics Package and support for an associated interoperable physics driver. 

While the first step of the NGGPS program will be to migrate the legacy spectral model dynamic core of EMC’s Global Forecast System (GFS) to the Finite Volume Cubed Sphere version 3 (FV3) from NOAA’s Geophysical Fluid Dynamics Lab (GFDL), this represents much more than just a dynamic model core change to the global model. Instead it represents the first step toward migration of EMC’s modeling suite to a unified modeling system across both spatial (mesoscale/regional and global) and temporal (weather, sub-seasonal and seasonal) scales. With their legacy of mesoscale/regional applications and the new global modeling work under the GMTB umbrella, the DTC is ideally situated to play an integral role in helping EMC and NOAA evolve our legacy modeling suite towards a unified modeling system. 

The evolution towards a unified system is more than just a NOAA imperative; it is a National one. As such, I am happy to report that we recently brought the vast majority of NOAA organizations involved in model research, development, testing and operations together with several of our key national partners for a November meeting at the David Skaggs Research Center in Boulder, CO to start developing a short-term strategy for a National unified modeling system.  In addition to EMC (who organized and chaired the meeting), the other NOAA participants included 8 NOAA labs from the Office of Oceanic and Atmospheric Research (OAR) and the National Ocean Service (NOS); the NOS Center for Operational Oceanographic Products and Services (CO-OPS); the new NWS Office of Water Prediction (OWP); and 3 program offices from NWS and OAR.  Representatives of these NOAA organizations were joined by members of 3 Labs from the National Center for Atmospheric Research (NCAR), including the Research and Applications Lab (RAL) of which DTC is a part; the NASA Global Modeling and Assimilation Office (GMAO); the Naval Research Lab (NRL); and the Joint Center for Satellite Data Assimilation (JCSDA).  All these organizations came together with the intent to begin development of a Strategic Implementation Plan (SIP) that can orchestrate collaborative activities over the next 2-3 years under a single, coordinated “play book”. 

Following this successful meeting, the next steps will be the formation of working groups composed of experts from each of these organizations, other partner organizations, and members of the broader R&D community to tackle specific functional areas of the SIP, to include such areas as governance, system architecture, model dynamics, model physics, data assimilation, and post processing, to name a few.  The output of these working groups will be brought together in a public community workshop, targeted for Spring 2017, to pull together the first draft of a comprehensive, integrated plan.  The power of this approach will be to harness the collective resources of many of our country’s top R&D institutions along with other partners from the broader research community, who can all work together under a single, coordinated plan towards a common goal of building a truly National unified modeling system.  I and the other members of the Environmental Modeling Center look forward to working with the DTC and our other strategic partners as we work towards this common goal. 


Bill Kuo

Summer 2017

The Developmental Testbed Center (DTC) was established in 2003 with a mission to facilitate research to operation transitions in regional Numerical Weather Prediction (NWP). The DTC fulfills this mission by (i) providing community support for regional operational systems, (ii) performing testing and evaluation of NWP innovations for possible operational implementation, and (iii) promoting interaction and collaboration between research and operational NWP communities through special workshops, visitor programs, and the publication of this newsletter.


The Developmental Testbed Center (DTC) was established in 2003 with a mission to facilitate research to operation transitions in regional Numerical Weather Prediction (NWP).

When the DTC was first established, the initial focus was the Weather Research and Forecasting (WRF) model. In fact, during the early days the DTC was called the “WRF” DTC. Over the years, the scope and activities of the DTC have expanded in response to the needs of operational centers and the research community. Today, the DTC provides support for five community systems, including WRF, UPP (Unified Post-Processor), HWRF (Hurricane WRF), GFDL vortex tracker (a component of HWRF), GSI-EnKF data assimilation system, and the MET (Model Evaluation Tools) verification system. This work has been valuable in encouraging the research community to use operational NWP systems for research applications, and has contributed to their continued improvement.

Currently, efforts are grouped into five task areas focused on the operational systems the DTC supports: Regional Ensembles, Hurricanes, Data Assimilation, Verification and Global Modeling. The Global Modeling task was added in 2015, in response to the request from the NWS NGGPS (Next Generation Global Prediction System) Program Office, to establish a Global Model Test Bed (GMTB). The development and community support of a Common Community Physics Package are the initial foci of GMTB.

A key objective of the NGGPS program is upgrading the current operational Global Forecast System (GFS) to run as a unified, fully-coupled model within the NEMS (NOAA Environmental Modeling System) infrastructure. This unified model is expected to improve hurricane track and intensity forecast, and extend weather forecasting out to 30 days, in addition to other objectives. The NGGPS program presents an exciting opportunity for the U.S. NWP community to collaborate on the development of a single modeling system, which can then be used to support both the research and operational sectors.

In the last issue of the DTC Transitions Newsletter, Environmental Modeling Center (EMC) Director Mike Farrar articulated how the migration of the legacy GFS spectral model dynamic core to the FV3 (Finite Volume Cubed Sphere Version 3) core represents a first step toward unified modeling. The consolidation of EMC’s modeling suite will concentrate resources and result in considerable savings.

The DTC Executive Committee has asked the DTC to develop a strategic vision for the next 10 years to ensure it evolves hand-in-hand with the operational center, and aligns its activities in support of the unified modeling transition. This is a very welcome request -- it has been challenging to support multiple modeling systems with limited resources.

This summer, DTC staff will begin to develop this new vision. The overarching question is: “What is the role of the DTC in the NGGPS era with unified modeling?” How should a complicated unified and fully-coupled Earth system model with multiple components be supported to the community, and what is the DTC’s role in the support of such a system? With limited resources, where should the DTC focus its testing and evaluation efforts to most effectively facilitate R2O transition? How effective are the DTC community engagement activities and should they be revised to further support collaboration and interaction between research and operational NWP communities?

Our goal is to develop a draft strategic vision by the end of the summer. The DTC Strategic Vision will be vetted with the DTC Science Advisory Board and the DTC Management Board, before it is submitted to the DTC Executive Committee for consideration.


Tom Hamill

Winter 2016

I’d like to highlight some recent work in model diagnostics by my one-time colleague in NOAA Physical Science Division, Thomas Galarneau. Tom was funded under a DTC visitor’s program grant, with additional funding through the Hurricane Forecast Improvement Project. As a DTC visitor in 2013, Tom applied a new diagnostic method for quantifying the phenomena responsible for errors in tropical cyclone (TC) storm tracks to an inventory of recent hurricanes. (See a related article about Tom’s work in the DTC Newsletter Winter-Spring 2014, page 4.) Tom has since moved on to NCAR, and then more recently to the University of Arizona.

Readers are likely aware that the prediction of tropical cyclone track, intensity, and genesis in the medium-range (120–180-h forecast leads) continues to be a formidable challenge for operational numerical weather prediction models.

While 36–84-h TC track predictions from the operational NCEP Global Forecast System (GFS) have significantly improved over the last ten years, predictions for forecast leads beyond 120-h have shown no such improvement. In order to examine and diagnose medium-range forecasts for the NCEP GFS, Tom and his NCAR colleagues Chris Davis and Bill Kuo developed the Real-Time Diagnosis of GFS Forecasts website [http://www.atmo.arizona.edu/~tgalarneau/realtime/gfs_diagnostics.html] that provides analyses and diagnoses of GFS forecasts in real time. To complement other websites that show the statistics of GFS forecast performance, this website provides synoptic charts and diagnostic calculations to link model physical processes to the evolution of the synoptic-scale flow in the GFS forecast. Analyses and diagnostics are generated four times daily, and they provide information on how the GFS has behaved over the previous 60-day period. A wide variety of charts can be generated at the web site above.

Consider one relatively simple example of a systematic error in the GFS identified with this web site. Since its inception, continuous monitoring of the GFS forecasts has revealed that the GFS systematically loses atmospheric water vapor over the tropical west Pacific in quiet regimes during the summer months. It appears that the GFS atmosphere overturns early in the forecast, producing a positive rainfall bias in the day-1 forecast over the tropics. The GFS atmosphere does not recover from the stabilization following the early burst of rainfall due to a low surface flux bias. As a consequence, the GFS atmosphere continues to dry and stabilize through the medium-range, resulting in the negative rainfall bias over the tropics by day 7. The drier conditions make it difficult for the GFS to accurately predict TC genesis in the medium range. Examination of rainfall forecasts over the last 60 days of 2015 shows that the dry bias in the tropics is also a problem during the cool season. The attached figure shows a systematic inability of medium-range GFS forecast to maintain tropical rains associated with a westerly wind burst along the equator in the tropical west-central Pacific. With tropical cyclone formation and propagation affected by this bias, one can expect that tropical-midlatitude interactions in the eastern Pacific are affected, which of course can affect the accuracy of downstream forecasts over the US.

Identification of major systematic biases in our forecast model is the first step toward better predictions. I personally would like to see DTC do even more in this arena, producing diagnostics that further aid the model developer in teasing out the underlying sources of model biases. Can the systematic errors Tom identified be attributed to convective parameterizations? To cloud-radiative feedbacks? The faster we can diagnose the ultimate potential sources of forecast bias, the more rapidly we can reallocate the development resources to address them, thus speeding the rate of forecast system improvement. Tom and his colleagues pioneering work is a very admirable first step in this process, for which NOAA and the weather enterprise is grateful.

Tom Hamill is a research scientist who is interested in all aspects of ensemble weather and climate prediction, from data assimilation to stochastic parameterization to post-processing and verification. Tom also is currently a member of the DTC management board and is a co-chair of the World Meteorological Data Assimilation and Observing Systems working group.


Time-mean (a) CMORPH-derived * and (b) GFS day-7 forecast (0000 UTC initializations only) daily rainfall (shaded in mm) for the 60-day period ending on 1 Jan 2016. * NOAA’s Climate Prediction Center MORPHing (CMORPH) technique.

Ralph Stoffler

Spring 2016

Ralph Stoffler, a member of the US government Senior Executive Service, is the USAF Director of Weather, and the USAF Deputy Chief of Staff for Operations. In this capacity, he is responsible for the development of weather and space environmental doctrine, policies, plans, programs, and standards in support of Army and Air Force operations. He is responsible for overseeing and advocating for Air Force weather resources and monitors the execution of the weather program. He is the functional manager for 4,300 total-force weather personnel and interfaces with Air Force major commands and the U.S. Army regarding full exploitation of Air Force weather resources and technology. He is also an AF representative on the DTC Executive Committee. See article on this page about USAF perspectives with the DTC and more.


Ralph Stoffler, Director of Weather, US Air Force

Paula Davidson

Autumn 2016

NOAA’s testbeds and proving grounds (NOAA TBPG) are an important link between research advances and applications, and especially NOAA operations. Some are long-recognized, like the Developmental Testbed Center (DTC), while others have been chartered more recently. With the 2015 launch of the Arctic Testbed in Alaska, twelve NOAA TBPG follow execution and governance guidelines to be formally recognized by NOAA. These facilities foster and host competitively-selected, collabora- tive transition testing projects to meet NOAA mission needs. Projects are supported through dedicated or in-kind facility sup- port, and programmatic resources both internal and external to NOAA. Charters and additional information on NOAA TBPG, as well as summaries of recent coordination activities and workshops, are posted at the web portal. See www.testbeds.noaa. gov.

NOAA’s testbeds and proving grounds (NOAA TBPG) are an important link between research advances and applications,and especially NOAA operations. Some are long-recognized, like the Developmental Testbed Center (DTC), while others have been chartered more recently. With the 2015 launch of the Arctic Testbed in Alaska, twelve NOAA TBPG follow execution and governance guidelines to be formally recognized by NOAA. These facilities foster and host competitively-selected, collabora- tive transition testing projects to meet NOAA mission needs. Projects are supported through dedicated or in-kind facility sup- port, and programmatic resources both internal and external to NOAA. Charters and additional information on NOAA TBPG, as well as summaries of recent coordination activities and workshops, are posted at the web portal. See www.testbeds.noaa. gov.

Along with adopting systematic guidelines for function, execution, and governance of NOAA TBPG, in 2011 NOAA instituted formal coordination among the TBPG, to better leverage progress across the spectrum of testing, and provide a consistent voice and advocacy for programs and practices involving the TBPG. The coordination committee hosts annual workshopsfeaturing collaborative testing on high-value mission needs, fosters practices consistent with rigorous, transparent testing and increased communication of test results, and provides a forum to advance program initiatives in transitions of research to operations and of operations to research.

NOAA’s TBPG conducts transition testing to demonstrate the degree of readiness of advanced research capabilities for operations/applications. Over the past two years, these facilities completed more than 200 transition tests, demonstrating readiness for NOAA operations for more than 70 candidate capabilities. More than half have already been deployed. Beyond the simple transition statistics, NOAA TBPG have generated a wealth of progress in developing science capabilities for use by NOAA and its partners through more engaged partnerships among researchers, developers, operational scientists and end- user communities. Incorporating appropriate operational systems and practices in development and testing is a key factor in speeding the integration of new capabilities into service and operations.

DTC, in collaboration with public and private-sector partners, plays an increasingly important role in NOAA transitions of advanced environmental modeling capabilities to operations, and with rigorous testing to evaluate performance and potential readiness for NOAA operations. Readiness criteria include capability-specific metrics for objective and subjective performance, utility, reliability and software engineering/production protocols. DTC facilitates R&D partners’ use of NOAA’s current and developmental community modeling codes in related research, leading to additional evaluation and incorpora- tion of partner-generated innovations in NOAA’s operational models.

NOAA programs that have recently supported projects conducted at NOAA TBPG, and especially at DTC, include the Next Generation Global Prediction System (NGGPS), Collaborative Science and Technology Applied Research Program, Climate Program Office, the US Weather Research Program, and the Hurricane Forecast Improvement Program. Under NGGPS auspic- es, the DTC added a new unit for testing prototypes for the NOAA’s next global prediction system. DTC’s contributions to the success of NGGPS will be the foundation for improved forecasts in critical mission areas such as high-impact severe/extreme weather in the 0-3 day time frame, in the 6-10 day time frame, and for weeks 3-4. As chair of NOAA’s TBPG coordinating committee, I am excited about the tremendous opportunity and capability that the DTC brings to these efforts to enhance NOAA’s science-based services.

Vijay Tallapragada EMC

Summer 2016

My association with the Developmental Testbed Center (DTC) dates back to early 2008 when the NCEP operational Hurricane Weather Research and Forecast (HWRF) modeling system developed at Environmental Modeling Center (EMC) was adopted to create a community modeling framework for hurricane model development supported by the Hurricane Forecast Improvement Project (HFIP). I enjoyed working with DTC in various capacities as the Hurricane Team Leader at EMC and the Development Manager of HFIP.

Operational hurricane model development undoubtedly has been one of the most successful initiatives by HFIP project that enabled a process for effective transition of research to operations (R2O), and the outcome is clearly evident in terms of tangible improvements in hurricane track and intensity forecast guidance from operational HWRF as demonstrated in real-time in the past few years. HWRF model has evolved as a unique high-resolution atmosphere-ocean-wave coupled system for all global tropical cyclones, serving various operational forecast agencies, researchers and private industry. HWRF is the only operational hurricane model in the world freely distributed and supported to the research community through extensive documentation, in-person and online tutorials, and user guides.

While HWRF’s development is centralized at EMC, it incorporates contributions from a variety of scientists spread out over several governmental laboratories and academic institutions. This distributed development scenario poses significant challenges: a large number of scientists need to learn how to use the model, operational and research codes need to stay synchronized to avoid divergence, and promising new capabilities need to be tested for operational consideration. DTC’s contributions for HWRF are pivotal in the areas of code management, advanced support for model developers and general users, and extensive testing and evaluation of new innovations.

My recent transition as Chief of the Global Climate and Weather Modeling Branch (GCWMB) at EMC coincided with two other major initiatives – the National Weather Service (NWS) Next Generation Global Prediction System (NGGPS) project, and the expansion of DTC’s role into global modeling through creation of the Global Modeling Test Bed (GMTB). Operational global modeling at NOAA/NCEP is the most significant attribute for the entire US weather enterprise. The NCEP Global Forecast System (GFS) stands as a back bone for the whole operational production suite. GCWMB at NCEP/EMC is responsible for developing, implementing and advancing the global modeling system that spans from weather to climate scales. With support from the NGGPS project, a new non-hydrostatic dynamic core is being selected for replacing the current spectral model for serving the future needs of the NWS. A community based NOAA Environmental Modeling System (NEMS) architecture will enable seamless development and integration of atmosphere, ocean, land, sea-ice, waves, and aerosols for weather, sub-seasonal and seasonal forecast capabilities. Similar to the HFIP project, the emphasis of NGGPS is to foster enhanced R2O capabilities for accelerated model development and transition to operations. GMTB has embarked on designing a Common Community Physics Package (CCPP) within NEMS that allows for systematic evaluation of advanced physics.

There is enormous talent in the US NWP community that has largely been untapped. With a focused approach, we can bring together the best in the field by adopting the community modeling concept. I am looking forward to continue working with DTC to strengthen the relationship between operational and research communities, and to realize the goals of NGGPS in becoming second to none in global weather prediction.


Frederick Toepfer

Summer 2015

The Next Generation Global Prediction System (NGGPS) project is a National Weather Service initiative to design, develop, and implement a next generation global prediction system to take advantage of evolving high performance computing architectures, to continue pushing toward higher resolutions needed to increase forecast accuracy at shorter timeframes, and to address growing service demands for increased skill at extended ranges (weeks to months). The NGGPS project goals are: (1.) expansion and implementation of critical weather forecasting research to operations (R2O) capabilities to accelerate the development and implementation of global weather prediction model advances and upgrades; (2.) continued improvement in data assimilation techniques; and (3.) improved software architecture and system engineering to support broad community interaction with the system. The introduction of Community Codes for all major components of the NGGPS and the establishment of a Global Modeling Test Bed (GMTB) to oversee community interaction with the codes are significant changes to the National Weather Service business model to advance numerical weather prediction skill in the US. Over the next five years, contributions from a wide sector of the numerical weather prediction community including NCEP, NOAA and other agency laboratories and private sector and universities, will be incorporated into an upgraded operational system to deliver a NGGPS that meets the evolving national prediction requirements.

Major work areas in the NGGPS project include selecting and further developing a new atmospheric dynamic core and improving model physics to better describe phenomenon at global to regional scales. Additional work will be conducted to accelerate development and implementation of weather prediction model components such as ocean, wave, sea ice, land surface and aerosol models, and improve coupling between these various components of the model system.

The DTC will play an important role in the NGGPS project. The new GMTB has been established as an extension of the current DTC. The GMTB is initially funded by the NGGPS project to assist in the development and testing of a Common Community Physics Package, as well as provide code management and support for an associated interoperable physics driver. The GMTB will also assist the NGGPS project in the development and testing of a sea ice model.

Active community participation in the development of the NGGPS is considered key to the success of the project. The DTC is perfectly positioned to assist in this aspect of the project. As the NGGPS Project Manager, I am excited about the DTC’s role, through the GMTB, in the project and anticipate a productive and successful relationship going forward.

Barb Brown

Winter 2015

The DTC was established a number of years ago to facilitate the transition of new capabilities in weather forecasting from research to operations (R2O), with a focus on the WRF model. Over the years, many things have changed – for example, the DTC now works with multiple sets of code (WRF, GSI, MET, HWRF, and so on) which will soon include global prediction systems – but that fundamental mission remains the same. The DTC accomplishes its goals through strong connections to the operational and research communities. These dual connections, forming the bridge between the communities, are what make the DTC unique.

In fact, the bridge is the key aspect of the DTC that has led to its success, and will lead to additional successes in the future as the DTC continues to grow and mature.



Connecting the research and operational communities through workshops (e.g., the recent physics workshop, see page 4), support and training on operational codes, and the DTC visitor program provide the keys to developing relationships that will lead to new successes in R2O. Moreover, the DTC’s independent testing and evaluation of new innovations developed by the research community, and its efforts to enable such testing by the research community (e.g., through the Mesoscale Modeling Evaluation Testbed, MMET) help speed the identification and transfer of new capabilities across that bridge. These key factors have the potential to lead to a vibrant and well connected R2O process.

It has been a great pleasure for me to work closely with the DTC over the last six years as a member of the Management Board and as the Director of NCAR’s Joint Numerical Testbed Program (JNTP). I feel lucky to be part of this grand effort to improve forecasting for our nation through community activities, and will enjoy watching the success of the DTC in the years to come. It is with pleasure that I hand over the reins of the JNTP to Dr. Joshua Hacker, who will bring new leadership, ideas, and energy to the DTC effort.

Bob Gall

Summer 2014

I was part of the Development Testbed Center from its beginnings as part of the WRF (Weather Research and Forecasting) model development, which in turn was part of the US Weather Research Program (USWRP). The years are beginning to blur for me but I believe the first discussions of a DTC in Boulder were during an IWG (Interagency Working Group) meeting of the USWRP at NCAR on October 22, 2002. At that meeting the vision for the DTC was stated as a facility which would:

• Provide for a rapid and direct transfer of new NWP research results into operational forecasting

• Evaluate strengths and weaknesses of new methods and models for NWP prior to consideration for operational implementation

• Evaluate strengths and weaknesses of current operational systems

And later we added:

• Do these in a way that doesn’t interfere with operations

More discussions followed, but the DTC project basically was underway by the next summer. I led the DTC from its inception until I left to be Development Manager of the HFIP (Hurricane Forecast Improvement Program) in 2009. During that time Steve Koch and I gradually built up the project both at NCAR and at GSD/ESRL in Boulder as a joint agency effort. Louisa Nance was its first employee and has been with the program ever since. The program was gradually expanded to what you see today and Bill Kuo took over as Head after I left.

As we began to spin up the HFIP Project in 2009 we realized early that one of its goals needed to be to make the operational NWP hurricane model system widely available. Only in that way could HFIP make effective use of ideas and technology from the community (broadly defined as university, government laboratory and other folks). The mission of EMC is to develop, test and implement the operational system—a full time job for the HWRF group—and they are not equipped to deal extensively with making the codes available to the community.

The DTC, on the other hand, was equipped for this task, and thus early in the HFIP program we began to fund a significant program to make the HWRF available to the community.

It was our intention to focus the HFIP program on a single model system (like the original idea for the WRF system) as a way to make maximum progress in improving the hurricane forecast guidance system. From the beginning we felt that system needed to be HWRF, principally because that system was being developed at EMC at the time and there was a team there focused on all aspects of the model development (core, physics and the initialization system) and how they all integrated together. The only other similar team in the US was the one running TC-COAMPS for NRL. Such a team was not in place for other hurricane NWP forecast systems in the US, such as the AHW (Advanced Hurricane WRF) being developed at NCAR. Since the central goal of HFIP is to develop the NCEP operational hurricane system into the best in the world, HWRF was the obvious choice. The DTC, which had an extensive knowledge base for making codes available to the community and to handle interactions with the community for HWRF, was also an obvious choice. HFIP has provided significant funding for the last several years to the DTC to set up and make available a code system in Boulder for HWRF, including documentation, and to work with EMC to coordinate that code with the most current operational HWRF codes. In addition we also provided funding for university projects to work with these codes. The end result, for HWRF, is a paradigm that is essentially equivalent to the original vision for both WRF and the DTC given above.

DTC Director

Winter 2014

One outcome of the DTC Science Advisory Board (SAB) meeting last September was establishment of an annually rotating SAB Chair. As the first to assume the role under this change, I welcome the future opportunity to address the DTC from the perspective of a researcher. Taking over from last year’s chair Mark Stoelinga is no small task; Mark efficiently and effectively led our discussions and drew several points of consensus from among many disparate ideas and concerns. I encourage those interested to browse the report that resulted from our meeting last September (which can be found on the dtcenter.org website). Here I step aside from Chair duties, and instead put forth what I see as the DTC’s key strengths, challenges, and opportunities.

During my short two years on the SAB, the DTC has made noteworthy improvements in its ability to disseminate and support operational codes, such as the WRF-NMM and the GSI. Although many point out that the DTC’s primary goal is capability transfer from research to operations, that goal cannot be realized without first opening the doors for the research community to adopt operational codes and methods. The ARW is well established in the research community, and may be difficult to supplant at any significant level. The GSI may prove more successful, and the DTC’s experience and support of the GSI opens a plausible path to a success in research to operations. Consider the following scenario: university investigators continue to expand use of complex data assimilation codes, and university use of the ARW-compatible GSI grows commensurately. With NCEP and AFWA invested in the GSI for some years to come, a viable opportunity for research to operations transfer emerges. With convection-allowing forecasts offering several important science challenges, it is not a stretch to think that a university investigator with sufficient computational resources would find a RAP or HRRR-like system desirable to facilitate research in the near future. NCEP and AFWA may see tangible contributions that follow.

In the present environment defined by funding uncertainty, the DTC continues to face the challenge of balancing its core mission against funding opportunities that may redirect limited staff time. A step into global modeling could present one such difficulty. Suppose an opportunity emerges to begin testing global models. In the absence of increased total funding, staff would necessarily be diverted away from valuable mesoscale model testing, and from supporting operational codes to the community. It is unlikely that large numbers of university investigators will, in the next few years, be running operations-grade global models with regularity and resolution needed to inform operational centers. Maintaining focus on growing strength in mesoscale model testing, especially at convection-allowing scales, should position the DTC for future research to operations transition opportunities.

Finally, the rising importance of the HRRR and HRRRE, and the continued emphasis on probabilistic forecasting at AFWA, give the DTC its greatest opportunity to realize research to operational transitions. The HRRR, and nearly all of AFWA’s operational NWP, are based on the widely used ARW. As noted above, the recent success in hosting GSI workshops, and supporting the code to the research community, positions DTC for future success.

After what some perceive as struggles during its first few years, the DTC has laid a solid foundation. Independent testing of operational models continues to be valuable, particularly to AFWA; the visitor program remains popular and effective at offering operational-relevant problems to the community. By remaining focused on its strengths while continuing its work making operational codes available to the research community, the DTC should realize more future success in line with its core mission.

Kevin Kelleher

Autumn 2014

During my first 15 months as the ESRL Global Systems Division Director, I have learned about the DTC and its role in the modeling community. The DTC has made remarkable gains in supporting the WRF model within the community that has contributed to the great success and usage of the model both nationally and internationally. I believe the DTC is unique in how it is funded and operated as a joint effort between NCAR and NOAA, along with partners from the Air Force. It is my observation that there is a significant effort to develop global models at resolutions traditionally associated with mesoscale/ regional models. Therefore, it is a good time for the mission of the DTC to be reviewed and possibly updated such that the DTC has a viable and robust future, should global models eventually begin to replace mesoscale/regional models within NOAA NCEP operations, for example. At GSD, we have recently reorganized in response to these changes in the modeling community.

All of our modeling efforts are now under a single Branch, the Earth Modeling Branch led by Dr. Stan Benjamin. Nearly all of the GSD DTC efforts now fall within this Branch, which includes researchers working on modeling scales from the storm scale through global scale. Having convenient access to such a wide range of talented personnel should benefit DTC tasking in the future. In addition, in my role as a member on the DTC Management Board, I have begun to work closely with NWS NCEP & EMC and Dr. Bill Kuo to work toward improving alignment of the current NCEP operational needs with the DTC mission, capabilities, and services.



Welcome Message by Bill Kuo

Spring 2013

Welcome to the first issue of a quarterly newsletter for the Developmental Testbed Center (DTC). The research to operations (R2O) transition in numerical weather prediction (NWP) is a major challenge facing the U.S. meteorological community. It has been recognized that the U.S. has the largest community around the world working on weather research and numerical modeling. Yet, most of these research results do not directly benefit operational NWP. The DTC was established in 2003 with a mission to facilitate the transition of research innovations in regional modeling into operations.

An effective R2O process requires active participation of research and operational communities. With this quarterly newsletter, we hope to provide a forum for discussion of important issues facing the NWP community. We will also provide updates on DTC activities that are of interest to the community.

We welcome articles submitted for consideration for publication in upcoming issues.


Bill Lapenta

Summer 2013

Dear Colleagues,

The end-to-end modeling systems in the NOAA operational numerical guidance suite are scientifically based, and research results must and do cross the “Valley of Death” into operations. However, the operational and research communities need to make this journey more efficient and cost effective. That’s one reason why we have testbeds like the DTC. During 20 years as a research scientist at NASA, I had the opportunity to work closely with NWS forecasters in Huntsville and offices across the Southeast. When I accepted a job at NOAA with EMC, I thought my understanding of what it takes to work in an operational environment was sound based on these earlier experiences. However, it soon became apparent that my perceptions about the transition of research into operations were woefully incomplete. I believe that there are many ways NOAA can build a better transition process between research and operations, and I would like to share my thoughts associated with the upgrade process at EMC in future releases of the DTC Newsletter.

Greetings from the Heartland!

Autumn 2013

The Air Force Weather Agency (AFWA) has long been a partner with the DTC since its “unofficial” early days of “core testing” (remember those), to its official charter membership signing in September of 2009, to date. Air Force Weather (AFW) recognized early, the essential role this organization could, and would, play in bolstering and mitigating an ever-growing US Air Force resource constrained terrestrial weather RDT&E environment. Further partnering with our NOAA National Centers for Environmental Prediction (NCEP) compatriots in this endeavor became even more than doubly beneficial to AFW mission needs. For over a decade, AFW has not had a terrestrial weather R&D lab to foster continuous environmental NWP advancements.

Among other alternative research avenues, the DTC was seen as a leveraging mechanism to enable and smooth the transition of terrestrial weather advancements into the operational NWP weather tool used by the USAF…WRF. After slow going in the early years, over the last 4+ years the DTC has been that research to operations (R2O) enabler we expected, but not in the conventional thinking sense many have of the DTC.

The highest priority mission AFWA has for the DTC is reference configuration testing and evaluation (T&E). T&E is an essential last step in AFW’s R2O process. To facilitate this, the DTC has set up a nearly “functionally equivalent” operational design of AFWA’s WRF model operations. In the past four WRF community release cycles, the DTC has T&E’d several promising reference configurations of WRF against AFWA’s operational configuration providing the final actionable detail needed to decide whether the new scientific advancement has a positive operational impact worthy of implementation. Having this fidelity enables AFWA to greatly reduce its R2O timelines if it otherwise had to rely on its own available resources.

Furthermore, the modeling community benefits from these configurations tests by building a performance baseline to track reference configurations. This should guide scientists toward fruitful R&D tracks and steer them from unfruitful approaches, ultimately providing further R2O efficiencies.

This T&E focus for R2O is why AFW has funded a Model Evaluation Tool (MET) solely developed and matured by DTC. A standardized tool, for standardized tests, for our common R2O future…a great beginning and a valuable partnership—DTC, AFW, & NOAA/NWS.