Tests Conducted at DTC Lead to Operational Implementation of Innovations in Physics and Data Assimilation in HWRF

Summer 2018

The Hurricane Weather Research and Forecast system (HWRF) is one of NOAA’s operational models used to predict the track, intensity, and structure of tropical cyclones. Each winter, scientists at NOAA’s Environmental Modeling Center (EMC), the Developmental Testbed Center (DTC), and NOAA’s Hurricane Research Division (HRD) perform testing and evaluation (T&E) on possible changes to the HWRF physics schemes, dynamic core, and data assimilation system that have the potential to improve HWRF predictions. Many of these potential changes are innovations from the research community that have been added to branches within the HWRF code repository in the past year, with guidance from the DTC. These branches are then retrieved from the repository by EMC and DTC staff to perform annual T&E. This yearly upgrade cycle illustrates the seamless exchange of innovations from the research community to operational testing environments, which is facilitated by the code management and developer support provided by the DTC.

This year, the DTC effort focused on T&E of two potential upgrades to model physics. The first looked at upgrades to the Rapid Radiative Transfer Model for General Circulation Models (RRTMG) radiation scheme made available by John Henderson and Michael Iacono of Atmospheric and Environmental Research (AER) through the DTC Visitor Program. The second replaced the Scale-Aware Simplified Arakawa-Schubert (SASAS) cumulus scheme with the Grell-Freitas scheme, based on work by Georg Grell (NOAA’s Global Systems Division), Saulo Freitas (NASA), and Evelyn Grell (NOAA’s Physical Sciences Division) that was funded by the Hurricane Forecast Improvement Project (HFIP). The DTC also participated in several experiments led by HRD and EMC to determine the impact of assimilating additional data to improve the HWRF initial conditions.

Each of these potential upgrades was first tested individually by running retrospective HWRF forecasts on a subset of tropical cyclones from the past three years in the North Atlantic ocean. For these initial tests, EMC selected sixteen storms that provided a mixture of storm intensities, storm motion directions, and previous operational model performance. For the RRTMG radiation scheme upgrades, the DTC ran nine of the sixteen storms before EMC staff decided the forecast improvements (~4% for both track and intensity) merited including the changes in the 2018 version of HWRF. Results from the Grell-Freitas experiment indicated the scheme was not yet ready for operational implementation. However, the results are informing additional changes to the code by the developers, who are working with DTC and EMC staff to test an improved version of their scheme later this summer. For the data addition experiment, the DTC ran 2–3 storms for each additional data type, which helped EMC determine that wind data from the Stepped Frequency Microwave Radiometer and inner-core dropsondes should be assimilated into HWRF in 2018.

Once the 2018 configuration of HWRF is finalized, DTC and EMC will work together to merge the final version of the code back to the HWRF trunk. This step will enable researchers to add additional innovations to the latest version of the code, ensuring that any scientific results are directly applicable to the operational HWRF, which will position the community well for next year’s HWRF pre-implementation tests. With additional opportunities for transition of research to operations in upcoming versions of the model, DTC staff look forward to continuing to lend their expertise in code management, developer support and T&E to the community!


Mean Track Error - Atlantic Basin (land and water)
Figure 1. Mean track errors with respect to forecast lead time for HWRF with RRTMG radiation scheme upgrades (H18R, red line) and the HWRF control (H18C, black line) experiments. Pairwise differences (H18C minus H18R) are shown in blue with 95% confidence intervals. Solid blue circles indicate lead times with statistically significant differences. The number of cases at each lead time is shown in gray at the top of the figure.



Mean absolute intensity errors
Figure 2. Mean absolute intensity errors with respect to forecast lead time for HWRF with RRTMG radiation scheme upgrades (H18R, red line) and the HWRF control (H18C, black line) experiments. Pairwise differences (H18C minus H18R) are shown in blue with 95% confidence intervals. Solid blue circles indicate lead times with statistically significant differences. The number of cases at each lead time is shown in gray at the top of the figure.


DTC MET Verification Tutorial

Spring 2018

The  DTC Verification team hosted a MET tutorial at NCAR January 31-February 2, 2018 in association with the semi-annual WRF tutorial. This event was the first in-residence MET tutorial since February 2015. There were 31 registered users and several new DTC staff dropped in for pertinent lectures.  The tutorial included a half day of lectures on verification basics plus two and a half days of presentations focused on many of the MET tools supplemented with practical sessions that demonstrated the tool.  The last day also included training on the METViewer database and display system, used for aggregating, stratifying and plotting statistics, and the newly developed MET+ python wrappers.

METv7.0 was released on March 5th. For more information on MET capabilities, check out the MET Users’ Page: https://dtcenter.org/met/users/

Evaluation of the new hybrid vertical coordinate in the RAP and HRRR

Autumn 2017

The terrain-following sigma coordinate has been implemented in many Numerical Weather Prediction (NWP) systems, including the Weather Research and Forecasting (WRF) model, and has been used with success for many years. However, terrain-following coordinates are known to induce small-scale horizontal and vertical accelerations over areas of steep terrain due to the reflection of topography in the model levels.  These accelerations introduce error into the model equations and can impact model forecasts, especially as errors are advected downwind of major mountain ranges.

This is a cross-section plot of one set of cold-start RAP simulations. The figure highlights the reduction in spurious noise above the Rocky Mountains.

Efforts to mitigate this problem have been proposed, including Klemp’s smoothed, hybrid-coordinate, in which the sigma coordinate is transitioned to a purely isobaric vertical coordinate at a specified level.  Initial idealized tests using this new vertical coordinate showed promising results with a considerable reduction in small-scale spurious accelerations.  

Based on these preliminary findings, the DTC was tasked to test and evaluate both the hybrid vertical coordinate and the terrain-following sigma coordinate within the RAP and HRRR forecast systems to assess impacts on retrospective cold-start and real-time forecasts.

The DTC conducted several controlled cold-start forecasts and one cycled experiment with the 13 km RAP, initialized from the GFS. This sample included days with strong westerly flow across the western CONUS, favoring vertically propagating mountain wave activity.  In addition, one cycled, 3-km HRRR experiment was initialized from the non-hybrid coordinate RAP.  The only difference between these retrospective runs was the  vertical coordinate.

This sample of forecasts indicated the hybrid vertical coordinate produced the largest impact at upper levels, where the differences in coordinate surfaces are most pronounced due to the reflection of terrain over mountainous regions.  As a result, wind speeds with the hybrid coordinate were generally increased near jet axes aloft as vertical and horizontal mixing of momentum decreased when compared with the terrain-following coordinate.  In addition, the depiction of vertical velocity at upper levels was greatly improved with reduced spurious noise and better correlation of vertical motion to forecast jet-like features.  A corresponding improvement was found in upper-level temperature, relative humidity, and wind speed verification when using the hybrid vertical coordinate.

The hybrid vertical coordinate will be implemented in the operational versions of RAPv4 and HRRRv3 in 2018.

This work was a collaborative effort between NOAA GSD, DTC, and NCAR MMM.


The 2016 Hurricane WRF System

Winter 2017

The community Hurricane Weather Research and Forecasting (HWRF) modeling system was upgraded to version 3.8a on November 21, 2016.  This release includes all components of the HWRF system: scripts, data preprocessing, vortex initialization, data assimilation, atmospheric and ocean models, coupler, postprocessor, and vortex tracker.  In addition to default operational features, the release includes capabilities to perform idealized tropical cyclone simulations run with alternate physics, and backwards compatibility for inner nest grid sizes.

The HWRF community modeling system currently has over 1300 registered users.  The public release includes updates to the user webpage, online practice exercises, datasets, and extensive documentation.  The release code is fully supported, with community support provided via the HWRF helpdesk, hwrf-help@ucar.edu.

Information about obtaining the codes, datasets, documentation and tutorials can be found at http://www.dtcenter.org/HurrWRF/users

Soil moisture sensitivity plots illustrating the new idealized capability. As soil moisture increases from left to right, the storm intensity increases (contoured values). Courtesy of Subramanian, 2016.

The NCEP 2016 operational implementation of HWRF and the HWRF v3.8a community release are compatible systems.  Starting in 2016, the default configuration runs with ocean coupling for all northern hemisphere oceanic basins, and uses Real-Time Ocean Forecast System (RTOFS) data for ocean initialization in the Eastern North Pacific Basin.  Two specific capabilities, a 40-member HWRF ensemble for the assimilation of Tail Doppler Radar (TDR) data that NCEP began running in 2015, and the addition of one-way wave coupling using WAVEWATCH III in 2016, are not currently supported to the general community.

  • Other notable upgrades in HWRF version 3.8a include:
  • Code upgrades including, WRF v3.8, GSI v3.5, and UPP v3.1.
  • Inner domain (d02, d03) sizes increased to 25ºx25º and 8.3ºx8.3º, respectively.
  • Reduced time step from 38 4/7 s to 30 s.
  • Data assimilation enabled by default for both Atlantic and Eastern North Pacific Basins.
  • Improved physics for all scales:
    • Cumulus parameterization updates, including enabling by default for all 3 domains and a new Scale Aware Simplified Arakawa Shubert (SAS) scheme.
    • New GFS Hybrid-Eddy Diffusivity Mass Flux PBL scheme.
    • Updated momentum and enthalpy exchange coefficients (Cd/Ch).
    • Enhanced Idealized capability with landfall.
  • Enhanced products including simulated brightness temperatures for new satellite sensors in all basins.

DTC visitor contributes enhanced idealized capability

As noted in the HWRFv3.8a updates, an enhanced idealized capability to include simulated landfall using the GFDL slab land surface physics scheme is included in the v3.8a release. This capability was contributed through a successful DTC visitor project by Subashini Subramanian (Purdue University), “Developing Landfall Capability in Idealized HWRF for Assessing the Impact of Land Surface on Tropical Cyclone Evolution”.  The new feature introduces a namelist switch for allowing the landfalling capability, which specifies the type of land surface and an initial land-surface temperature to be used over land.  The default configuration introduces a homogeneous land surface that can be modified to account for heterogeneity. Additionally, the direction of land motion is a user-defined option. Work is underway to extend this capability to include other land-surface physics options.

The Unified Post Processor

Summer 2017

Post-processing is an essential but often overlooked component of numerical weather prediction and encompasses a broad range of concepts, methods, and tools to make raw model output more useful. The Unified Post Processor (UPP) can compute a variety of diagnostic fields, interpolate to pressure levels or specified (pre-defined or custom) grids, and de-stagger grids. Examples of the products include:

  • T, Z, humidity, wind, cloud water, cloud ice, rain, and snow on isobaric levels
  • SLP, shelter level T, humidity, and wind fields
  • Precipitation-related fields
  • PBL-related fields
  • Severe weather products (i.e. CAPE, Vorticity, Wind shear)
  • Radiative/Surface fluxes
  • Cloud related fields
  • Aviation products
  • Radar reflectivity products
  • Satellite look-alike products

The UPP produces GRIB1 and GRIB2 output files that can be used directly by a number of plotting packages and the Model Evaluation Tools (MET) verification package.

UPP Components version 3.1.

The UPP is used to post-process operational models such as the Global Forecast System (GFS), GFS Ensemble Forecast System (GEFS), North American Mesoscale (NAM), Rapid Refresh (RAP), High Resolution Rapid Refresh (HRRR), Short Range Ensemble Forecast (SREF), and Hurricane WRF (HWRF) applications.  The DTC serves as a bridge between operations and the community, and provides UPP software and support for the Weather Research and Forecasting (WRF) modeling core.  Since the UPP is used in operations; users can mimic the production of operational products through the community UPP distribution. Another advantage is its efficient handling of large datasets because it’s a parallelized code. 

One of the more popular features among community users is the ability leverage the Community Radiative Transfer Model (CRTM) to output synthetic satellite products.  Other favored features include vertical interpolations of certain products, such as radar reflectivity ¼ km above ground level (AGL), and the horizontal grid manipulation capability. In addition, users have recently leveraged UPP as a tool to post-process WRF simulations into GRIB output. Required fields can then be used as input to initialize another WRF simulation.

The DTC’s UPP team works directly with community developers to incorporate their contributions into the code base, and serves as a liaison to integrate new features into the operational code.  The UPP team also continues to expand and improve documentation to help the community use and contribute to the UPP software package.  Look for a new online tutorial coming later this year!

UPP v3.1 is the most recent version available, and was released in the Fall 2016.  The next release can be expected in Summer or Fall of 2017.  More information can be found on the UPP website: http://www.dtcenter.org/upp/users/.

HWRF Training at Home and Abroad

Spring 2016

The DTC hurricane team has provided training opportunities to learn the Hurricane Weather Research and Forecasting (HWRF) system to both general users and active developers over the past several months.

The community HWRF modeling system (version 3.7a released in August 2015) is compatible with the NCEP 2015 operational implementation, which includes high-resolution deterministic tropical cyclone numerical guidance for all global oceanic basins. Due to the demonstrated skill and advanced capabilities of the HWRF model, there is a great deal of international interest for research and operational use. In order to meet these demands and foster collaborations, an HWRF tutorial was held at the Nanjing University of Information Science and Technology (NUIST) in Nanjing, China. DTC hurricane team members Ligia Bernardet and Christina Holt participated along with members of the Environmental Modeling Center (EMC) HWRF team. The tutorial, held 1-2 December 2015, attracted 84 participants and received positive feedback.

“ In addition to the tutorials aimed at general users working with the publicly released code, the DTC also responded to developer requests for specialized training. ”

Following the China tutorial, the DTC co-hosted an HWRF tutorial with the EMC HWRF team in College Park, MD at the NOAA Center for Weather and Climate Predication. This tutorial spanned three days from 25-27 January 2016. Tutorial attendees heard over 12 hours of lectures covering all aspects of the HWRF system, as well as enrichment lectures on the HWRF multi-storm modeling system, the HWRF ensemble predication system, HYCOM ocean coupling, and forecast verification. Invited speakers participated from various institutions, including NCEP/EMC, University of Rhode Island (URI), AOML/HRD and DTC. In addition to lectures, students received 6 hours of hands-on practical sessions. The event was well received from participants, many who unexpectedly attended the tutorial remotely due to the 25+ inches of snow that fell over the DC area the weekend prior!

Presentations and materials for the College Park, MD and Nanjing, China tutorials are posted at: http://www.dtcenter.org/HurrWRF/users/tutorial/index.php.

In addition to the tutorials aimed at general users working with the publicly released code, the DTC also responded to developer requests for specialized training. To meet the needs of active developers working with the HWRF repository code, the DTC hosted two separate HWRF specific Python trainings; one in conjunction with the HFIP annual review meeting in Miami, FL, and a second joined to the HWRF tutorial in College Park, MD. Training materials and resources from the developer trainings are available at: http://www.dtcenter.org/HurrWRF/developers/docs/documents.php

Typhoon Symposium and HWRF Tutorial--Nanjing, China group photo .

HWRF Operational Implementation and Public Release

Winter 2016

With the conclusion of the 2015 hurricane season, assessments of model performance indicate that the upgraded 2015 Hurricane WRF (HWRF) model provided superior forecast guidance to the National Hurricane Center (NHC), with marked improvements over the previous HWRF system.

The unified HWRF system, for which the DTC provides the operational codes to the research community, is a cornerstone of HWRF’s success.

The community HWRF modeling system was upgraded to version 3.7a on August 31, 2015.  This release includes all components

of the HWRF system, including: scripts, data preprocessing, vortex initialization, data assimilation, atmospheric and ocean models, coupler, postprocessor, and vortex tracker (see Figure on the left).  Additionally, the capability to perform idealized tropical cyclone simulations is included (Figure in upper right). The HWRF community modeling system currently has over 1100 registered users.  The DTC provides resources for these users through updates to the user webpage, online practice exercises, datasets, and extensive documentation consistent with the latest release code.  With the HWRF v3.7a release, the HWRF helpdesk was migrated to a new tracking system (hwrf-help@ucar.edu), providing support for all aspects of the code.  Information about obtaining the codes, datasets, documentations, and tutorials can be found at the DTC HWRF user webpage: http://www.dtcenter.org/HurrWRF/users.

The HWRF v3.7a public release is compatible with the NCEP 2015 operational implementation of HWRF.  The HWRF model consists of a parent domain and two storm following two-way interactive nest domains.  Starting with the 2015 operational season, the default HWRF horizontal resolution increased to 18/6/2 km (from 27/9/3 km), and NCEP expanded high-resolution deterministic tropical cyclone forecast numerical guidance to all global oceanic basins for operations.  NCEP is running HWRF configurations with reduced complexity for global basins other than the Atlantic and Eastern North Pacific basins operationally.  However, the HWRF public release includes flexibility and alternate configuration options, such as running with full complexity including atmosphere-ocean coupled mode with data assimilation for all oceanic basins.  Additionally, the HWRF v3.7a maintains backwards compatibility to run the 27/9/3 km resolution.  One unsupported capability of the HWRF system is the use of an HWRF ensemble.

Improvements to the HWRF physics for the 2015 operational HWRF system demonstrate successful R2O transitions facilitated by the DTC.  The DTC acts as a conduit for code management and R2O by maintaining the integrity of the unified HWRF code and assisting developers with transitioning their innovations into the operational code.  Specifically, the DTC successfully facilitated R2O transitions for upgrades to radiation parameterization and PBL improvements that were implemented for the 2015 operational HWRF system.

Data Assimilation Study for TC Intensity

Summer 2015

The hybrid Ensemble Kalman Filter (EnKF)-Gridpoint Statistical Interpolation (GSI) data assimilation system was implemented at NCEP for its Global Forecasting System (GFS) in May 2012.

Schematic illustration of the hybrid EnKF-GSI data assimilation procedure. Dashed line indicates the optional re-centering step in the hybrid system.

This implementation led to significant improvements to global forecasts, including those of tropical storms. It can be noted that this improvement occurred while most current operational regional applications still use global rather than regional ensembles in their hybrid system. To bridge this gap, the DTC investigated the improvement of tropical storm intensity forecasts by using a regional ensemble in the GSI-hybrid data assimilation system.

A complete hybrid EnKF-GSI for the Hurricane WRF (HWRF) system was developed for the regional ensemble experiments, and results were compared to those obtained with the 2014 HWRF system. A two-way hybrid system was set up based on the GFS data assimilation scheme, using the GSI deterministic analysis to re-center the ensemble members at each analysis time. This re-centering step was found to reduce the ensemble spread for tropical cyclone (TC) center locations and intensity, so a one-way hybrid system that skipped the re-centering step was also developed.

Results showed that the operational system (Figure below, green) generated the lowest bias at the analysis time, but over time the bias showed a rapid “spin-down” from stronger to weaker wind forecasts than observed. (A similar spin-down issue was also noted using the 2015 HWRF system, but with smaller biases.)  The one-way hybrid system (red), which used a regional ensemble, performed better than the two-way hybrid system (blue), and also outperformed the 2014 operational configuration and the GSI hybrid system using GFS ensemble (without vortex initialization, cyan), for TC intensity forecasts beyond the 12-hour forecast lead time.

The DTC also performed experiments to further investigate the initial spin-down issue and found that it is related to an imbalance issue triggered by data assimilation. Experiments show that applying dynamic constraints could help ease such an imbalance. However, more research is required to find an optimal solution that reduces such imbalance-induced noise while still achieving desirable analysis increments.

Bias of (a) Maximum surface wind speed, and (b) Minimum sea level pressure for all the forecasts as a function of forecast lead time

Bridges to Operations

Autumn 2014

Researchers from the DTC plan to provide numerical model runs from a preliminary version of the North American Rapid Refresh Ensemble system (Pre-NARRE) to the Hydrometeorological Testbed of the Weather Prediction Center (HMT/ WPC) during their current Winter Exercise. The DTC Ensemble Task will run the ensemble system (most likely on the NOAA hjet computing system) and post-process some of the results for HMT/WPC. Members of the ensemble (eight in total) will be produced from both WRF/ RUC and NMMB dynamical cores, and will include different combinations of microphysical, planetary boundary layer, surface physics, convective parameterization, and initial and boundary condition options (as in the chart below). Although the WPC will evaluate the runs on the CONUS domain, the computational domain will be set to the larger existing RAP domain, at 13 km resolution out to 24-48h, depending on computing resources. One hopeful outcome of the experiment will be an opportunity to compare NARRE forecasts with parallel runs from the Environmental Modeling Center’s (EMC) operational regional ensemble forecast system (SREF), which will be provided by EMC. In addition, results from the experiment will be used to extend previous assessments of NARRE performance to wintertime regimes.

Contributed by Isidora Jankov and Ed Tollerud.

Version of the North American Rapid Refresh Ensemble system (Pre-NARRE) provided to the Hydrometeorological Testbed of the Weather Prediction Center (HMT/WPC)

Innovation in HWRF 2013 Baseline

Spring 2013

One of the regional numerical weather prediction models used operationally by the National Weather Service is the Hurricane WRF (HWRF), a coupled model with atmospheric and ocean components that exchange fluxes of short- and long-wave radiation, momentum, moisture, and heat. The momentum flux is particularly important because the strong winds in tropical cyclones cause turbulence and upwelling in the ocean, which can lead to transport of cold water from deep in the ocean towards the surface, reducing the storm’s energy source and causing it to weaken.


A comparison between the ocean cooling in HWRF against observational buoy data, performed by the Hurricane Research Division of NOAA’s Atlantic Oceanographic and Meteorological Laboratory, showed that the ocean surface cooling in HWRF is too small. The DTC worked with the NOAA Environmental Modeling Center and oceanographers from the University of Rhode Island to formulate a test in which the momentum flux in the ocean model was altered to be more physically consistent. The figure below shows the mean intensity error as a function of lead time for 2012. The black curve is the control and the red curve is the forecast with modified fluxes with 95% confidence intervals.


Results aggregated over all 2012 Atlantic storms showed the more physical flux reduced the 5-kt positive intensity bias of the operational model to near-zero. This change has been incorporated by EMC into the 2013 HWRF baseline, and is expected to be adopted operationally for the 2013 hurricane season.

Support for Operational DA at AFWA

Autumn 2013

Unlike some other forecast model components, a data assimilation (DA) system is usually built to be flexible in order to be run by different forecast systems at varying scales.

Its testing and evaluation must therefore be performed in the context of a specific application; in other words, it must be adaptable to different operational requirements as well as to research advances. Established in 2009, the DTC DA team started providing data assimilation support and testing and evaluation for Air Force Weather Agency (AFWA) mesoscale applications throughout its global theaters. This task has become one important component of the DTC’s effort to accelerate transitions from research to operations (R2O). Between 2009 and 2011, the focus of extensive DA testing for AFWA at the DTC was to provide a rational basis for the choice of the next generation DA system. Various analysis techniques and systems were selected by AFWA for testing, including WRF Data Assimilation (WRFDA), Gridpoint Statistical Interpolation (GSI), and the NCAR Ensemble Adjustment Kalman Filter. During this testing, the impacts of different data types, background error generation, and observation formats were also investigated.

“The developmental experiment outperformed the baseline”

Testing activity by the DTC DA team took a sharp turn in August 2012. To assist AFWA in setting up an appropriate configuration for their 2013 implementation of GSI, the DTC adapted their DA testbed to complement AFWA’s pre-implementation parallel tests in real-time. In support of providing new code and configurations, the team now performs two types of tests for AFWA:

The baseline experiment is usually generated by running the current operational or parallel system at AFWA. Whenever an AFWA baseline is updated, the DTC checks its reproducibility (or similarity) using the DTC functionally-similar testing environment to ensure that any following tests are comparable, and that there is no code divergence between research and operations. One such test conducted during the summer of 2013 (see figure next page) revealed that wind analysis fits to observations in AFWA forecasts were not reproduced by the DTC due to an inadvertent AFWA code change reading their own conventional data files. Other data assimilation components and applications (new configurations, techniques, observations, etc.) can also be tested in the DTC end-to-end DA testbed, see figure to the left.


During DTC real-time tests of the AFWA 2013 implementation, the AFWA GO index (a multivariate combined statistical score) dropped when the (then) AFWA parallel run configuration was used. When the GO index exceeded 1 (i.e., before November), the developmental experiment (which used the DTC-suggested configuration) outperformed the baseline (here, GFS-initialized). For wind variables in particular, the DTC configuration significantly improved the wind analyses. Further retrospective tests narrowed down the contributing factors, and the DTC suggested that the North American Mesoscale (NAM) static background errors generated by NCEP be used. AFWA adopted this configuration for its first GSI implementation in its global coverage domains in July 2013.