The DTC has been tasked by the US Air Force to investigate new approaches to evaluate cloud forecast predictions. Accurate cloud forecasts are critical to the Air Force national intelligence mission because clouds can mask key targets, obscure sensors, and are a hazard to Remotely Piloted Aircraft. This work that will help forecast users and developers understand their  characteristics of these predictions, and suggest ways to make the predictionsm more accurate.

Clouds have significant impacts on many kinds of decisions.  Among other applications, accurate cloud forecasts are critical to the national intelligence mission. Clouds can mask key targets, obscure sensors, and are a hazard to Remotely Piloted Aircraft. The locations of clouds, as well as other cloud characteristics (e.g., bases, tops), are difficult to predict because clouds are 3three-dimensional and they form and dissipate quickly at multiple levels in the atmosphere. In addition, cloud predictions integrate across multiple components of numerical weather prediction systems. Evaluation of cloud predictions is not straightforward for many of the same reasons.

The DTC has been tasked by the US Air Force to investigate new approaches to evaluate cloud forecast predictions that will help forecast users and developers understand their characteristics and suggest ways to make the predictions more accurate.

The DTC effort, in collaboration with staff at the Air Force’s 557th Weather Wing, focuses on testing a variety of verification approaches. , including tTraditional verification methods for continuous and categorical forecasts provide a baseline evaluation of quality (e.g.,  Mean Error, Mean Absolute Error, Gilbert Skill Score, Probability of Detection). that provide a baseline evaluation of quality, sSpatial methods (e.g., the Method for Object-based Diagnostic Evaluation [MODE]) and field deformation approaches that provide greater diagnostic information about cloud prediction capabilities., Nand new distance metrics that characterize the distances between forecast and observed cloud features. This evaluation will help identify new tools, including a cloud-centric NWP index, to consider for implementation and operational application in the Model Evaluation Tools (MET) verification software suite.

For the evaluation, the team is focusing initially on forecast and observed total cloud amount (TCA) fractional cloud coverage datasets for six cloud products for one week-periods for each of four seasons, for six cloud products:

• WorldWide Merged Cloud Analysis (WWMCA) developed by the Air Force;
• A WWMCA reanalysis (WWMCAR) product that includes latent observations not included in the real-time version of WWMCA;
• Forecasts (out to 72 h) of TCA from the USAF Global Air Land Weather Exploitation Model (GALWEM) which is the Air Force implementation of the United Kingdom’s Unified Model;
• TCA forecasts (out to 72 h) from the NCEP Global Forecast System (GFS) model;
• Biascorrected versions of the GALWEM and GFS predictions (GALWEM-DCF and GFS-DCF); and
• Shortterm TCA predictions (out to 9 h) from the Advective cloud model (ADVCLD).

Datasets used in the evaluation were from one week-periods for each of four seasons.

Methods and results

Results of the application of the various verification methods indicated that continuous approaches are not very meaningful for evaluating cloud predictions, particularly because due to they are discontinuous in nature of clouds.  In contrast, categorical approaches can provide information that is potentially quite useful, particularly when applied to thresholds that are relevant for AF decision-making (e.g., overcast, clear conditions), and when the results are presented using a multivariate approach such as the performance diagrams first applied by Roebber (WAF, 2009).  The MODE spatial method also shows great promise for diagnosing errors in cloud predictions (e.g., size biases, displacements).  However, more effort is required to identify optimal configurations of the MODE tool for application to clouds for AF decision making.

Initial testing of field deformation methods indicated that these approaches have a good are potentially of being useful for evaluation of cloud forecasts. Field deformation methods evaluate how much a forecast would have to change in order to best match the observed field. Information about the amount and type of deformation required can be estimated, along with the resulting reduction in error.

The results also indicated that, in general, cloud amount forecasts lend themselves to verification through binary image metrics because a cloud’s presence or absence can be ascertained through categories of cloud amount thresholds. These metrics can provide very succinct information about many aspects of forecast performance in this context without having to resort to complicated, computationally expensive techniques. For example, Baddley’s ∆ metric gives an overall useful summary of how well two cloud-amount products compare in terms of size, shape, orientation and location of clouds., and tThe Mean Error Distance (MED) gives meaningful information about misses and false alarms, but is sensitive to small changes in the field. In addition to the distance metrics, a geometric index that measures three geometric characteristics (area, connectivity, and shape ) could potentially provide additional useful information, especially when the cloud field is not too complex (i.e., is comprised of a small number of features).

Ongoing and future efforts

Ongoing efforts on this project are focused on extending the methods to global cloud amounts (the initial work focused on North America), and further refinements and tests of the methods.  For example, MODE configurations are being identified in collaboration with the AF 557th Weather Squadron, to ensure the configurations are relevant for AF decision-making.  In addition, canonical evaluations (i.e., with “artificial” but realistic cloud distributions) of the distance metrics [1] are being examined to determine if any unknown biases or poor behavior exist that would influence the application of these methods. As these extensions are completed, a set of tools will be identified that provide meaningful – and complete – information about performance of TCA forecasts.  Further efforts will focus on other cloud parameters such as cloud bases and tops.

The canonical evaluations only apply to the distance metrics, not all of the methods.

As the new Director of the Environmental Modeling Center (EMC), I have the pleasure of leading nearly two hundred world-class scientists, engineers and other staff in developing, transitioning, improving and maintaining a suite of global and regional environmental models to support the National Weather Service (NWS) mission of the protecting life and property for our country.  To meet the challenges of this mission, EMC has formed strategic partnerships with numerous community organizations over the years to collaboratively manage the existing suite of models, as well as to build the next generation of environmental models.  The DTC has been a vital partner for developmental testing, verification, and community support activities related to large segments of EMC’s modeling suite.  While the genesis of DTC began with WRF and so originally focused on regional models and related applications, the DTC has begun to expand their efforts into global modeling in support of the Next Generation Global Prediction System (NGGPS).  In particular, DTC has formed a Global Model Test Bed (GMTB) to support NGGPS development on a variety of tasks, most notably to assist in the development and testing of Common Community Physics Package and support for an associated interoperable physics driver. 

While the first step of the NGGPS program will be to migrate the legacy spectral model dynamic core of EMC’s Global Forecast System (GFS) to the Finite Volume Cubed Sphere version 3 (FV3) from NOAA’s Geophysical Fluid Dynamics Lab (GFDL), this represents much more than just a dynamic model core change to the global model. Instead it represents the first step toward migration of EMC’s modeling suite to a unified modeling system across both spatial (mesoscale/regional and global) and temporal (weather, sub-seasonal and seasonal) scales. With their legacy of mesoscale/regional applications and the new global modeling work under the GMTB umbrella, the DTC is ideally situated to play an integral role in helping EMC and NOAA evolve our legacy modeling suite towards a unified modeling system. 

The evolution towards a unified system is more than just a NOAA imperative; it is a National one. As such, I am happy to report that we recently brought the vast majority of NOAA organizations involved in model research, development, testing and operations together with several of our key national partners for a November meeting at the David Skaggs Research Center in Boulder, CO to start developing a short-term strategy for a National unified modeling system.  In addition to EMC (who organized and chaired the meeting), the other NOAA participants included 8 NOAA labs from the Office of Oceanic and Atmospheric Research (OAR) and the National Ocean Service (NOS); the NOS Center for Operational Oceanographic Products and Services (CO-OPS); the new NWS Office of Water Prediction (OWP); and 3 program offices from NWS and OAR.  Representatives of these NOAA organizations were joined by members of 3 Labs from the National Center for Atmospheric Research (NCAR), including the Research and Applications Lab (RAL) of which DTC is a part; the NASA Global Modeling and Assimilation Office (GMAO); the Naval Research Lab (NRL); and the Joint Center for Satellite Data Assimilation (JCSDA).  All these organizations came together with the intent to begin development of a Strategic Implementation Plan (SIP) that can orchestrate collaborative activities over the next 2-3 years under a single, coordinated “play book”. 

Following this successful meeting, the next steps will be the formation of working groups composed of experts from each of these organizations, other partner organizations, and members of the broader R&D community to tackle specific functional areas of the SIP, to include such areas as governance, system architecture, model dynamics, model physics, data assimilation, and post processing, to name a few.  The output of these working groups will be brought together in a public community workshop, targeted for Spring 2017, to pull together the first draft of a comprehensive, integrated plan.  The power of this approach will be to harness the collective resources of many of our country’s top R&D institutions along with other partners from the broader research community, who can all work together under a single, coordinated plan towards a common goal of building a truly National unified modeling system.  I and the other members of the Environmental Modeling Center look forward to working with the DTC and our other strategic partners as we work towards this common goal. 

As a visiting scientist at the University of Maryland in 2005, Man Zhang learned of Colorado’s reputation of being a great place to live. When her husband had an opportunity to come to Colorado as a visiting Ph.D. student, they didn’t think twice about moving. They love Colorado and have been in the Boulder area for almost 10 years.

Man was a post-doc at CIRA in the ensemble data assimilation group. She applied the hybrid variational-ensemble data assimilation system (HVEDAS) developed at CIRA to the NOAA operational Hurricane Weather Research and Forecasting (HWRF) modeling system to directly assimilate satellite observations in the tropical cyclone inner core. She also demonstrated the potential impacts of cloud-affected satellite radiances on the tropical cyclone inner-core region analyses and forecasts. “What I like most is learning new things,” Man says, “models are always evolving.”

Man joined NOAA/GSD/CIRES in July 2013 to support Nonhydrostatic Icosahedral Model (NIM) development. Since March 2016, Man has worked with the Global Model Test Bed (GMTB) to create software reference documentation on the Global Forecast System (GFS) physics suite (using Doxygen) to provide technical information to scientists and developers. She also works closely with physics developers at GMTB to modify the parameters that they want to experiment with in GFS-NOAA Earth Modeling System (NEMS). Recently, she has been running GFS-NEMS with the Grell-Freitas cumulus scheme. “It is interesting to think about the science behind tunable parameters,” she says.

Man is thankful her brother and his family, and her parents live close - all relocating to Colorado from the Hubei Province in Central China. Her 8 year old daughter and 3 year old son keep her busy with piano, dancing, and indoor climbing classes. They like road trips as a family and have explored most of national parks in the West and some in the East, with Utah’s Arches National Park and the Great Smoky Mountains National Park being their favorites. Over the holidays they enjoyed time up in Steamboat to ski and relax in the hot springs.

The community Hurricane Weather Research and Forecasting (HWRF) modeling system was upgraded to version 3.8a on November 21, 2016.  This release includes all components of the HWRF system: scripts, data preprocessing, vortex initialization, data assimilation, atmospheric and ocean models, coupler, postprocessor, and vortex tracker.  In addition to default operational features, the release includes capabilities to perform idealized tropical cyclone simulations run with alternate physics, and backwards compatibility for inner nest grid sizes.

The HWRF community modeling system currently has over 1300 registered users.  The public release includes updates to the user webpage, online practice exercises, datasets, and extensive documentation.  The release code is fully supported, with community support provided via the HWRF helpdesk, hwrf-help@ucar.edu.

Information about obtaining the codes, datasets, documentation and tutorials can be found at http://www.dtcenter.org/HurrWRF/users

The NCEP 2016 operational implementation of HWRF and the HWRF v3.8a community release are compatible systems.  Starting in 2016, the default configuration runs with ocean coupling for all northern hemisphere oceanic basins, and uses Real-Time Ocean Forecast System (RTOFS) data for ocean initialization in the Eastern North Pacific Basin.  Two specific capabilities, a 40-member HWRF ensemble for the assimilation of Tail Doppler Radar (TDR) data that NCEP began running in 2015, and the addition of one-way wave coupling using WAVEWATCH III in 2016, are not currently supported to the general community.

• Other notable upgrades in HWRF version 3.8a include:
• Code upgrades including, WRF v3.8, GSI v3.5, and UPP v3.1.
• Inner domain (d02, d03) sizes increased to 25ºx25º and 8.3ºx8.3º, respectively.
• Reduced time step from 38 4/7 s to 30 s.
• Data assimilation enabled by default for both Atlantic and Eastern North Pacific Basins.
• Improved physics for all scales:
  • Cumulus parameterization updates, including enabling by default for all 3 domains and a new Scale Aware Simplified Arakawa Shubert (SAS) scheme.
  • New GFS Hybrid-Eddy Diffusivity Mass Flux PBL scheme.
  • Updated momentum and enthalpy exchange coefficients (Cd/Ch).
  • Enhanced Idealized capability with landfall.
• Enhanced products including simulated brightness temperatures for new satellite sensors in all basins.

DTC visitor contributes enhanced idealized capability

As noted in the HWRFv3.8a updates, an enhanced idealized capability to include simulated landfall using the GFDL slab land surface physics scheme is included in the v3.8a release. This capability was contributed through a successful DTC visitor project by Subashini Subramanian (Purdue University), “Developing Landfall Capability in Idealized HWRF for Assessing the Impact of Land Surface on Tropical Cyclone Evolution”.  The new feature introduces a namelist switch for allowing the landfalling capability, which specifies the type of land surface and an initial land-surface temperature to be used over land.  The default configuration introduces a homogeneous land surface that can be modified to account for heterogeneity. Additionally, the direction of land motion is a user-defined option. Work is underway to extend this capability to include other land-surface physics options.

As visitors to the DTC in 2016, Michael Iacono and John Henderson of Atmospheric and Environmental Research (AER) used the Hurricane Weather Research and Forecasting model (HWRF) to investigate an improved way of representing the vertical overlap of partial cloudiness and showed that this process strongly influences the transfer of radiation in the atmosphere and can impact the evolution of simulated tropical cyclones.

Clouds are a critical element of Earth’s climate because they strongly affect both the incoming solar (or shortwave) energy, which fuels the climate system, and the thermal (or longwave) energy passing through the atmosphere. Understanding the way that clouds absorb, emit, and scatter radiation is essential to modeling their role effectively.

One limitation to simulating clouds accurately is the challenge of representing their variability on scales smaller than the typical grid spacing of a dynamical model such as HWRF. Individual cloud elements are often sub-grid in size, and radiative transfer through fractional clouds strongly depends on whether they are vertically correlated such as for deep, convective clouds, or uncorrelated such as for randomly situated shallow cumulus under high clouds.

Using the Rapid Radiative Transfer Model for Global Climate Models (RRTMG) radiation code in HWRF, the primary objective of this project is to examine the effect of replacing the default maximum-random (MR) cloud overlap assumption with an exponential-random (ER) method, which has been shown to be more realistic relative to radar measurements within vertically deep clouds. The MR approach forces a condition of maximal overlap throughout adjacent partly cloudy layers, while the ER method relaxes this restriction by allowing the correlation to transition exponentially from maximum to random with vertical distance through the cloudy layers.

A first step in assessing this change in HWRF is to show that it alters radiative heating rates enough to affect the development of a tropical cyclone (TC), since heating rates, along with surface fluxes, are the primary means by which a radiation code influences a dynamical model. For Hurricane Joaquin, a 2015 Atlantic basin storm with an unusual track that was challenging to predict, each overlap method causes longwave and shortwave heating rates to evolve very differently within and near the storm over multiple five-day HWRF forecast cycles. Over time, these changes modify the temperature, moisture and wind fields that exert a considerable influence on the predicted strength and movement of Joaquin.

The full impact on TC track and intensity remains under investigation, since the cases studied to date respond very differently. Hurricane Joaquin track forecasts are dramatically altered in some forecast cycles, while more modest track changes are seen for storms embedded in strong steering flows such as East Pacific Hurricane Dolores from 2015 and Atlantic Hurricane Gonzalo from 2014. Intensity impacts are also case-dependent with improvement seen in some Joaquin forecast cycles and degraded intensity forecasts in other cases.

Our interaction with the DTC was a rewarding opportunity to acquire new insights on this topic, and we will pursue further research collaborations with the DTC and the NOAA/EMC Hurricane Team in the future.

The DTC hosted its annual DTC Science Advisory Board (SAB) meeting on September 14-15, 2016.  This annual event provides an opportunity for the DTC to present a review of DTC key accomplishments over the past three years to representatives of the research community and to collect input directed at shaping a strategy for the future.

 “The DTC enables and supports a wide variety of research-to-operations and operations-to-research activities,” said SAB Chair Russ Schumacher. “The SAB discussed ways that these activities can be further strengthened, including through model evaluation and maintaining operational model codes.  A topic of particular discussion was NOAA and NCEP’s goal to move toward a unified modeling approach.  This brings with it some great opportunities to connect the research and operational communities to enhance weather prediction, but will also pose challenges in familiarizing researchers with new modeling systems.”

The agenda included an Operational Partners Session, an opportunity for the new Environmental Modeling Center (EMC) Director Dr. Mike Farrar to present a vision for NOAA’s unified modeling, followed by the Air Force outlook for its modeling suite by Dr. John Zapotocny. DTC task area presentations reviewed key accomplishments with a focus on research to operations, and presented thoughts for possible research to operations activities, risks, and challenges within the coming three years. Break-out task area group discussions were productive, followed by the SAB recommendations session. The recommendations are detailed in the DTC SAB meeting summary, but the following are a few highlights.

With NCEP’s transition to a unified modeling system centered around the Finite Volume Cubed Sphere (FV3) dynamic core, the SAB voiced their belief that the community will need the DTC’s leadership in supporting FV3 and the NOAA Environmental Modeling System (NEMS) in future years, and the DTC should build up internal expertise in advance of these transitions. They noted no other organization in the U.S. has a core responsibility to be an unbiased evaluator of Numerical Weather Prediction (NWP) models, and advised the DTC to not lose sight of their unique function. As the NGGPS paradigm emerges, the SAB encourages the DTC leadership to keep close attention on where future DTC funding might be anticipated, and make sure future SAB members have expertise in those funding areas.

The SAB recommended the DTC build an interactive research community to hear from active users with fresh ideas and provide a conduit between research now and possible operations in the future. It would be a good forum for interacting with active scientists in other areas of modeling and would be a promising mechanism for getting users involved in the DTC Visitor Program. They also encouraged further engagement with the global ensemble, convection-allowing ensemble, data assimilation, and verification communities.

To support DTC-supported code and address the DTC staffing problem, the SAB suggested a graduate student model. This could build and enhance capabilities of upcoming researchers relevant to the needs of operational models. It would also benefit the university community by supporting students with a gap in funding -- a win-win situation for both the DTC and the university.

The SAB recommended the DTC to continue their work to put supported codes into Docker containers, an open-source project that automates the deployment of applications inside software containers for community use. This significantly reduces challenges associated with setting up and running code on different platforms, and building the libraries the codes use. They also suggested clarifying the roles of model and code developers, the DTC, and the user community in accessing, supporting, and adding new innovations to reduce possible confusion and redundancy.

Because all DTC task areas require robust verification tools to achieve their objectives, the SAB recommended strengthening and supporting collaborations that already exist between the verification task and the other task areas. This may involve increasing the flexibility of Model Evaluation Tools (MET) to support the needs of both DTC tasks and communities. The SAB also indicated there needs to be clear pathways for those who develop new verification tools or methods to have those tools incorporated into MET. They also encouraged the DTC to have more year-to-year continuity in testing and evaluation activities to increase productivity and yield more fruitful outcomes, and to continue to thoughtfully balance these activities on a task-by-task basis.

The external community Science Advisory Board acts as a sounding board to assist the DTC Director, and provides advice on emerging NWP technologies, strategic computer resource issues, selection of code for testing and evaluation, and selection of candidates for the visiting scientist program. Members are nominated by the Management Board, and the Executive Committee provides final approval of SAB nominations for a 3-year term. Current members of the DTC Science Advisory Board can be found at http://dtcenter.org under governance.

You can join weekly webinars to discuss the performance of EMC’s forecast/analysis systems from a synoptic and mesoscale perspective.

• The weekly webinar is led by EMC’s Model Evaluation Group and participants include model developers, NCEP service centers, NWS regional and field offices, DTC staff, academic community and private sector.
• Webinars serve as a forum for EMC to reach out to forecast and user community

Interested in participating?  Contact Glenn White or Geoff Manikin

Here's a conference abstract of Glenn's on this group too
https://ams.confex.com/ams/97Annual/webprogram/Paper313180.html