2018 DTC Community Unified Forecast System Test Plans and Metrics Workshop

Summer 2018

The 2018 DTC Community Unified Forecast System Test Plan and Metrics Workshop was held at NOAA’s National Center for Weather and Climate Prediction (NCEP) on July 30 - August 1, 2018. The major goal of this workshop was to work towards a community test plan with common validation and verification metrics for the emerging Unified Forecast System (UFS). The plan will serve as a guide for the Numerical Weather Prediction (NWP) community for testing and evaluating new developments for the UFS models and components by comparison of both historical and real-time forecasts using observations and analyses, through standardized hierarchical testing.

The workshop organization was led by the Developmental Testbed Center (DTC), and the organizing committee was representative of various aspects of the NWP verification and validation (V&V) enterprise, including voices from those working on research, development, transitions, and operations. The membership of the organizing committee was:

  • Curtis Alexander - NOAA/Earth Systems Research Laboratory Global Systems Division (GSD)

  • Ligia Bernardet - CU/Cooperative Institute for Research in Environmental Studies (CIRES) at NOAA/GSD and DTC

  • Tara Jensen - National Center for Atmospheric Research (NCAR) and DTC

  • Jim Kinter - George Mason University/Center for Ocean-Land-Atmosphere Studies (COLA)

  • Sherrie Morris (NOAA Office of Science and Technology Integration (OSTI)

  • Jason Levit (NOAA/NCEP/Environmental Modeling Center (EMC)

  • Ryan Torn (State University of New York at Albany)

  • Ivanka Stajner (NOAA OSTI on detail to NOAA/NCEP/EMC)

The workshop was attended by approximately 100 participants, cross-cutting through various sectors of the V&V community, including international (Taiwan Central Weather Bureau and European Center for Medium-range Weather Forecasting (ECMWF), universities, NASA, NOAA National Environmental Satellite, Data, and Information Service (NESDIS), research laboratories, Office of the Federal Coordinator for Meteorological Services and Supporting Research, National Weather Service, testbeds, Navy, U.S. Air Force, and the private sector.

The workshop had a mix of presentations, discussion periods, and working sessions in which participants contributed to the three topic-based breakout sessions: test plans, metrics, and hierarchical testing.  Metrics for all spatial and temporal scales for numerical weather prediction models and emerging topics such as the verification of convective allowing models, coupled earth system models, and ensemble systems were discussed.

The last activity in the workshop was a summary of the working sessions’ discussions by their leads, which was presented to workshop participants and members of the UFS Strategic Implementation Plan (SIP) meeting. A report on the progress made during the workshop will be available in the next few months.

A link to the workshop page is here: 2018 DTC Community Unified Forecast SystemTest Plans and Metrics Workshop

 

2018 DTC Community Unified Forecast System Test Plans and Metrics Workshop Attendees
2018 DTC Community Unified Forecast System Test Plans and Metrics Workshop Attendees

 


DTC staff host AMS Short Course on Containers

Spring 2018

A major hurdle for running new software systems is often building and compiling the necessary code on a particular computer platform. In recent years, the concept of using “containers” has been gaining momentum in the numerical weather prediction (NWP) community. This new container technology allows for the complete software system to be bundled (operating system, libraries, code, etc.) and shipped to users in order to reduce the spin-up time, leading to a more efficient setup process. A core mission of the DTC is to assist the research community in efficiently demonstrating the merits of new model innovations.  The development and support of end-to-end NWP containers is in direct support of that mission.

In recent years, a number of NWP software components (including pre-processing, the model itself, post-processing, graphics generation, and statistics computation and visualization) were implemented into Docker containers to better assist the user community. The work conducted by DTC staff leveraged previous efforts of the Big Weather Web (http://bigweatherweb.org), which initially established software containers for the WRF Pre-Processing System (WPS), Weather Research and Forecasting (WRF) model, and NCAR Command Language (NCL) components. From there, DTC staff expanded the containerized tools to include the Unified Post-Processor (UPP), Model Evaluation Tools (MET), and METViewer. Through this complementary work, a full end-to-end NWP system was established, allowing for verification of the model output and visualization of the statistical output.

Several DTC staff (Kate Fossell, John Halley Gotway, and Tara Jensen, and Jamie Wolff) hosted a short course at the 98th Annual AMS meeting in Austin, TX on 6 January 2018 that offered hands-on experience with the established software containers. In preparation for the short course, an online tutorial was created that can be accessed at: https://dtcenter.org/met/docker-nwp/tutorial/container_nwp_tutorial/index.php. If you are an undergraduate/graduate student, university faculty, or researcher who is interested in these new tools, please check it out! Participants of the 2018 short course were complementary of the day-long tutorial, and the DTC plans to offer it again next year at the Annual meeting in Phoenix, AZ – stay tuned for more information regarding future training opportunities.


DTC staff and trainers

WRF Users' Workshop - June 2017

Autumn 2017

The first Weather Research and Forecasting model (WRF) Users’ Workshop was held in 2000. Since then, eighteen annual workshops have been organized and hosted by the National Center for Atmospheric Research in Boulder, Colorado to provide a platform where developers and users can share new developments, test results, and feedback. This exchange ensures the WRF model continues to progress and remain relevant.

The workshop program has evolved through the years. In 2006, instructional sessions were introduced, with the first focused on the newly developed WRF Pre-processing System (WPS). The number of users has grown since the WRF Version 3 release in 2008, so a lecture series on the fundamentals of physics was introduced in 2010 to train users to better understand and apply the model. Since that time, the series has covered microphysics, planetary boundary layer (PBL) and land surface physics, convection and atmospheric radiation. The series then expanded to address dynamics, modeling system best practices, and computing.

The 18th WRF Users’ Workshop was held June 12 – 16, 2017. The workshop was attended by 180 users from 20 countries, including 57 first time attendees, and 130 papers were presented. The first afternoon of the workshop, four lectures covered the basics of ensemble forecasting, model error, verification and virtualization of ensemble forecast products. The following days included nine sessions on a wide range of WRF model development and applications. On Friday, five mini-tutorials were offered on WRF-Hydro, Model for Prediction Across Scales (MPAS) for WRF Users, Visualization and Analysis Platform for Ocean, Atmosphere, and Solar Researchers (VAPOR), NCAR Common Language (NCL) and WRF-Python. All workshop presentations are available from http://www.mmm.ucar.edu/wrf/users/workshops/WS2017/WorkshopPapers.php.

The WRF Modeling System Development session included the annual update, plus status reports on WRF Data Assimilation (WRFDA), WRF-Chem, WRF software, Gridpoint Statistical Interpolation (GSI), Hurricane WRF (HWRF) and WRF-Hydro. A hybrid vertical coordinate was introduced in Version 3.9 for the Advanced Research WRF (ARW) that may potentially improve prediction in the upper-air jet streak region. Another notable addition to the model is the predicted particle properties or P3 scheme, a new type of microphysics.

Both data assimilation and model physics were improved in the operational application of WRF in the Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR) models. Advances were made in the Grell-Freitas cumulus scheme, Mellor-Yamada-Nakanishi-Niino (MYNN) PBL scheme and the Rapid Update Cycle (RUC) Land Surface Model (LSM). The HWRF operational upgrade included a scale-aware Simplified Arakawa Schubert (SAS) cumulus scheme, new Ferrier-Aligo microphysics schemes, and improved data assimilation. Other development and applications of WRF were also presented. Notably, the large-eddy simulation (LES) capability has been extended to many real-data applications in recent years.

There were two discussions during the workshop devoted to physics suites and model unification. Two suites of pre-selected physics combinations are now available in V3.9 that are verified to work well together for weather prediction applications. The second discussion was about model unification between WRF and the newer MPAS. While the two models remain independent, both are supported by the community and aspects of their development effort can be shared.

The next WRF Users’ Workshop will in June 2018.


Science Advisory Board Meeting - Sept 14-15

Winter 2017

The DTC hosted its annual DTC Science Advisory Board (SAB) meeting on September 14-15, 2016.  This annual event provides an opportunity for the DTC to present a review of DTC key accomplishments over the past three years to representatives of the research community and to collect input directed at shaping a strategy for the future.

 “The DTC enables and supports a wide variety of research-to-operations and operations-to-research activities,” said SAB Chair Russ Schumacher. “The SAB discussed ways that these activities can be further strengthened, including through model evaluation and maintaining operational model codes.  A topic of particular discussion was NOAA and NCEP’s goal to move toward a unified modeling approach.  This brings with it some great opportunities to connect the research and operational communities to enhance weather prediction, but will also pose challenges in familiarizing researchers with new modeling systems.”

The agenda included an Operational Partners Session, an opportunity for the new Environmental Modeling Center (EMC) Director Dr. Mike Farrar to present a vision for NOAA’s unified modeling, followed by the Air Force outlook for its modeling suite by Dr. John Zapotocny. DTC task area presentations reviewed key accomplishments with a focus on research to operations, and presented thoughts for possible research to operations activities, risks, and challenges within the coming three years. Break-out task area group discussions were productive, followed by the SAB recommendations session. The recommendations are detailed in the DTC SAB meeting summary, but the following are a few highlights.

With NCEP’s transition to a unified modeling system centered around the Finite Volume Cubed Sphere (FV3) dynamic core, the SAB voiced their belief that the community will need the DTC’s leadership in supporting FV3 and the NOAA Environmental Modeling System (NEMS) in future years, and the DTC should build up internal expertise in advance of these transitions. They noted no other organization in the U.S. has a core responsibility to be an unbiased evaluator of Numerical Weather Prediction (NWP) models, and advised the DTC to not lose sight of their unique function. As the NGGPS paradigm emerges, the SAB encourages the DTC leadership to keep close attention on where future DTC funding might be anticipated, and make sure future SAB members have expertise in those funding areas.

The SAB recommended the DTC build an interactive research community to hear from active users with fresh ideas and provide a conduit between research now and possible operations in the future. It would be a good forum for interacting with active scientists in other areas of modeling and would be a promising mechanism for getting users involved in the DTC Visitor Program. They also encouraged further engagement with the global ensemble, convection-allowing ensemble, data assimilation, and verification communities.

To support DTC-supported code and address the DTC staffing problem, the SAB suggested a graduate student model. This could build and enhance capabilities of upcoming researchers relevant to the needs of operational models. It would also benefit the university community by supporting students with a gap in funding -- a win-win situation for both the DTC and the university.

The SAB recommended the DTC to continue their work to put supported codes into Docker containers, an open-source project that automates the deployment of applications inside software containers for community use. This significantly reduces challenges associated with setting up and running code on different platforms, and building the libraries the codes use. They also suggested clarifying the roles of model and code developers, the DTC, and the user community in accessing, supporting, and adding new innovations to reduce possible confusion and redundancy.

 

Because all DTC task areas require robust verification tools to achieve their objectives, the SAB recommended strengthening and supporting collaborations that already exist between the verification task and the other task areas. This may involve increasing the flexibility of Model Evaluation Tools (MET) to support the needs of both DTC tasks and communities. The SAB also indicated there needs to be clear pathways for those who develop new verification tools or methods to have those tools incorporated into MET. They also encouraged the DTC to have more year-to-year continuity in testing and evaluation activities to increase productivity and yield more fruitful outcomes, and to continue to thoughtfully balance these activities on a task-by-task basis.

 

The external community Science Advisory Board acts as a sounding board to assist the DTC Director, and provides advice on emerging NWP technologies, strategic computer resource issues, selection of code for testing and evaluation, and selection of candidates for the visiting scientist program. Members are nominated by the Management Board, and the Executive Committee provides final approval of SAB nominations for a 3-year term. Current members of the DTC Science Advisory Board can be found at http://dtcenter.org under governance.

 

 

NGGPS Atmospheric Physics Workshop

Summer 2017

On 8-9 November 2016, more than 80 scientists from a broad cross-section of the physics development community gathered for the Next Generation Global Prediction System (NGGPS) Atmospheric Physics Workshop at the NOAA Center for Weather and Climate Prediction. The workshop provided an opportunity for the NGGPS Physics Team to revisit and refine its near- and long-term priorities for advancing the National Centers for Environmental Prediction (NCEP) global physics suite and identify key areas that need attention. Current plans are to deliver the advanced physics suite by October 2018. 

Workshop participants proposed the following approaches to advance NCEP’s global physics suite:

  • Upgrade the radiation code to RRTMGP, a restructured and modern version of RRTMG (Rapid Radiative Transfer Model for Global Climate Models), to allow more interactions between advanced schemes, such as different radiative processes for separate distributions of cloud ice and snow, and to significantly increase the speed of this computationally costly component of the physics.
  • Determine an approach for selecting a microphysics scheme from a list of strong candidates that meets NGGPS priorities/goals and proceed with an open selection process.  This approach should include both testing metrics and an assessment of how the scheme would set up the physics suite for future advancement.  General consensus of the breakout group discussions was that the Thompson scheme is a strong candidate for NGGPS testing.
  • Convection and boundary-layer schemes would follow the idea of having an evolved suite and an advanced suite that can be compared with suitable metrics. It is recognized that decisions regarding these physics components need to be taken in the context of a suite with other candidate components.
  • The land-surface model will include updated data, and will evolve based on process studies for individual components. The final model will likely come from selected Noah-MP options.


The workshop discussions laid the groundwork for a collaborative framework that will allow the research and operational communities to efficiently and effectively accelerate the advancement of NCEP’s global physics suite.  At the center of this framework is the Interoperable Physics Driver (IPD)/Common Community Physics Package (CCPP) concept and a clearly defined and documented systematic process to select and advance innovations that define the composition of future operational physics suites.

While the workshop discussions made great strides towards defining the key aspects of this collaborative framework, many of the details of the decision-making process still remain to be defined and vetted.  As these final details are worked out, it will be imperative to make sure the process is open and transparent, and takes into account the needs of both the research and operational communities.

Sea Ice Modeling Workshop

Spring 2016

A sea ice modeling workshop was convened on 3-4 February 2016 in Boulder, CO for the purpose of informing NOAA on the inclusion and selection of a community-contributed sea ice model into the future Next-Generation Global Prediction System (NGGPS). Another workshop goal was to identify potential research and development opportunities and gaps.

The workshop was hosted at the National Center for Atmospheric Research (NCAR) by the DTC’s Global Model Test Bed, and counted as sponsors the National Weather Service, the NOAA Office of Atmospheric Research Climate Program Office, and the Office of Naval Research. Sixty-five scientists attended the workshop, representing a broad spectrum of research and operational organizations, such as universities, NOAA, Navy, Department of Energy (DoE), the US National Ice Center, National Snow and Ice Data Center, NCAR, NASA, and DTC. International participation included representatives from the UK Meteorological Office, U. Reading, U. Toronto, and Environment Canada.

NOAA’s NGGPS will be a single fully coupled Earth modeling system with application to forecasts from days to seasons, spanning spatial scales from 1 to 25 km. While the sea ice model to be selected for inclusion in NGGPS needs to have good performance for all forecast applications over these time and space scales, this workshop focused on the short- and medium-term sea ice forecast needs. Workshop participants reviewed several state-of-the-art sea ice modeling efforts, along with various Earth modeling systems, such as the NCEP Keeping Ice’S Simplicity (KISS) model, the DoE Community Ice CodE (CICE) model, the NOAA Geophysical Fluid Dynamics Laboratory Sea Ice Simulator (SIS), the NOAA National Centers for Environmental Prediction Climate Forecast System (CFS), the Canadian Regional Ice-Ocean Prediction System (RIOPS), the ESRL Regional Arctic System Model (RASM) and the Navy Research Laboratory Arctic Cap Nowcast/Forecast System (ACNFS).

It was noted that the majority of advanced sea ice models have similar physical parameterizations, and that the differences among advanced models is smaller than the uncertainty due to initial conditions and external forcing. Given that the use of a community-contributed and supported model in NGGPS was raised as a priority for model selection, participants recommended the tentative adoption of the CICE model, pending follow-up testing and addressing concerns raised regarding the CICE model governance and potential for introduction of numerical artifacts due to differences in staggering between the grids used in the ocean models and in CICE.

The framework for follow-up testing was discussed and important points regarding resolution, domain, coupling, verification metrics, and observations were raised. The importance of the ocean model for ice model performance was also discussed, and will need to be a factor in the test design. These issues will be considered by a tiger team formed to design and conduct the tests. A variety of synergistic project efforts in the scientific community were identified and recommendations for future model development were put forth. Test results will be used to prioritize future model development efforts.

For more information on the NGGPS sea ice modeling workshop and its presentations, please visit http://www.dtcenter.org/events/workshops16/seaice


Sea Ice Workshop group photo

New HWRF Developers Website: R2O for Hurricane Model Development

Winter 2015

The mission of the DTC is to accelerate the rate of transition of new research and development to operational numerical weather prediction models. To that end, the DTC makes NCEP operational models, such as the Hurricane WRF, available to the general community through yearly releases of stable, well-tested, and well-documented codes, which are supported through a help desk.

While the DTC has hundreds of registered HWRF users, only a small subset of them actually contribute innovations, raising questions about the return on the DTC’s investment.



To address this concern, an additional type of support, targeted to this select group of active developers, has been launched by the DTC with support from the Hurricane Forecast Improvement Project (HFIP). Through the HWRF developers’ website (http://www.dtcenter.org/HurrWRF/developers), scientists external to EMC can request access to the HWRF code repositories, giving them access to retrieve and contribute to experimental codes. They can also obtain information about the HWRF code management process and the steps to get their code made available for consideration by EMC. Finally, scientists can get training on advanced HWRF aspects not made available to the general community, such as the HWRF build system and HWRF automation with the Rocoto Workflow Manager System.

This new DTC service, which goes well beyond what is provided to the general community through public releases, has been extensively used by many HWRF developers, and has been particularly helpful to the principal investigators funded by HFIP.

An HWRF Tutorial in Taiwan

Summer 2014
Tim Brown-DTC, Qingfu Liu-EMC, Yong Kwon-formerly of EMC, Ligia Bernardet-DTC, Vijay Tallapragada-EMC, and Sam Trahan-EMC some of the HWRF instructors, May 2014, Taipei, Taiwan.

The Hurricane Weather Research and Forecasting model (HWRF) is a U.S. operational hurricane prediction model used by the National Hurricane Center for tropical cyclone track and intensity forecasts in its basins of responsibility: North Atlantic and Eastern North Pacific. However, HWRF can be employed in any basin. In 2013 the HWRF real-time runs conducted by the NOAA Environmental Modeling Center (EMC) for the West Pacific basin were found to be very valuable by the Joint Typhoon Warning Center (JTWC). Because of the demonstrated skill of the HWRF model and its advanced capabilities, there has been a strong interest in HWRF from the research community as well as the international weather centers that are responsible for tropical cyclones forecasting. Currently, there are more than 1000 registered users for HWRF. With a goal of encouraging the participation of international research and operational community in the development and applications of HWRF, the NOAA Hurricane Forecast Improvement Project (HFIP) sponsored an HWRF tutorial in Taipei, Taiwan, 22-23 May 2014. The HWRF Tutorial was held immediately following the Workshop on Numerical Prediction of Tropical Cyclones, 20-21 May 2014, which was attended by about 60 scientists from Taiwan, U.S., China, Japan, S. Korea, India, Vietnam, Thailand, the Philippines, and Malaysia. Fred Toepfer, HFIP Program Director, gave a keynote speech at the workshop. The HWRF Tutorial was organized jointly by DTC, EMC, HFIP, Taiwan’s Central Weather Bureau (CWB), and the Taiwan Typhoon Flood Research Inst (TTFRI). Twenty-six students from Malaysia, the UK, Thailand, Vietnam, USA, Singapore, and Taiwan participated. The tutorial instructors included Robert Gall of HFIP, Vijay Tallapragada, Young Kwon, Sam Trahan, Qingfu Liu, and Chanh Kieu of EMC, and Timothy Brown and Ligia Bernardet of DTC. The feedback from the students was overwhelmingly positive, in spite of the torrential rain of 14 inches in 24 hours which fell in Taipei during the event! We anticipate an increased use of HWRF in the West Pacific typhoon community in the years to come, which will lead to valuable collaboration on the continued development on HWRF.


Students in the classroom

Vijay Tallapragada lecturing

Community Software Maintenance and Support

Winter 2014

One function of the DTC has been to archive and maintain important model-related code, and to make it available to operational and research segments of the meteorological community. As Laurie Carson describes it, the code maintenance and support function has important objectives in both O2R and R2O arenas: for the former, providing operational software to the research community, and for the latter, facilitating transfer of research capabilities to operational software packages. DTC’s approach is based on a philosophy that community software is a resource shared with a broad community of (distributed) developers specifically including the capabilities of operational systems. Two keys to its success are periodic public releases that include new capabilities and techniques, and effective user support. The chart summarizes present and planned DTC software support activities in five principal areas: WRF model updates and support, data assimilation (GSI) code releases and support, the end-to-end operational hurricane forecast system (HWRF), verification package maintenance and support (MET), and planning for a future community package of the NOAA Environmental Modeling System (NEMS) that includes the NMMB model. Some community code now supported in this way has derived from DTC visitor projects; an example is the field alignment technique described in the 2012 visitor project of Sai Ravela (summary available at http://www.dtcenter. org/visitors/year_archive/2012/). For further description of the DTC community software efforts, see http://www. dtcenter.org/code/.

 



“The DTC software maintenance task has both O2R and R2O objectives.”

As the chart indicates, another community outreach-related DTC activity involves arranging and contributing to workshops and tutorials to facilitate use of these community model and analysis packages. A future issue of Transitions will summarize recent and upcoming events of this kind.

The 2014 GSI Community Tutorial

Autumn 2014

The DTC hosted the 5th Community Gridpoint Statistical Interpolation (GSI) Tutorial on July 14-16 of this summer at the NCAR Foothills Laboratory in Boulder, Colorado. One of several outreach events sponsored recently by the DTC, this tutorial was held in collaboration with other major GSI development teams from around the United States. With an ultimate goal of providing operational capabilities to the research community, this series of tutorials has become a primary training resource whereby both operational and research users can gain knowledge essential to running and further developing GSI.

The tutorial this year was a three-day venture that included both invited lectures and practical hands-on sessions relevant to GSI. Within the program were lectures designed to cover both fundamental (e.g., compilation, execution, and di agnos t i c s ) and advanced (pre-processing, radiance and radar data assimilation, hybrid techniques, and GSI infrastructure) topics.

Lecturers and practical session instructors were invited from major GSI development/support teams, including NCEP/EMC, NASA/GMAO, NOAA/ESRL, and NCAR/MMM, along with DTC members from NOAA/ESRL and NCAR/ RAL. The principal guest speaker from the university community this year was Dr. Milija Zupanski from Colorado State University. Attended by 41 students from the U. S. and other international agencies and universities, the tutorial easily reached maximum capacity.

Tutorial presentations and lectures are posted at http:// www.dtcenter.org/com-GSI/users/docs/index.php. For more information about the GSI system itself and its community support, please visit: http://www.dtcenter. org/com-GSI/users/index.php.



On July 17 after the Community Gridpoint Statistical Interpolation (GSI) Community Tutorial (summarized on page 4), the GSI Review Committee also met at the NCAR Foothills Laboratory in Boulder. Established in 2010, this committee continues to coordinate GSI development from both operational and research communities, and is also responsible for reviewing and approving GSI code updates. During general review of ongoing GSI development efforts and future plans for GSI, the committee specifically discussed potential community support of the NOAA Ensemble Kalman Filter (EnKF) system, which is currently a part of the GSI-based hybrid ensemble variational system of the NOAA Global Forecast System (GFS), and a potential candidate for other operational applications. The decision was then made to establish code management for this EnKF system that follows the existing GSI code management protocol. As a consequence, the GSI review committee effectively becomes a joint review committee for both GSI and EnKF, and new membership (NOAA/ESRL, and the University of Maryland as a deputy member) was approved to represent the EnKF development effort. This new DA review committee thus includes members from NCEP/ EMC, NOAA/ESRL, NASA/GMAO, NESDIS, AFWA, NCAR, the University of Maryland, and the DTC.


Object-based Verification at WPC

Summer 2013

The Weather Prediction Center (WPC) at NCEP has been using MODE to supplement its traditional verification techniques since April 2010. The Method for Object-based Diagnostic Evaluation (MODE), a utility that is part of the MET verification package, has been developed with substantial support from the DTC. Both are systematically expanded and maintained for specific DTC tasks and an array of outside users. MODE output is available to WPC forecasters in real time through an internal website that displays graphical verification for forecasts of 24 hr precipitation valid at 1200 UTC (see the figure). Forecasters can select the forecast lead time (Day 1 – 36 hr, Day 2 – 60 hr, or Day 3 – 84 hr) and precipitation threshold (0.50 in, 1.0 in, and 2.0 in), then view the corresponding verification for WPC forecasts and 9 numerical models.


“Two things that seem to resonate with our forecasters the most are the real time aspect of the website and the visual nature of the comparison.”

The graphical nature of the MODE verification allows for a quick comparison of forecasts in a way that goes beyond traditional threat scores and bias values to consider other measures of forecast quality (distance between forecast and observed objects, differences in angle of orientation and object size, etc.). The most recent update to the website attempts to better quantify these qualities by adding statistical comparisons of the interest value and the displacement distance between matched objects to complement the traditional graphical comparisons. Future plans include additional statistical information on the website, including longer term summaries (monthly, annually, etc.), and making the website available to the public.

Running MODE on a national scale at an operational center like WPC can present some unique challenges since MODE must be able to correctly identify precipitation objects from meteorological phenomena as varied as cool season synoptic scale storms to warm season convection. Determining the ideal configuration is still a work in progress, but it is an essential piece of the puzzle in order to build forecaster confidence in the utility of object-based verification.


DTC Science Advisory Board Meeting

Autumn 2013

Given its mission to facilitate the research to operations transition in numerical weather prediction, the DTC has a mandate to stay connected with both the research and operational NWP communities.

As a means to that end, the DTC Science Advisory Board (SAB) was established to provide (i) advice on strategic directions, (ii) recommendations for new code or new NWP innovations for testing and evaluation, and (iii) reviews of DTC Visitor Program proposals and recommendations for selection.

The third meeting of the SAB (the first with the new members announced in an earlier DTC Newsletter) was held recently (25-27 September 2013) in Boulder. To stimulate communication between the research and operational NWP communities, the DTC invited Geoff DiMego and Vijay Tallapragada to present future plans for mesoscale and hurricane modeling at the National Centers for Environmental Prediction’s (NCEP) Environmental Modeling Center (EMC), and Mike Horner to give an Air Force Weather Agency (AFWA) modeling update. Both informative presentations stimulated considerable discussion.

John Murphy, Chair of the DTC Executive Committee (EC), presented NWS’s view on research to operations and the important role of the DTC. Col. John Egentowich, also a member of DTC EC, spoke positively of the contributions of the DTC to Air Force weather prediction modeling. During summary and discussion periods, SAB members provided many valuable recommendations for the DTC to consider during their planning for the new DTC Annual Operating Plan (AOP) for 2014. These recommendations will be detailed in a DTC SAB Meeting Summary. Of these, several were of particular interest.

The SAB recommended that the DTC and EMC develop a community model-testing infrastructure at NCEP/EMC. The goal for such an infrastructure would be to allow visiting scientists easy access to EMC operational models, enabling them to collaborate with EMC scientists to perform model experiments using alternative approaches. Such an infrastructure would hopefully facilitate accelerated R2O transition in NWP.

The SAB voiced their belief that operational centers will have significantly more computing resources in the near future, putting nationwide high-resolution mesoscale ensemble forecasting within reach. Given that possibility, they recommended that DTC should help facilitate transition to cloud-permitting scale ensemble forecasting with multiple physics. The current members of the DTC Science Advisory Board can be found at http://www.dtcenter.org under governance.