NUOPC User Workshop

DTC & NUOPC Ensemble Design Workshop

  • About
  • Schedule
  • Logistics
  • Registration & Remote Information


10 - 12 September 2012


National Center for Atmospheric Research (NCAR)
Foothills Lab Campus
EOL Atrium
3450 Mitchell Lane
Boulder, Colorado 80301


This workshop will consider science issues governing the optimal use and configurations of single and multi-model ensembles (MMEs) at all scales and the simulation, quantification and presentation of forecast uncertainty. It is being jointly sponsored by the National Unified Operational Prediction Capability (NUOPC) and the Ensemble Developmental Testbed Center (DTC) Ensemble Task (DET) and will help to drive future scientific work and operational configurations. The focus of the workshop is on the design of the core of a multi-model ensemble prediction system (EPS), which includes simulation of forecast uncertainty (from both the analysis and the model), post processing of raw ensemble output, and verification of probabilistic forecast skill. The scope also includes all scales of motion (both global and regional modeling, and/or a combination thereof via adaptive grids) and all time scales including short (0-3days), medium (4 – 20 days) and up to and including inter-seasonal and annual.


The workshop will be broken into five half-day sessions, each session covering a topic of interest. These sessions will begin with three presentations on the topic of that session, including one invited talk. Following the talks, there will be group discussion to address session questions or additional questions arising from the talks.

1. MONDAY MORNING: Simulation of Model Uncertainty: Relative scientific merits and drawbacks of the multi-model vs stochastic forcing using a single-model framework.

Questions: What are the relative merits and drawbacks of MME approaches versus ensemble prediction with a unified model? Empirically, what evidence is there to support each approach? Scientifically what justification is there to support the idea that one or the other is sound? What experiments need to be performed to clarify what approach should be embraced for future systems? How do we weigh scientific benefit versus computational cost?

2. MONDAY AFTERNOON: Simulation of Model Uncertainty: Solutions. Given the discussions of the morning session, how can present ensemble systems advance to enable representation of model uncertainty?

Questions: What aspects of the MME should be replicated in a single model context (if any)? What does one have to do to a global or regional ensemble to get it to mimic the multi-model distribution, and what is the physical implication? What could be done to the GFS or NOGAPS model (or COAMPS, WRF ARW or NMMB, in regional applications) to get it to mimic the multi-model distribution, and how would that method differ from model to model? What fraction of the MME value comes from the multi analysis initial distribution of ensemble members and ensemble initial perturbations, and what fraction from the different models?

3. TUESDAY MORNING: Basic Configuration: MME issues. Given the discussions of the previous day, how can ensembles systems such as NAEFS, NUE and SREF deal with model uncertainty.

Questions: Is the only practical solution to address model uncertainty individually in each model independent of other models? Does one model say anything about the model uncertainty of a second model? Do two models? Do three model ensembles say anything about the uncertainty of a fourth? Do previous model runs (12h old or 24h old) allow us to reconfigure the ensemble to better understand/deal with model uncertainty?

4. TUESDAY AFTERNOON: Basic Configuration: Determine how to objectively tune ensemble design aspects such as number of members, forecast length, model resolution, update frequency, etc. in order to optimally use existing computing resources in meeting the needs of the end users.

Questions: What makes sense for our existing, evolving multi-model ensembles (NUOPC and SREF)? Is an equal number of members from each model the correct number of members? NUOPC specific: Do the operational centers all reduce to 15 members if NCEP starts running FIM as a global ensemble? Should the NCEP SREF, or other regional ensembles, make similar decisions? Is there actually a way to evolve the three or four global models toward a single system running at three centers? Should we try to evolve toward a single system? Additionally – should mesoscale ensemble systems move toward a single model paradigm?

5. WEDNESDAY MORNING: Verification and Post Processing

    a) Determine if current metrics (when properly employed) sufficiently evaluate the skill
        and value of probabilistic predictions such that the results can be used to
        effectively guide further advancements.
    b) Determine if standards for applying the metrics and guidance on analysis and
        interpretation are sufficient to promote optimal use of the metrics.
    c) Determine the best approach(es) for application in an operational setting.
    d) For a multi-model ensemble, determine what post processing techniques can
        identify and remove artificial clustering.

Questions: In order to maximize performance, determine ensemble run frequency, number of members, length of forecast and optimum resolution, we need to adequately measure performance against a value metric. Can we design a metric or set of metrics to answer these questions for all situations? For a given forecast? As a table? Questions: Can we identify artificial clustering due to model climate or due to structural interdependence among models? Are there ways to post process NUE output to eliminate or greatly reduce artificial clustering?

6. WEDNESDAY AFTERNOON: EXECUTIVE SESSIONS: Funding, Organization, and Leadership. Discussion of workshop results and how to move ensemble initiatives and recommendations forward. Discussion of post processing options.



Day One

        9:00       Welcoming Remarks
        9:10       Workshop Introduction

9:20-12:30     First Session: Simulation of Model Uncertainty

       12:30       Lunch

13:30-17:30   Second Session: Simulation of Model Uncertainty

       17:30       Adjourn

Day 2

9:00-12:00     Third Session: Basic Configuration

      12:00       Lunch

13:00-17:30   Fourth Session: Basic Configuration

      17:30       Adjourn

Day 3

9:00-12:00     Fifth Session: Verification and Post Processing


Special Lodging Deals

Millennium Harvest House Boulder
1345 28th Street
Boulder, CO 80302
+1 303 443 3850

Government Rate of $109 plus tax.

Reservations can be made in the following four ways.

On-Line: Make reservations via the internet:

                             Ensemble Workshop


Select Group Code, located under the Corp/Promo Code and use the following code:


E-Mail: Guests may e-mail their reservation request to

Fax: Guests may fax their reservation request to (303) 442-3821 using the attached fax information page. Word doc form

Telephone: Guests may call the hotel toll free at (800) 545-6285 and refer to the group code.

The room block will be available for booking until Friday, August 17, 2012

Travel Links

The workshop will be held at the NCAR Foothills Lab, 3450 Mitchell Lane, in Boulder, Colorado. A detailed Boulder Map is available, as well as a more specific NCAR / Hotel map. Below you will find directions from Denver International Airport and transportation links.

When you arrive at Denver International Airport (DIA):

Transportation between the Denver International Airport and Boulder

About Boulder:


Transportation via NCAR shuttle will be provided each morning and evening to the hotel listed. Please indicate on your registration form if you plan to use this service.


Email: Pam Johnson ( to register for the meeting.

Please provide the following contact information.

Abbreviation (acronym) for your institution:
Name to be used for Name Badge:

Will you be using the NCAR Shuttle?

REMOTE INFORMATION - The meeting will be available for remote attendance by GoToMeeting.

If you'd like to participate remotely, email: Pam Johnson ( to register.

Please provide the following contact information.

Abbreviation (acronym) for your institution:

GoToMeeting Information: Will be provided at a later time.