As a DTC Visitor in 2015, Paul Roebber explored an idea for generating ensemble weather predictions known as evolutionary programming (EP). The method relies on a gradually and increasingly restrictive cost function to produce and to evaluate succeeding generations of a population of algorithms until such time as a best ensemble solution is determined based on cross-validation. The approach was developed by Roebber to produce baseline prediction equations equivalent to linear or nonlinear multiple regression equations (a kind of model output statistics or MOS) modified by if-then conditionals and using observations as well as numerical weather prediction (NWP) model output.
The prime objective of his DTC Visitor project was to explore possible improvements to the method. A first step, using the Yellowstone supercomputer, was to consider the relative contribution of large ensemble populations, numbering from 3,000 to as many as 500,000 possible members, to ensemble diversity. As illustrated in the figure below for 60 hour forecasts of minimum temperature, smaller as well as very large EP ensembles outperform the GFS 21-member ensemble MOS forecasts in both a deterministic (RMSE) and probabilistic (Brier Skill Score; BSS) sense, but the increase in ensemble size (indicated by the size of the bubbles) provides only minor additional skill.
Specific issues explored in the context of next-day heavy convective rainfall forecasting included: the performance of the method regionally and locally, compared to multiple logistic regression (MLR) and artificial neural networks (ANN); and ensemble member selection for use in bias calibration such as Bayesian Model Combination.
As illustrated in the performance diagram in the figure below for regional forecasts of rainfall in excess of 1.5 inches, the MLR and EP demonstrate comparable skill, and both superior to that of a trained ANN. The slightly different performance characteristics (higher hits and false alarms versus lower hits and false alarms) of the three methods suggests the possibility of combining the information in useful ways operationally. Insights gained from this work are leading to several collaborations with NOAA scientists related to adaptive systems and deep learning networks.
