The Jive verification system and heatwave warning verification

Part 1 -The Jive verification system at the Bureau of Meteorology-Jive is a Python-based verification system that was originally developed to assess the quality of seven-day weather forecasts issued by the Australian Bureau of Meteorology. It is based on xarray, allowing Jive to do n-dimensional verification “in-memory”. For larger datasets that don’t fit into memory, it scales using Dask which allows parallel computations.
Work on Jive began in 2015 in response to the need to verify the Bureau’s weather forecasts to understand their quality and in 2022 became an operational system. This presentation gave an overview of the verification system and highlighted some examples of the positive impacts that verification from Jive has had on the Bureau in streamlining the forecast production process while improving the skill of the forecasts.
Part 2 – Verification of the Bureau’s new heatwave warning system using the Fixed Risk Multicategorical (FIRM) score.
In Australia, heatwaves cause more fatalities and illnesses than any other natural disaster. For several years, the Bureau of Meteorology has issued heatwave forecasts based on the Excess Heat Factor (EHF). A new multicategorical warning service was released to the public this summer. BoM’s heatwave warnings are verified using a new verification score called the Fixed Risk Multicategorica (FIRM) score (Taggart et al. 2022). The FIRM score is a consistent scoring function for multicategorical forecasts that are based on fixed risk measures. This contrasts with most other multicategorical verification scores where the optimal probability to warn is based around sample base rates rather than risk measures suitable for use.
This presentation described how the FIRM score is calculated and show verification results that were produced with Jive. Challenges with accurately forecast heatwaves based on the EHF index (particularly in the tropics) was discussed.