Deterministic Skill Scores
Skill scores can be a more rounded way of speaking to a forecast’s quality. They often combine aspects of the previously-listed scalar statistics and can serve as a starting point for creating your own skill score that is better suited to your forecasts’ properties.
Three of the most popular scores are Heidke Skill Score (HSS), the Hanssen-Kuipers Discriminant (HK), and the Gilbert Skill Score (GSS) and it would be worthwhile to mention them here.
Heidke Skill Score (HSS)
The HSS is a measure of the proportion correct that would be achieved by a “random” forecast, which is located in the denominator of the HSS equation. In this instance, random denotes a forecast that is completely independent of the observation dataset. In practice, the random forecast is often a climatology or persistence forecast. By combining the probability of a correct “yes” forecast (i.e. a hit) with the probability of a correct “no” forecast (i.e. a correct rejection) the resulting equation is
HSS can range from -1 to 1, with a perfect forecast receiving a score of 1. The equation presented above is a compact version which uses a grouped term, C2. The C2 term expands to
This is an important version of HSS, as METplus also calculates a modified HSS, in addition to the more traditional approach, that allows users to control what the C2 term is. See how to use these skill scores in METplus!
Hanssen-Kuipers Discriminant (HK)
HK is known by several names, including the Peirce Skill Score and the true skill statistic. How ever it is referred to, the theory and equation of HK remains the same. This skill score is similar to HSS (ranges from -1 to 1, perfect forecast is 1, etc.), but instead of a comparison to a truly random forecast, HK is formulated for a random forecast that shows no bias in the denominator. This changes the focus of the verification away from the skill of the forecast compared to random chance, and instead shows how well the forecast delineated between observed “yes” events and observed “no” events. That appears in equation form as
which is the expanded version of the POD minus the POFD. Because of its basis in POD, HK can be similarly affected by infrequent events and is suggested as a more useful skill score for frequent events. See how to use this skill score in METplus!
Gilbert Skill Score (GSS)
Finally, GSS looks at the correspondence between a forecasted event and the observed “yes” event. Sometimes called the Equitable Threat Score, the GSS is a good option for those forecasted events where the observed “yes” event is rare. This is due to its omission of correct negatives in the equation, which, with a rare observed “yes” event, would be fairly large. The GSS is given as
And ranges from -1 to 1, with a perfect forecast receiving a score of 1. Similar to HSS, a compact version of GSS is presented using the C1 term. This term expands to
