The behavior of Grid-Stat is controlled by the contents of the configuration file passed to it on the command line. The default Grid-Stat configuration file may be found in the $MET_BASE/config/GridStatConfig_default file. The configurations used by the test script may be found in the met-8.0/scripts/config/GridStatConfig* files. Prior to modifying the configuration file, users are advised to make a copy of the default:
cp $MET_BASE/config/GridStatConfig_default $MET_TUTORIAL_DATA/config/GridStatConfig_POP_12
The configurable items for Grid-Stat are used to specify how the verification is to be performed. The Grid-Stat configuration file should look very similar to the one for Point-Stat. The configurable items include specifications for the following:
- The verification domain.
- The forecast fields to be verified at the specified vertical level or accumulation interval.
- The threshold values to be applied.
- The economic cost-loss value ratios to be evaluated.
- The reference climatological dataset.
- The areas over which to aggregate statistics - as predefined grids or configurable lat/lon polylines.
- The confidence interval methods to be used.
- The smoothing methods to be applied (as opposed to interpolation methods).
- The types of verification methods to be used.
You may find a complete description of the configurable items in the MET Users Guide or in the $MET_BASE/config/README file. Please take some time to review them.
For this tutorial, we'll run Grid-Stat twice - once to verify the 12-hour accumulated precipitation output of PCP-Combine and once to apply the probabilistic verification methods to a 12-hour probability of precipitation forecast. In the first run, we'll use NetCDF for both the forecast and observation files. In the second run, we'll use a GRIB forecast file and a NetCDF observation file. While we'll use Grid-Stat to verify only one field at a time, it may be configured to verify more than one field at a time.
Open up the $MET_TUTORIAL_DATA/config/GridStatConfig_APCP_12 file for editing with your preferred text editor and edit it as follows:
- In the fcst dictionary, set
field = [
{
name = "APCP_12";
level = "(*,*)";
cat_thresh = [ >0, >=5.0, >=10.0 ];
}
];To verify the NetCDF variable of that name and apply the 3 thresholds listed. Accumulated precipitation is in millimeters.
- Set obs = fcst;
To use the settings from the fcst dictionary above. - In the mask dictionary, set grid = [ "G212" ]; To accumulate statistics over NCEP Grid 212 domain.
- In the mask dictionary, set
poly = [ "${MET_TUTORIAL_DATA}/output/gen_vx_mask/CONUS_G212_poly.nc",
"${MET_BASE}/poly/EAST.poly",
"${MET_BASE}/poly/WEST.poly" ];To accumulate statistics over the entire CONUS using the NetCDF output of the Gen-Poly-Mask tool and over the regions defined by the EAST and WEST polyline files.
- In the nbrhd dictionary, set width = [ 3, 5 ];
To select two neighborhood sizes over which to accumulate neighborhood statistics. - In the nbrhd dictionary, set cov_thresh = [ >=0.5, >=0.75 ];
To define the fractional coverage threshold values of interest. - Set
output_flag = {
fho = NONE;
ctc = BOTH;
cts = BOTH;
mctc = NONE;
mcts = NONE;
cnt = BOTH;
sl1l2 = BOTH;
sal1l2 = NONE;
vl1l2 = NONE;
val1l2 = NONE;
vcnt = NONE;
pct = NONE;
pstd = NONE;
pjc = NONE;
prc = NONE;
eclv = NONE;
nbrctc = BOTH;
nbrcts = BOTH;
nbrcnt = BOTH;
grad = NONE;
};To indicate that contingency table counts (CTC), contingency table statistics (CTS), continuous statistics (CNT), scalar partial sums (SL1L2), neighborhood contingency table counts (NBRCTC), neighborhood contingency table statistics (NBRCTS), and neighborhood continuous statistics (NBRCNT) should be output.
- In the nc_pairs_flag dictionary, check that diff = true;
To indicate that the NetCDF difference field should be output.
Note that we are not requesting multi-category contingency table output, MCTC and MCTS lines. While we are specifying multiple thresholds (>0.0, >=5.0, >=10.0), they are not all of the same type (">" versus ">=") which would cause an error.
Save and close this file and open up the $MET_TUTORIAL_DATA/config/GridStatConfig_POP_12 file for editing with your preferred text editor and edit it as follows:
- In the fcst dictionary, set
field = [
{
name = "POP";
level = "Z0";
prob = TRUE;
cat_thresh = [ >=0.0, >=0.25, >=0.50, >=0.75, >=1.0 ];
}
];To verify the 12-hour probability of precipitation forecast from the input GRIB file and apply the probabilistic thresholds listed.
- Set the obs dictionary to
obs = {
field = [
{
name = "APCP_12";
level = "(*,*)";
cat_thresh = [ >=0.0 ];
}
];
};To verify against the NetCDF variable of that name in the observation file and define the probabilistic event as any non-zero precipitation.
- In the mask dictionary, set grid = [ "G212" ]; To accumulate statistics over NCEP Grid 212 domain.
- In the mask dictionary, set
poly = [ "${MET_TUTORIAL_DATA}/output/gen_vx_mask/CONUS_G212_poly.nc",
"${MET_BASE}/poly/EAST.poly",
"${MET_BASE}/poly/WEST.poly" ];To accumulate statistics over the entire CONUS using the NetCDF output of the Gen-Poly-Mask tool and over the regions defined by the EAST and WEST polyline files.
- Set
output_flag = {
fho = NONE;
ctc = NONE;
cts = NONE;
mctc = NONE;
mcts = NONE;
cnt = NONE;
sl1l2 = NONE;
sal1l2 = NONE;
vl1l2 = NONE;
val1l2 = NONE;
vcnt = NONE;
pct = BOTH;
pstd = BOTH;
pjc = BOTH;
prc = BOTH;
eclv = NONE;
nbrctc = NONE;
nbrcts = NONE;
nbrcnt = NONE;
grad = NONE;
};To indicate that probability contingency table counts (PCT), probability statistics (PSTD), joint/continuous probabilistic statistics (PJC), and probabilistic ROC curve points (PRC) should be output.
- In the nc_pairs_flag dictionary, check that diff = true;
To indicate that the NetCDF difference field should be output.
Save and close this file.
