Output

The output of Ensemble-Stat is a NetCDF file containing the gridded fields of ensemble forecast values, a NetCDF file containing the gridded fields of observation ranks (if gridded observations provided), and one or more ASCII files containing statistics summarizing the verification performed. In this example, the output is written to the $MET_TUTORIAL_DATA/output/ensemble_stat directory as we requested on the command line.

All of the ASCII statistics output is written to the file ending in .stat. While other MET statistics tools write many output line types, Ensemble-Stat currently writes six, ECNT, RHIST, PHIST, ORANK, SSVAR, and RELP. The ECNT line type contains continuous ensemble statistics. The RHIST line type contains counts for a ranked histogram. The PHIST line type contains counts for a probability integral transform histogram. The ORANK line type is similar to the matched pair (MPR) output of Point-Stat. For each point observation value, one ORANK line is written out containing the observation value, it's rank, and the corresponding ensemble values for that point. The SSVAR line contains binned spread/skill information. Lastly, the RELP line is the relative position, which is similar to the rank histogram line type, but no ranking is done. For each observation location, the ensemble variance is computed at that point. Those variance values are binned based on the ens_ssvar_bin_size configuration setting, and one SSVAR line is written for each bin summarizing the data it contains.

The STAT file contains all the ASCII output while the _ecnt.txt, _rhist.txt, _phist.txt, _orank.txt, _ssvar.txt, and _relp.txtfiles contain the same data but sorted by line type. Since so much data can be written for the ORANK line type, we recommend disabling the output of the optional text file using the output_flag parameter in the configuration file.

Since the lines of data in these ASCII file are so long, we strongly recommend configuring your text editor to NOT use dynamic word wrapping. The files will be much easier to read that way.

Open up the $MET_TUTORIAL_DATA/output/ensemble_stat/ensemble_stat_20100101_120000V_rhist.txt RHIST file using the text editor of your choice and note the following:

• There are 6 lines in this output file resulting from using 3 verification regions (FULL, NWC, and SWC) and two observations datasets (ADPSFC point observations and gridded observations).
• Each line contains columns for the observations ranks (RANK_#), the continuous ranked probability score (CRPS), and the ignorance score (IGN).
• There is output for 7 ranks - since we verified a 6-member ensemble, there are 7 possible ranks the observation values could attain.

Close this file, and open up the $MET_TUTORIAL_DATA/output/ensemble_stat/ensemble_stat_20100101_120000V_phist.txt PHIST file, and note the following:

• There are 5 lines in this output file resulting from using 3 verification regions (FULL, NWC, and SWC) and two observations datasets (ADPSFC point observations and gridded observations), where the ADPSFC point observations for the SWC region were all zeros for which the probability integral transform is not defined.
• Each line contains columns for the BIN_SIZE and counts for each bin. The bin size is set in the configuration file using the ens_phist_bin_size field. In this case, it was set to .05, therefore creating 20 bins (1/ens_phist_bin_size).

Close this file, and open up the $MET_TUTORIAL_DATA/output/ensemble_stat/ensemble_stat_20100101_120000V_orank.txt ORANK file, and note the following:

• This file contains 1866 lines, 1 line for each observation value falling inside each verification region (vx_mask).
• Each line contains 42 columns header information, the observation location and value, it's rank, and the 6 values for the ensemble members at that point.
• When there are ties, Ensemble-Stat randomly assigns a rank from all the possible choices. This can be seen in the SWC masking region where all of the observed values are 0 and the ensemble forecasts are 0 as well. Ensemble-Stat randomly assigns a rank between 1 and 7.

Close this file. You have opened the _rhist.txt, _phist.txt, and _orank.txt output files, but not the _ecnt.txt, _relp.txt, and _ssvar.txt files. For more detailed information regarding those files, take a look at Table 9.2, Table 9.5, and Table 9.7, respectively, in the METv8.0 User's Guide.

se the ncview utility to view the NetCDF ensemble fields file:

This file contains variables for the following:

1. Ensemble Mean
2. Ensemble Standard Deviation
3. Ensemble Mean minus 1 Standard Deviation
4. Ensemble Mean plus 1 Standard Deviation
5. Ensemble Minimum
6. Ensemble Maximum
7. Ensemble Range
8. Ensemble Valid Data Count
9. Ensemble Relative Frequency (for 3 thresholds)

The output of any of these summary fields may be disabled using the output_flag parameter in the configuration file.

Use the ncview utility to view the NetCDF gridded observation rank file:

This file is only created when you've verified using gridded observations and have requested its output using the output_flag parameter in the configuration file. Click through the variables in this file. Note that for each of the three verification areas (FULL, NWC, and SWC) this file contains 4 variables:

1. The gridded observation value
2. The observation rank
3. The probability integral transform
4. The ensemble valid data count

Close this file.