The behavior of Stat-Analysis is controlled by the contents of the configuration file or the job command passed to it on the command line. The default Stat-Analysis configuration may be found in the $MET_BASE/config/STATAnalysisConfig_default file. The configuration used by the test script may be found in the met-8.0/scripts/config/STATAnalysisConfig file. Prior to modifying the configuration file, users are advised to make a copy of the default:

Open up the $MET_TUTORIAL_DATA/config/STATAnalysisConfig_tutorial file for editing with your preferred text editor.

For this tutorial, we'll set up a configuration file to run a few jobs. Then, we'll show an example of running a single analysis job on the command line.

The Stat-Analysis configuration file has two main sections. The items in the first section are used to filter the STAT data being processed. Only those lines which meet the filtering requirements specified are retained and passed down to the second section. The second section defines the analysis jobs to be performed on the filtered data. When defining analysis jobs, additional filtering parameters may be defined to further refine the STAT data with which to perform that particular job.

As a word of caution, the Stat-Analysis tool is designed to be extremely flexible. However, with that flexibility comes potential for improperly specifying your job requests, leading to unintended results. It is the user's responsibility to ensure that each analysis job is performed over the intended subset of STAT data. The -dump_row job command option is useful for verifying that the analysis was performed over the intended subset of STAT data.

We'll configure the Stat-Analysis tool to analyze the results of the Point-Stat tool output and aggregate scores for the EAST and WEST verification regions. Edit the $MET_TUTORIAL_DATA/config/STATAnalysisConfig_tutorial file as follows:

• Set fcst_var = [ "TMP" ]; To only use STAT lines for temperature (TMP).
• Set fcst_lev = [ "P850-750" ]; To only use STAT lines for the forecast level specified.
• Set obtype = [ "ADPUPA" ]; To only use STAT lines verified with the ADPUPA message type.
• Set vx_mask = [ "EAST", "WEST" ]; To use STAT lines computed over the EAST or WEST verification polyline regions.
• Set line_type = [ "CTC" ]; To only use the CTC lines.
• Set jobs as follows:

Save and close this file. The four jobs listed above achieve the following:

1. Filter out those STAT lines which meet the filtering criteria, and write them to an output STAT file.
2. Aggregate the CTC lines which have the INTERP_PNTS column set to 1 and write the lines which meet the filtering criteria to an output STAT file.
3. Aggregate the CTC lines which have the INTERP_PNTS column set to 25 and write the lines which meet the filtering criteria to an output STAT file.
4. Do the same as the third job, but write out the aggregated contingency table stats (CTS) rather than the aggregated contingency table counts (CTC).

Note that all four jobs use the -dump_row job command option which dumps the lines of STAT data used for this job to the specified file name. We'll look at these files to ensure that the jobs ran over the intended subsets of STAT data.

Next, run Stat-Analysis on the command line using the following command:

Stat-Analysis is now performing the analysis jobs we requested in the configuration file. It is writing the output of our four jobs to the file we specified using the -out command line argument. It should run in only a couple of seconds since we're analyzing such a small sample of STAT data. In general though, Stat-Analysis can be used to process very large amounts of data, a whole season's worth, in a relatively short amount of time.

By Case and STAT Output

Notice that jobs 2 and 3 do basically the same thing but for different -interp_pnts values. We can actually perform those job in a much simpler way. The -by job command option specifies one or more columns which define case information. Those case columns are concatenated and the job is performed for each unique case found in the data.

Also notice that while the output file generated above contains aggregated statistics, it is missing the 22 header STAT header columns. The -out_stat job command option specfies that name for an output STAT file which does contain those header columns.

Let's rerun jobs 2 and 3 but on the command line using both the -by and -out_stat options:

Note that this single job was run on the command line with no configuration file. The multiple values for -vx_mask are specified as a comma-separated list (specifying -vx_mask multiple times works too. Open up the output STAT file ($MET_TUTORIAL_DATA/output/stat_analysis/ctc_by_interp_pnts.stat) and notice the content of the VX_MASK column (EAST,WEST). For string header columns (e.g. VX_MASK, OBTYPE, FCST_VAR) multiple values are simply concatenated while for date/time columns (e.g. FCST_VALID_BEG, FCST_VALID_END, FCST_LEAD) the min/max values are reported.

Next, rerun that same job but use the -set_hdr option to explicitly specify the contents of the output VX_MASK column:

Open up the output file and check the VX_MASK column.

You can use the -by option an arbitrary number of times to define case information. Try rerunning with -by INTERP_PNTS,VX_MASK.

Process Probabilistic Output

Next, we'll run an analysis job on the probabilistic output from the Grid-Stat tool on the command line:

The output of this Stat-Analysis job is printed to the screen since we didn't redirect the output to a file using the -out command line option. This job has aggregated two probability contingency table count (PCT) STAT lines, one for the EAST and one for the WEST, and it has written out the corresponding statistics (PSTD) STAT line. The output for this job includes the following 3 lines:

• The JOB_LIST line lists the job command options that were used to perform this job.
• The COL_NAME line consists of the column names for the statistics listed in the next line.
• The PSTD line consists of the output for the probabilistic statistics (PSTD) STAT line. However, only the columns that appear after the LINE_TYPE column are shown.

As we did above, let's switch to using the -by vx_mask option:

This job has aggregated the probability contingency table count (PCT) STAT lines by vx_mask and has written out the corresponding statistics (PSTD) STAT line. There is one output line per vx_mask (CONUS, EAST, G212, WEST) for a total of four lines in this case.

Next, we'll run a job to look at verification of wind direction in the output of the Point-Stat tool. Just as we did above, we'll use the EAST and WEST verification regions to aggregate the vector partial sums (VL1L2) for winds and look at errors in the wind direction. Run the following job on the command line:

The output of this Stat-Analysis job includes the following 4 lines: the JOB_LIST, COL_NAME, ROW_MEAN_WDIR, and AGGR_WDIR. See the MET Users Guide for details on these output lines.

Process Ensemble Output

Lastly, we'll aggregate together two ranked histograms from the Ensemble-Stat output, the ones for the NWC and SWC verification areas. Execute the following command:

This job aggregates together the 358 ranks from the NWC region with the 382 ranks from the SWC region and writes out the aggregated counts. This aggregation is only possible because the number of ranks in each input line (7) remains constant.

The output of Stat-Analysis is printed to the screen by default, as we saw, unless you redirect it to an output file using the -out command line option. View the output of the first Stat-Analysis command we ran by opening the $MET_TUTORIAL_DATA/output/stat_analysis/stat_analysis.out file using the text editor of your choice. Note the following:

• The output of the first job to simply filter the STAT data consists of a FILTER line listing the filtering parameters applied.
• The output of the second and third jobs consists of 3 lines each: the JOB_LIST, COL_NAME, and CTC lines. This is the same type of output that was printed to the screen for the Stat-Analysis job run on the command line.
• The output of the fourth job consists of 3 lines: the JOB_LIST, COL_NAME, and CTS lines.

Close this file, and open up the following four files to examine the STAT data over which these jobs were run:

• $MET_TUTORIAL_DATA/output/stat_analysis/job1_filter.stat
• $MET_TUTORIAL_DATA/output/stat_analysis/job2_aggr_ctc_1.stat
• $MET_TUTORIAL_DATA/output/stat_analysis/job3_aggr_ctc_25.stat
• $MET_TUTORIAL_DATA/output/stat_analysis/job4_aggr_stat_cts.stat

In this example, the second, third, and fourth jobs read two CTC STAT lines each, using the masking regions EAST and WEST. So these three jobs aggregate contingency table counts across two verification regions. Users are strongly encouraged to use the -dump_row job command option to verify that the analysis job was run over the intended subset of STAT data. Close these files when you have finished reviewing the STAT data.