Series-Analysis Tool: Configure

The behavior of Series-Analysis is controlled by the contents of the configuration file passed to it on the command line. The default Series-Analysis configuration file may be found in the data/config/SeriesAnalysisConfig_default file.

The configurable items for Series-Analysis are used to specify how the verification is to be performed. The configurable items include specifications for the following:

• The forecast fields to be verified at the specified vertical level or accumulation interval.
• The threshold values to be applied.
• The area over which to limit the computation of statistics - as predefined grids or configurable lat/lon polylines.
• The confidence interval methods to be used.
• The smoothing methods to be applied.
• The types of statistics to be computed.

You may find a complete description of the configurable items in the series_analysis configuration file section of the MET User's Guide. Please take some time to review them.

• Set the fcst dictionary to
  fcst = {
     field = [
        {
          name  = "APCP";
          level = [ "A3" ];
        }
     ];
  }

  To request the GRIB abbreviation for precipitation (APCP) accumulated over 3 hours (A3).

• Delete obs = fcst; and insert
  obs = {
     field = [
        {
          name  = "APCP_03";
          level = [ "(*,*)" ];
        }
     ];
  }

  To request the NetCDF variable named APCP_03 where its two dimensions are the gridded dimensions (*,*).

• Look up a few lines above the fcst dictionary and set
  cat_thresh = [ >0.0, >=5.0 ];

  To define the categorical thresholds of interest. By defining this at the top level of config file context, these thresholds will be applied to both the fcst and obs settings.

• In the mask dictionary, set
  grid = "G212";

  To limit the computation of statistics to only those grid points falling inside the NCEP Grid 212 domain.

• Set
  block_size = 10000;

  To process 10,000 grid points in each pass through the data. Setting block_size larger should make the tool run faster but use more memory.

• In the output_stats dictionary, set
     fho    = [ "F_RATE", "O_RATE" ];
     ctc    = [ "FY_OY", "FN_ON" ];
     cts    = [ "CSI", "GSS" ];
     mctc   = [];
     mcts   = [];
     cnt    = [ "TOTAL", "RMSE" ];
     sl1l2  = [];
     pct    = [];
     pstd   = [];
     pjc    = [];
     prc    = [];

  For each line type, you can select statistics to be computed at each grid point over the series. These are the column names from those line types. Here, we select the forecast rate (FHO: F_RATE), observation rate (FHO: O_RATE), number of forecast yes and observation yes (CTC: FY_OY), number of forecast no and observation no (CTC: FN_ON), critical success index (CTS: CSI), and the Gilbert Skill Score (CTS: GSS) for each threshold, along with the root mean squared error (CNT: RMSE).

Series-Analysis Tool: Run

Note that the previous set of PCP-Combine commands could easily be run by looping through times in METplus Wrappers! The MET tools are often run using METplus Wrappers rather than typing individual commands by hand. You'll learn more about automation using the METplus Wrappers throughout the tutorial.

The statistics we requested in the configuration file will be computed separately for each grid location and accumulated over a time series of eight three-hour accumulations over a 24-hour period. Each grid point will have up to 8 matched pair values.

Note how long this command line is. Imagine how long it would be for a series of 100 files! Instead of listing all of the input files on the command line, you can list them in an ASCII file and pass that to Series-Analysis using the -fcst and -obs options.

Series-Analysis Tool: Output

The output of Series-Analysis is one NetCDF file containing the requested output statistics for each grid location on the same grid as the input files.

You may view the output NetCDF file that Series-Analysis wrote using the ncdump utility. Run the following command to view the header of the NetCDF output file:

In the NetCDF header, we see that the file contains many arrays of data. For each threshold (>0.0 and >=5.0), there are values for the requested statistics: F_RATE, O_RATE, FY_OY, FN_ON, CSI, and GSS. The file also contains the requested RMSE and TOTAL number of matched pairs for each grid location over the 24-hour period.

Next, run the ncview utility to display the contents of the NetCDF output file:

Click through the different variables to see how the performance varies over the domain. Looking at the series_cnt_RMSEvariable, are the errors larger in the south eastern or north western regions of the United States?

Why does the extent of missing data increase for CSI for the higher threshold? Compare series_cts_CSI_gt0.0 to series_cts_CSI_ge5.0. (Hint: Find the definition of Critical Success index (CSI) in the MET User's Guide and look closely at the denominator.)

Try running Plot-Data-Plane to visualize the observation rate variable for non-zero precipitation (i.e. series_fho_O_RATE_gt0.0). Since the valid range of values for this data is 0 to 1, use that to set the -plot_range option.

Setting block_size to 10000 still required 3 passes through our 185x129 grid (= 23865 grid points). What happens when you increase block_size to 24000 and re-run? Does it run slower or faster?