Series-Analysis Tool: Configure

Start by making an output directory for Series-Analysis and changing directories:

The behavior of Series-Analysis is controlled by the contents of the configuration file passed to it on the command line. The default Series-Analysis configuration file may be found in the data/config/SeriesAnalysisConfig_default file. Prior to modifying the configuration file, users are advised to make a copy of the default:

The configurable items for Series-Analysis are used to specify how the verification is to be performed. The configurable items include specifications for the following:

• The forecast fields to be verified at the specified vertical level or accumulation interval.
• The threshold values to be applied.
• The area over which to limit the computation of statistics - as predefined grids or configurable lat/lon polylines.
• The confidence interval methods to be used.
• The smoothing methods to be applied.
• The types of statistics to be computed.

You may find a complete description of the configurable items in section 13.2.3 of the MET User's Guide. Please take some time to review them.

For this tutorial, we'll run Series-Analysis to verify a time series of 3-hour accumulated precipitation. We'll use GRIB1 for the forecast files and NetCDF for the observation files. Since the forecast and observations are different file formats, we'll specify the name and level information for them slightly differently.

Open up the SeriesAnalysisConfig_tutorial file for editing with your preferred text editor and edit it as follows:

• Set the fcst dictionary to
  
  fcst = {
     field = [
        {
          name  = "APCP";
          level = [ "A3" ];
        }
     ];
  }

  To request the GRIB abbreviation for precipitation (APCP) accumualted over 3 hours (A3).

• Delete obs = fcst; and insert
  
  obs = {
     field = [
        {
          name  = "APCP_03";
          level = [ "(*,*)" ];
        }
     ];
  }

  To request the NetCDF variable named APCP_03 where it's two dimensions are the gridded dimesnions (*,*).

• Set cat_thresh = [ >0.0, >=5.0 ];
  To define the categorical thresholds of interest. By defining this at the top level of config file context, these thresholds will be applied to both the fcst and obs settings.
• In the mask dictionary, set grid = "G212";
  To limit the computation of statistics to only those grid points falling inside the NCEP Grid 212 domain.
• Set block_size = 10000;
  To process 10,000 grid points in each pass through the data. Setting block_size larger should make the tool run faster but use more memory.
• In the output_stats dictionary, set
  
     fho    = [ "F_RATE", "O_RATE" ];
     ctc    = [ "FY_OY", "FN_ON" ];
     cts    = [ "CSI", "GSS" ];
     mctc   = [];
     mcts   = [];
     cnt    = [ "TOTAL", "RMSE" ];
     sl1l2  = [];
     pct    = [];
     pstd   = [];
     pjc    = [];
     prc    = [];

  For each line type, you can select statistics to be computed at each grid point over the series. These are the column names from those line types. Here, we select the forecast rate (FHO: F_RATE), observation rate (FHO: O_RATE), number of forecast yes and observation yes (CTC: FY_OY), number of forecast no and observation no (CTC: FN_ON), critical success index (CTS: CSI), and the Gilbert Skill Score (CTS: GSS) for each threshold, along with the root mean squared error (CNT: RMSE).

Save and close this file.

Series-Analysis Tool: Run

First, we need to prepare our observations by putting 1-hourly StageII precipitation forecasts into 3-hourly buckets. Run the following PCP-Combine commands to prepare the observations:

Note that the previous set of PCP-Combine commands could easily be run by looping through times in a shell script or through a METplus use case! The MET tools are often run using a scripting language rather than typing individual commands by hand. You'll learn more about automation using the METplus python scripts.

Next, we'll run Series-Analysis using the following command:

The statistics we requested in the configuration file will be computed separately for each grid location and accumulated over a time series of eight three-hour accumulations over a 24-hour period. Each grid point will have up to 8 matched pair values.

Note how long this command line is. Imagine how long it would be for a series of 100 files! Instead of listing all of the input files on the command line, you can list them in an ASCII file and pass that to Series-Analysis using the -fcst and -obs options.

Series-Analysis Tool: Output

The output of Series-Analysis is one NetCDF file containing the requested output statistics for each grid location on the same grid as the input files.

You may view the output NetCDF file that Series-Analysis wrote using the ncdump utility. Run the following command to view the header of the NetCDF output file:

In the NetCDF header, we see that the file contains many arrays of data. For each threshold (>0.0 and >=5.0), there are values for the requested statistics: F_RATE, O_RATE, FY_OY, FN_ON, CSI, and GSS. The file also contains the requested RMSE and TOTAL number of matched pairs for each grid location over the 24-hour period.

Next, run the ncview utility to display the contents of the NetCDF output file:

Click through the different variables to see how the performance varies over the domain. Looking at the series_cnt_RMSEvariable, are the errors larger in the south eastern or north western regions of the United States?

Why does the extent of missing data increase for CSI for the higher threshold? Compare series_cts_CSI_gt0.0 to series_cts_CSI_ge5.0. (Hint: Find the definition of CSI in Appendix C of the MET User's Guide and look closely at the denominator.)

Try running Plot-Data-Plane to visualize the observation rate variable for non-zero precipitation (i.e. series_fho_O_RATE_gt0.0). Since the valid range of values for this data is 0 to 1, use that to set the -plot_range option.

Setting block_size to 10000 still required 3 passes through our 185x129 grid (= 23865 grid points). What happens when you increase block_size to 24000 and re-run? Does it run slower or faster?