Prequisites: Software

The following is a full list of software that is required to utilize all the functionality in METplus.
NOTE: We will not by running the cyclone plotter or grid plotting scripts in this tutorial. So the additional libraries mentioned below are not installed.

• Python 2.7.x
• Additional Python libraries: - required by the cyclone plotter and grid plotting scripts.
  • numpy
  • matplotlib - not installed, Is is not required for this tutorial.
  • cartopy - not installed, It is not required for this tutorial.
• MET 8.1
• R - used by tcmpr_plotter_wrapper, Wraps the MET plot_tcmpr.R script
• ncap2 - NCO netCDF Operators
• wgrib2
• ncdump - NetCDF binaries
• convert - Utilitity from ImageMagick software suite

Getting The Software

For this tutorial we'll use the METplus v2.2 release. We will configure METplus to run the met-8.1 software, which has already been installed in a common location and is available through a module file.

Please use the following instructions to setup METplus on your tutorial machine:

• Unpack the release tarball and create a METplus symbolic link (copy and paste the following commands):

Sample Data Input and Location of Output

The Sample Input Data for running this METplus tutorial is provided in a shared location on-disk and does not require copying to your home directory. How to set the path to the sample input data and location of the output when running METplus will be explained later in the tutorial, utilizing METplus configuration files.

Just note that:
When running METplus use cases, all output is placed in your ${HOME}/metplus_tutorial/output directory.
When running MET on the command line, all output will be written to ${HOME}/metplus_tutorial/output/met_output

Your Runtime Environment

For this tutorial, the MET software has been setup as a loadable module file. We need to add the following command to your environment so the MET software is available when you login.

Edit the .usermodlogin file to configure your runtime environment to load the MET software:

Open the following file with the editor of your choice:

Add the following to the LAST LINE of the file:

Save and close that file, and then source it to apply the settings to the current shell. The next time you login, you do not have to source this file again.:

The system PATH environment variable is set to allow the METplus application to be run from any directory.
As a note, there is also an optional JLOGFILE environment variable (Job Log), which indicates where JLOGS will be saved. JLOGS provide an overview of the METplus python modules being run and is intended to be used when running METplus within a workflow management systems on an HPC cluster. If this environment variable is unset, then output will be directed to stdout (your display). JLOGFILE will not be defined during the tutorial, If interested, more information regarding the JLOGFILE can be found in the METplus Users Guide. You will now edit the system PATH.

Edit the .usermodrc file to configure your runtime environment for use with METplus:

open the following file with the editor of your choice:

Add the following to update your system PATH setting:
NOTE: If the PATH variable already exists in your .usermodrc file,
than make certain to ADD the line below after the existing setting.

csh:

bash:

Save and close that file, and then source it to apply the settings to the current shell. The next time you login, you do not have to source this file again.:

METplus directory structure

Brief description and overview of the METplus/ directory structure.

• doc/ - Doxygen files for building the html API documentation.
• doc/METplus_Users_Guide - LyX files and the user guide PDF file.
• internal_tests/ - for engineering tests
• parm/ - where config files live
• README.md - general README
• sorc/ - executables and documentation build system
• ush/ - python scripts

METplus default configuration files

The METplus default configuration files metplus_system.conf, metplus_data.conf, metplus_runtime.conf, and metplus_logging.conf are always read by default and in the order shown. Any additional configuration files passed in on the command line are than processed in the order in which they are specified. This allows for each successive conf file the ability to override variables defined in any previously processed conf files. It also allows for defining and setting up conf files from a general (settings used by all use cases, ie. MET install dir) to more specific (Plot type when running track and intensity plotter) structure. The idea is to created a hiearchy of conf files that is easier to maintain, read, and manage. It is important to note, running METplus creates a single configuration file, which can be viewed to understand the result of all the conf file processing.

TIP: The final metplus conf file is defined here:
metplus_runtime.conf:METPLUS_CONF={OUTPUT_BASE}/metplus_final.conf
Use this file to see the result of all the conf file processing, this can be very helpful when troubleshooting,

NOTE: The syntax for METplus configuration files MUST include a "[section]" header with the variable names and values on subsequent lines.

Unless otherwise indicated, all directories are relative to your ${HOME}/METplus directory.

The parm/metplus_config directory - there are four config files:

1. metplus_system.conf
   • contains "[dir]" and "[exe]" to set directory and executable paths
   • any information specific to host machine
2. metplus_data.conf
   • Sample data or Model input location
   • filename templates and regex for filenames
3. metplus_runtime.conf
   • contains "[config]" section
   • var lists, stat lists, configurations, process list
   • anything else needed at runtime
4. metplus_logging.conf
   • contains "[dir]" section for setting the logcation of log files.
   • contains "[config]" section for setting various logging configuration options.

The parm/use_cases directory - The use cases:
This is where the use cases you will be running exist. The use case filenaming and directory structure is by convention. It is intended to simplify management of common configuration items for a specific use case. Under the use_cases directory is a subdirectory for each use case. Under that is an examples subdirectory and a more general configuration file that contains information that is shared by examples of the use case.

• For example, the track and intensity use case conf directory and file structure.
• track_and_intensity - directory
• track_and_intensity/track_and_intensity.conf - Any common conf file settings shared by track and intensity examples
• track_and_intensity/examples - directory to hold various track and intensity example conf files.
• track_and_intensity/examples/tcmpr_mean_median.conf - specific use case

Within each use case there is a met_config subdirectory that contains MET specific config files for that use case. The parm/use_cases/<use case>/met_config directory - store all your MET config files, e.g.:

• GridStatConfig_anom (for MET grid_stat)
• TCPairsETCConfig (for MET tc_pairs)
• SeriesAnalysisConfig_by_init (for MET series_analysis)
• SeriesAnalysisConfig_by_lead (for MET series_analysis)

Create a METplus custom conf file

In this section you will create a custom configuration file that you will use in each call to METplus.

The paths in this practical assume: You have installed METPlus under your $HOME directory and You are using the shared installation of MET. If not, than you need to adjust your paths accordingly.

Perform the following commands ...

Change to your ${HOME}/METplus/parm directory and copy your custom conf file.

NOTE: By placing your custom configuration file under the parm directory, you do not have to specify the path to mycustom.conf in the command when refering to it. The commands in this tutorial are setup that way, so make sure to indeed create this file under the METplus/parm directory.

NOTE: A METplus conf file is not a shell script.
You CAN NOT refer to environment variables as you would in a shell script, or as in the command above ie. ${HOME}.
Instead, in a conf file, you must define as {ENV[HOME]}

Reminder: Make certain to maintain the KEY = VALUE pairs under their respective current [sections] in the conf file.

Below is the exact content of your mycustom.conf file you just copied to your parm directory. It is instructive to REVIEW the settings to understand what changes were REQUIRED in order to run METplus with the default example use cases from a clean download of the software.

Also Note: [config] is not required if you aren't setting any values in this file for that section, but you can add it in case you want to add options in that section later - which you will be doing.

Overriding conf files

This page is meant to be instructive and to reinforce the concept of default configuration variables and overriding conf variables. This page will override your previous set OUTPUT_BASE configuration variable, for your first use case.

The values in the default metplus_system.conf, metplus_data.conf, metplus_runtime.conf, and metplus_logging.conf configuration files can be overridden in the use case conf files and/or a user's custom configuration file. Recall the general use case layout:

• ${HOME}/METplus/parm/use_cases/<use_case>/<use_case>.conf
• ${HOME}/METplus/parm/use_cases/<use_case>/examples/<some_example>.conf

Some variables in the system conf are set to '/path/to' and must be overridden to run METplus, such as OUTPUT_BASE in metplus_system.conf.

Reminder: When adding variables to be overridden, make sure to place the variable under the appropriate section.
For example, [config], [dir], [exe]. If necessary, refer to the default appropriate parm/metplus_config conf files to determine the [section] that corresponds to the variable you are overriding. The value set will be the last one in the sequence of configuration files. See OUTPUT_BASE/metpus_final.conf to see what values were used for a given METplus run.

Example:
View the ${HOME}/METplus/parm/metplus_config/metplus_system.conf file and notice how OUTPUT_BASE = /path/to . This implies it is REQUIRED to be overridden to a valid path.

If you are NOT using a custom conf file, you will need to set the values in metplus_config. We recommend using a custom conf file to avoid conflicts if using a shared METplus installation. It is also beneficial when upgrading to a new version of METplus, as you will not have to scan the metplus_config directory to find all of the changes you have made.

Running METplus

Running METplus involves invoking the python script, master_metplus.py from any directory followed by a list of configuration files using the -c option for each additional conf file (conf files are specified relative to the METplus/parm directory).

Reminder: The default set of conf files are always read in and processed in the following order;
metplus_system.conf, metplus_data.conf, metplus_runtime.conf, metplus_logging.conf.

Example: Run the default configuration

If you have configured METplus correctly and run master_metplus.py with only your custom conf file, it will generate a usage statement to indicate that other config files are required to perform a useful tasks. It will generate an error statement if something is amiss.

Run master_metplus.py on the command line:

THE EXAMPLES BELOW SHOW THE SYNTAX FOR PASSING IN CONF FILES AND ARE NOT MEANT TO BE RUN.
However, feel free to run the command and become familiar with the errors that are generated when referring to a path and/or conf file that does not exist.

Remember: Additional conf files are processed after the metplus_config files in the order specified on the command line.
Order matters, since each successive conf file will override any variable defined in a previous conf file.

NOTE: The processing order allows for structuring your conf files from general (variables shared-by-all) to specific (variables shared-by-few).

Syntax: Running a use_case configuration

Syntax: Runing a use_case/examples configuration

Syntax: Runing a use_case/examples configuration with your CUSTOM conf.

Since PCP-Combine performs a simple operation and reformatting step, no configuration file is needed.

Start by making an output directory for PCP-Combine and changing directories:

Now let's run PCP-Combine twice using some sample data that's included with the MET tarball:

The "\" symbols in the commands above are used for ease of reading. They are line continuation markers enabling us to spread a long command line across multiple lines. They should be followed immediately by "Enter". You may copy and paste the command line OR type in the entire line with or without the "\".

Both commands run the sum command which searches the contents of the -pcpdir directory for the data required to create the requested accmululation interval.

In the first command, PCP-Combine summed up 4 3-hourly accumulation forecast files into a single 12-hour accumulation forecast. In the second command, PCP-Combine summed up 12 1-hourly accumulation observation files into a single 12-hour accumulation observation. PCP-Combine performs these tasks very quickly.

We'll use these PCP-Combine output files as input for Grid-Stat. So make sure that these commands have run successfully!

When PCP-Combine is finished, you may view the output NetCDF files it wrote using the ncdump and ncview utilities. Run the following commands to view contents of the NetCDF files:

The ncview windows display plots of the precipitation data in these files. The output of ncdump indicates that the gridded fields are named APCP_12, the GRIB code abbreviation for accumulated precipitation. The accumulation interval is 12 hours for both the forecast (3-hourly * 4 files = 12 hours) and the observation (1-hourly * 12 files = 12 hours).

Plot-Data-Plane Tool

The Plot-Data-Plane tool can be run to visualize any gridded data that the MET tools can read. It is a very helpful utility for making sure that MET can read data from your file, orient it correctly, and plot it at the correct spot on the earth. When using new gridded data in MET, it's a great idea to run it through Plot-Data-Plane first:

Next try re-running the command list above, but add the convert(x)=x/25.4; function to the config string (Hint: after the level setting but before the last closing tick) to change units from millimeters to inches. What happened to the values in the colorbar?

Now, try re-running again, but add the censor_thresh=lt1.0; censor_val=0.0; options to the config string to reset any data values less 1.0 to a value of 0.0. How has your plot changed?

The convert(x) and censor_thresh/censor_val options can be used in config strings and MET config files to transform your data in simple ways.

We have run examples of the PCP-Combine -sum command, but the tool also supports the -add and -subtract commands. While the -sum command defines a directory to be searched, for -add and -subtract we tell PCP-Combine exactly which files to read and what data to process. The following command adds together 3-hourly precipitation from 4 forecast files, just like we did in the previous step with the -sum command:

By default, PCP-Combine looks for accumulated precipitation, and the 03 tells it to look for 3-hourly accumulations. However, that 03 string can be replaced with a configuration string describing the data to be processed. The configuration string should be enclosed in single quotes. Below, we add together the U and V components of 10-meter wind from the same inpute file. You would not typically want to do this, but this demonstrates the functionality. We also use the -name command line option to define a descriptive output NetCDF variable name:

While the -add command can be run on one or more input files, the -subtract command requires exactly two. Let's rerun the wind example from above but do a subtraction instead:

Now run Plot-Data-Plane to visualize this output. Use the -plot_range option to specify a the desired plotting range, the -title option to add a title, and the -color_table option to switch from the default color table to one that's good for positive and negative values:

Start by making an output directory for Gen-Vx-Mask and changing directories:

Since Gen-Vx-Mask performs a simple masking step, no configuration file is needed.

We'll run the Gen-Vx-Mask tool to apply a polyline for the CONUS (Contiguous United States) to our model domain. Run Gen-Vx-Mask on the command line using the following command:

Re-run using verbosity level 3 and look closely at the log messages. How many grid points were included in this mask?

Gen-Vx-Mask should run very quickly since the grid is coarse (185x129 points) and there are 244 lat/lon points in the CONUS polyline. The more you increase the grid resolution and number of polyline points, the longer it will take to run. View the NetCDF bitmap file generated by executing the following command:

Notice that the bitmap has a value of 1 inside the CONUS polyline and 0 everywhere else. We'll use the CONUS mask we just defined in the next step.

You could trying running plot_data_plane to create a PostScript image of this masking region. Can you remember how?

Notice that there are several ways that gen_vx_mask can be run to define regions of interest, including the use of Shapefiles!

Start by making an output directory for Grid-Stat and changing directories:

The behavior of Grid-Stat is controlled by the contents of the configuration file passed to it on the command line. The default Grid-Stat configuration file may be found in the data/config/GridStatConfig_default file. Prior to modifying the configuration file, users are advised to make a copy of the default:

Open up the GridStatConfig_tutorial file for editing with your preferred text editor.

The configurable items for Grid-Stat are used to specify how the verification is to be performed. The configurable items include specifications for the following:

• The forecast fields to be verified at the specified vertical level or accumulation interval
• The threshold values to be applied
• The areas over which to aggregate statistics - as predefined grids, configurable lat/lon polylines, or gridded data fields
• The confidence interval methods to be used
• The smoothing methods to be applied (as opposed to interpolation methods)
• The types of verification methods to be used

You may find a complete description of the configurable items in section 8.3.2 of the MET User's Guide. Please take some time to review them.

For this tutorial, we'll configure Grid-Stat to verify the 12-hour accumulated precipitation output of PCP-Combine. We'll be using Grid-Stat to verify a single field using NetCDF input for both the forecast and observation files. However, Grid-Stat may in general be used to verify an arbitrary number of fields. Edit the GridStatConfig_tutorial file as follows:

• Set:
  

  fcst = {
     wind_thresh = [ NA ];
  
     field = [
        {
          name       = "APCP_12";
          level      = [ "(*,*)" ];
          cat_thresh = [ >0.0, >=5.0, >=10.0 ];
        }
     ];
  }
  obs = fcst;

  To verify the field of precipitation accumulated over 12 hours using the 3 thresholds specified.

• Set:
  

  mask = {
     grid = [];
     poly = [ "../gen_vx_mask/CONUS_mask.nc",
              "MET_BASE/poly/NWC.poly",
              "MET_BASE/poly/SWC.poly",
              "MET_BASE/poly/GRB.poly",
              "MET_BASE/poly/SWD.poly",
              "MET_BASE/poly/NMT.poly",
              "MET_BASE/poly/SMT.poly",
              "MET_BASE/poly/NPL.poly",
              "MET_BASE/poly/SPL.poly",
              "MET_BASE/poly/MDW.poly",
              "MET_BASE/poly/LMV.poly",
              "MET_BASE/poly/GMC.poly",
              "MET_BASE/poly/APL.poly",
              "MET_BASE/poly/NEC.poly",
              "MET_BASE/poly/SEC.poly" ];
  }

  To accumulate statistics over the Continental United States (CONUS) and the 14 NCEP verification regions in the United States defined by the polylines specified. To see a plot of these regions, execute the following command:
  okular /classroom/wrfhelp/MET/8.0/share/met/poly/ncep_vx_regions.pdf&

• In the boot dictionary, set:
  

  n_rep = 500;

  To turn on the computation of bootstrap confidence intervals using 500 replicates.

• In the nbrhd dictionary, set:
  

  width      = [ 3, 5 ];
  cov_thresh = [ >=0.5, >=0.75 ];

  To define two neighborhood sizes and two fractional coverage field thresholds.

• Set:
  

  output_flag = {
     fho    = NONE;
     ctc    = BOTH;
     cts    = BOTH;
     mctc   = NONE;
     mcts   = NONE;
     cnt    = BOTH;
     sl1l2  = BOTH;
     sal1l2 = NONE;
     vl1l2  = NONE;
     val1l2 = NONE;
     pct    = NONE;
     pstd   = NONE;
     pjc    = NONE;
     prc    = NONE;
     eclv   = NONE;
     nbrctc = BOTH;
     nbrcts = BOTH;
     nbrcnt = BOTH;
     grad   = BOTH;
  }

  To compute contingency table counts (CTC), contingency table statistics (CTS), continuous statistics (CNT), scalar partial sums (SL1L2), neighborhood contingency table counts (NBRCTC), neighborhood contingency table statistics (NBRCTS), and neighborhood continuous statistics (NBRCNT).

Next, run Grid-Stat on the command line using the following command:

Grid-Stat is now performing the verification tasks we requested in the configuration file. It should take a minute or two to run. The status messages written to the screen indicate progress.

In this example, Grid-Stat performs several verification tasks in evaluating the 12-hour accumulated precipiation field:

• For continuous statistics and partial sums (CNT and SL1L2), 15 output lines each:
  (1 field * 15 masking regions)
• For contingency table counts and statistics (CTC and CTS), 45 output lines each:
  (1 field * 3 raw thresholds * 15 masking regions)
• For neighborhood methods (NBRCNT, NBRCTC, and NBRCTS), 90 output lines each:
  (1 field * 3 raw thresholds * 2 neighborhood sizes * 15 masking regions)

To greatly increase the runtime performance of Grid-Stat, you could disable the computation of bootstrap confidence intervals in the configuration file. Edit the tutorial/config/GridStatConfig_tutorial file as follows:

• In the boot dictionary, set:
  

  n_rep = 0;

  To disable the computation of bootstrap confidence intervals.

Now, try rerunning the Grid-Stat command listed above and notice how much faster it runs. While bootstrap confidence intervals are nice to have, they take a long time to compute, especially for gridded data.

The output of Grid-Stat is one or more ASCII files containing statistics summarizing the verification performed and a NetCDF file containing difference fields. In this example, the output is written to the current directory, as we requested on the command line. It should now contain 10 Grid-Stat output files beginning with the grid_stat_ prefix, one each for the CTC, CTS, CNT, SL1L2, GRAD, NBRCTC, NBRCTS, and NBRCNT ASCII files, a STAT file, and a NetCDF matched pairs file.

The format of the CTC, CTS, CNT, and SL1L2 ASCII files will be covered for the Point-Stat tool. The neighborhood method and gradient output are unique to the Grid-Stat tool.

• Rather than comparing forecast/observation values at individual grid points, the neighborhood method compares areas of forecast values to areas of observation values. At each grid box, a fractional coverage value is computed for each field as the number of grid points within the neighborhood (centered on the current grid point) that exceed the specified raw threshold value. The forecast/observation fractional coverage values are then compared rather than the raw values themselves.
• Gradient statistics are computed on the forecast and observation gradients in the X-direction.

Since the lines of data in these ASCII files are so long, we strongly recommend configuring your text editor to NOT use dynamic word wrapping. The files will be much easier to read that way.

Execute the following command to view the NetCDF output of Grid-Stat:

Click through the 2d vars variable names in the ncview window to see plots of the forecast, observation, and difference fields for each masking region. If you see a warning message about the min/max values being zero, just click OK.

Now dump the NetCDF header:

View the NetCDF header to see how the variable names are defined.

Notice how *MANY* variables there are, separate output for each of the masking regions defined. Try editing the config file again by setting apply_mask = FALSE; and gradient = TRUE; in the nc_pairs_flag dictionary. Re-run Grid-Stat and inspect the output NetCDF file. What affect did these changes have?

We have now successfully run the PCP-Combine and Grid-Stat tools to verify 12-hourly accumulated preciptation for a single output time. We did the following steps:

• Identified our forecast and observation datasets.
• Constructed PCP-Combine commands to put them into a common accumulation interval.
• Configured and ran Grid-Stat to compute our desired verification statistics.

Now that we've defined the logic for a single run, the next step would be writing a script to automate these steps for many model initializations and forecast lead times. Rather than every MET user rewriting the same type of scripts, use METplus to automate these steps in a use case!

Answers to Exercises from Session 1

These are the answers to the exercises from the previous page. Feel free to ask a MET representative if you have any questions!

ANSWER 1.1: add_rh - Add another field to grid_stat

Instructions: Modify the METplus configuration files to add relative humidity (RH) at pressure levels 500 and 250 (P500 and P250) to the output.