Prequisites: Software

The following is a full list of software that is required to utilize all the functionality in METplus.
NOTE: We will not by running the cyclone plotter or grid plotting scripts in this tutorial. So the additional libraries mentioned below are not installed.

• Python 2.7.x
• Additional Python libraries: - required by the cyclone plotter and grid plotting scripts.
  • numpy
  • matplotlib - not installed, Is is not required for this tutorial.
  • cartopy - not installed, It is not required for this tutorial.
• MET 8.1 - already installed locally on the tutorial machines.
• R - used by TCMPRPlotter wrapper, Wraps the MET plot_tcmpr.R script
• ncap2 - NCO netCDF Operators
• wgrib2 - only needed for series analysis wrappers (SeriesByLead and SeriesByInit)
• ncdump - NetCDF binaries
• convert - Utilitity from ImageMagick software suite

Prequisites: Environment

Before running these instructions, you will need to ensure that you have a few environment variables set up correctly. If they are not set correctly, these instructions will not work properly.

1. Check that you have METPLUS_TUTORIAL_DIR set correctly:

If you don't see a path in your user directory output to the screen, set this environment variable in your user profile before continuing.

2. Check that you have METPLUS_BUILD_BASE, MET_BUILD_BASE, and METPLUS_DATA set correctly:

If any of these variables are not set, please set them. They will be referenced throughout the tutorial.

3. Check that you have loaded the MET module correctly:

You should see the usage statement for Point-Stat. The version number listed should correspond to the version listed in MET_BUILD_BASE. If it does not, you will need to either reload the met module, or add ${MET_BUILD_BASE}/bin to your PATH.

Getting The Software

For this tutorial we'll use the METplus v2.2 release, which has been installed in a common location. We will configure METplus to run the met-8.1 software, which has also been installed in a common location. We will copy the parameter files from the shared METplus location to your own directory so you can modify them without changing the shared installation settings.

Please use the following instructions to setup METplus on your tutorial machine:

1. Create a directory to store your METplus files (include the version number for traceability):

2. Create a symbolic link so that you can refer to the METplus directory instead of METplus-2.2 (you may need to remove the link if it already exists):

3. List the contents of the metplus_tutorial directory:

You should see something similar to this:

4. Copy the parm directory from the shared METplus location to your home directory:

5. Create directory to store configuration files you will create:

For reference, the following are instructions to download and install METplus on your own machine.

• If you wish, download and unpack the release tarball and create a METplus symbolic link:

DO NOT RUN THESE COMMANDS FOR THE TUTORIAL!

Sample Data Input and Location of Output

The Sample Input Data used to run the examples in this tutorial already exist in a shared location. For reference, here are example instructions to obtain the sample input data.

DO NOT RUN THESE COMMANDS FOR THE TUTORIAL!

This will create a directory named 'grid_to_grid' under '/d1/METplus_Data' that contains all of the sample input data needed to run the grid_to_grid use cases (found in parm/use_cases/grid_to_grid/examples). Other sample data tarballs are available under 'Assets' on the METplus GitHub releases page.

When running METplus wrapper use cases, all output is placed in the ${METPLUS_TUTORIAL_DIR}/output directory.

When running MET on the command line, all output will be written to ${METPLUS_TUTORIAL_DIR}/output/met_output

Your Runtime Environment

For this tutorial, the MET software is available via a module file. This has already been loaded for all users, so the MET software is available when you login.

The PATH environment variable is set to allow the METplus scripts to be run from any directory.

1. Edit the .bash_profile file to configure your runtime environment for use with METplus:

Add the following at the end of the file to set the PATH and METPLUS_PARM_BASE environment variables. Also update LD_LIBRARY_PATH to avoid runtime NetCDF linker errors:

bash:

csh:

Save and close that file, then source it to apply the settings to the current shell. The next time you login, you do not have to source this file again:

If you did everything correctly, running 'which master_metplus.py' should show you the path to the script in the shared location, ${METPLUS_BUILD_BASE}:

METplus directory structure

Brief description and overview of the METplus/ directory structure.

• doc/ - Doxygen files for building the html API documentation.
• doc/METplus_Users_Guide - LyX files and the user guide PDF file.
• internal_tests/ - for engineering tests
• parm/ - where config files live
• README.md - general README
• sorc/ - executables and documentation build system
• ush/ - python scripts

METplus default configuration files

The METplus default configuration files metplus_system.conf, metplus_data.conf, metplus_runtime.conf, and metplus_logging.conf are always read by default and in the order shown. Any additional configuration files passed in on the command line are than processed in the order in which they are specified. This allows for each successive conf file the ability to override variables defined in any previously processed conf files. It also allows for defining and setting up conf files from a general (settings used by all use cases, ie. MET install dir) to more specific (Plot type when running track and intensity plotter) structure. The idea is to created a hiearchy of conf files that is easier to maintain, read, and manage. It is important to note, running METplus creates a single configuration file, which can be viewed to understand the result of all the conf file processing.

The final metplus conf file is defined here:
metplus_runtime.conf:METPLUS_CONF={OUTPUT_BASE}/metplus_final.conf
Use this file to see the result of all the conf file processing, this can be very helpful when troubleshooting,

NOTE: The syntax for METplus configuration files MUST include a "[section]" header with the variable names and values on subsequent lines.

Unless otherwise indicated, all directories are relative to your ${METPLUS_PARM_BASE} directory.

The metplus_config directory - there are four config files:

1. metplus_system.conf
   • contains "[dir]" and "[exe]" to set directory and executable paths
   • any information specific to host machine
2. metplus_data.conf
   • Sample data or Model input location
   • filename templates and regex for filenames
3. metplus_runtime.conf
   • contains "[config]" section
   • var lists, stat lists, configurations, process list
   • anything else needed at runtime
4. metplus_logging.conf
   • contains "[dir]" section for setting the location of log files.
   • contains "[config]" section for setting various logging configuration options.

The use_cases directory - The use cases:
This is where the use cases you will be running exist. The use case filenaming and directory structure is by convention. It is intended to simplify management of common configuration items for a specific use case. Under the use_cases directory is a subdirectory for each use case. Under that is an examples subdirectory and a more general configuration file that contains information that is shared by examples of the use case.

• For example, the track and intensity use case conf directory and file structure.
• track_and_intensity - directory
• track_and_intensity/track_and_intensity.conf - use case configuration file
• track_and_intensity/examples - directory to hold various track and intensity example conf files.
• track_and_intensity/examples/tcmpr_mean_median.conf - specific use case
• track_and_intensity/examples/README - File containing instructions to run the use case examples in this directory.

Within each use case there is a met_config subdirectory that contains MET specific config files for that use case. The use_cases/<use case>/met_config directory - store all your MET config files, e.g.:

• GridStatConfig_anom (for MET grid_stat)
• TCPairsETCConfig (for MET tc_pairs)
• SeriesAnalysisConfig_by_init (for MET series_analysis)
• SeriesAnalysisConfig_by_lead (for MET series_analysis)

Modify your METplus conf files

In this section you will modify the configuration files that will be read for each call to METplus.

The paths in this practical assume:

• You have copied the parm directory from the shared METplus location into your ${METPLUS_TUTORIAL_DIR}/METplus directory
• You have added the shared METplus ush directory to your PATH
• You have set the METPLUS_PARM_BASE environment variable to your copied parm directory
• You are using the shared installation of MET.

If not, then you need to adjust accordingly.

1. Try running master_metplus.py. You should see the usage statement output to the screen.

2. Now try to pass in the example.conf configuration file found in your parm directory under use_cases/wrappers/examples

You should see an error message stating that OUTPUT_BASE was not set correctly. You will need to configure the METplus wrappers to be able to run a use case.

The values in the default metplus_system.conf, metplus_data.conf, metplus_runtime.conf, and metplus_logging.conf configuration files are read in first when you run master_metplus.py. The settings in these files can be overridden in the use case conf files and/or a user's custom configuration file.

Some variables in the system conf are set to '/path/to' and must be overridden to run METplus, such as OUTPUT_BASE in metplus_system.conf.

3. View the metplus_system.conf file and notice how OUTPUT_BASE = /path/to . This implies it is REQUIRED to be overridden to a valid path.

Note: The default installation of METplus has /path/to values for MET_INSTALL_DIR, TMP_DIR, and INPUT_BASE. These values were set in the shared METplus configuration when it was installed. This was done because these settings will likely be set to the same values for all users.

4. Change directory to your ${METPLUS_PARM_BASE} directory and modify the METplus base configuration files using the editor of your choice (vi listed as an example).

If METPLUS_PARM_BASE is set correctly, you should now be in ${METPLUS_TUTORIAL_DIR}/METplus/parm/metplus_config

Note: A METplus conf file is not a shell script.
You CAN NOT refer to environment variables as you would in a shell script or command prompt, i.e. ${HOME}.
Instead, you must reference the environment variable $HOME as {ENV[HOME]}

Reminder: Make certain to maintain the KEY = VALUE pairs under their respective current [sections] in the conf file.

5. Set the [dir] INPUT_BASE variable to the location of the METplus sample data that will be used in the exercises

6. Set the [dir] OUTPUT_BASE variable to the location you will write output data
   Set the [dir] MET_INSTALL_DIR variable to the location where MET is installed (this should already be done when METplus was installed in the shared location)
   Set the [dir] TMP_DIR variable to a location you have write permissions

NOTE: When installing METplus, you will need to set the full path to the non-MET executables in the metplus_system.conf file if they are not found in the user's path. This step was completed when METplus was installed in the shared location.

Creating user conf files

You can create additional configuration files to be read by the METplus wrappers to override variables.

Reminder: When adding variables to be overridden, make sure to place the variable under the appropriate section.
For example, [config], [dir], [exe]. If necessary, refer to the default appropriate parm/metplus_config conf files to determine the [section] that corresponds to the variable you are overriding. The value set will be the last one in the sequence of configuration files. See OUTPUT_BASE/metpus_final.conf to see what values were used for a given METplus run.

1. Create a new configuration file in your user_config directory and override the [dir] OUTPUT_BASE variable to point to a different location.

Copy and paste the following text into the file and save it.

We will test out using these configurations on the next page.

Running METplus

Running METplus involves invoking the python script master_metplus.py followed by a list of configuration files using the -c option for each additional conf file.

Reminder: The default set of conf files are always read in and processed in the following order;
metplus_system.conf, metplus_data.conf, metplus_runtime.conf, metplus_logging.conf.

If you have configured METplus correctly and run master_metplus.py without passing in any configuration files, it will generate a usage statement to indicate that other config files are required to perform a useful task. It will generate an error statement if something is amiss.

1. Review the example.conf configuration file

2. Call the master_metplus.py script, passing in the example.conf configuration file with the -c command line option. You should see logs output to the screen.

Note: If master_metplus.py can't be found, you may need to add it to the path. Optionally, you can specify the full path to the script.

Note: The environment variable METPLUS_PARM_BASE determines where to look for relative paths to configuration files. If the environment variable is not set, the script will look in the parm directory that is associated with the version of master_metplus.py that is being called. Optionally, you can specify the full path to the configuration files.

OPTIONAL:

3. Check the directory specified by the OUTPUT_BASE configuration variable. You should see that files and sub-directories have been created

4. Review the master log file to see what was run. Compare the log output to the example.conf configuration file to see how they correspond to each other. The log file will have today's date in the filename

You will notice that METplus ran for 4 valid times, processing 4 forecast hours for each valid time. For each run, it used the input template to determine which file to look for.

5. Now run METplus passing in the example.conf from the previous run AND your newly created configuration file.

6. Check the directory specified by the OUTPUT_BASE configuration variable that you set in the change_output_base.conf configuration file. You should see that files and sub-directories have been created in the new location.

Remember: Additional conf files are processed after the metplus_config files in the order specified on the command line.
Order matters, since each successive conf file will override any variable defined in a previous conf file.

Note: The processing order allows for structuring your conf files from general (variables shared-by-all) to specific (variables shared-by-few).

Since PCP-Combine performs a simple operation and reformatting step, no configuration file is needed.

1. Start by making an output directory for PCP-Combine and changing directories:

2. Now let's run PCP-Combine twice using some sample data that's included with the MET tarball:

The "\" symbols in the commands above are used for ease of reading. They are line continuation markers enabling us to spread a long command line across multiple lines. They should be followed immediately by "Enter". You may copy and paste the command line OR type in the entire line with or without the "\".

Both commands run the sum command which searches the contents of the -pcpdir directory for the data required to create the requested accmululation interval.

In the first command, PCP-Combine summed up 4 3-hourly accumulation forecast files into a single 12-hour accumulation forecast. In the second command, PCP-Combine summed up 12 1-hourly accumulation observation files into a single 12-hour accumulation observation. PCP-Combine performs these tasks very quickly.

We'll use these PCP-Combine output files as input for Grid-Stat. So make sure that these commands have run successfully!

When PCP-Combine is finished, you may view the output NetCDF files it wrote using the ncdump and ncview utilities. Run the following commands to view contents of the NetCDF files:

The ncview windows display plots of the precipitation data in these files. The output of ncdump indicates that the gridded fields are named APCP_12, the GRIB code abbreviation for accumulated precipitation. The accumulation interval is 12 hours for both the forecast (3-hourly * 4 files = 12 hours) and the observation (1-hourly * 12 files = 12 hours).

Plot-Data-Plane Tool

The Plot-Data-Plane tool can be run to visualize any gridded data that the MET tools can read. It is a very helpful utility for making sure that MET can read data from your file, orient it correctly, and plot it at the correct spot on the earth. When using new gridded data in MET, it's a great idea to run it through Plot-Data-Plane first:

Next try re-running the command list above, but add the convert(x)=x/25.4; function to the config string (Hint: after the level setting but before the last closing tick) to change units from millimeters to inches. What happened to the values in the colorbar?

Now, try re-running again, but add the censor_thresh=lt1.0; censor_val=0.0; options to the config string to reset any data values less 1.0 to a value of 0.0. How has your plot changed?

The convert(x) and censor_thresh/censor_val options can be used in config strings and MET config files to transform your data in simple ways.

We have run examples of the PCP-Combine -sum command, but the tool also supports the -add, -subtract, and -derive commands. While the -sum command defines a directory to be searched, for -add, -subtract, and -derive we tell PCP-Combine exactly which files to read and what data to process. The following command adds together 3-hourly precipitation from 4 forecast files, just like we did in the previous step with the -sum command:

By default, PCP-Combine looks for accumulated precipitation, and the 03 tells it to look for 3-hourly accumulations. However, that 03 string can be replaced with a configuration string describing the data to be processed. The configuration string should be enclosed in single quotes. Below, we add together the U and V components of 10-meter wind from the same input file. You would not typically want to do this, but this demonstrates the functionality. We also use the -name command line option to define a descriptive output NetCDF variable name:

While the -add command can be run on one or more input files, the -subtract command requires exactly two. Let's rerun the wind example from above but do a subtraction instead:

Now run Plot-Data-Plane to visualize this output. Use the -plot_range option to specify a the desired plotting range, the -title option to add a title, and the -color_table option to switch from the default color table to one that's good for positive and negative values:

Now view the results:

While the PCP-Combine -add and -subtract commands compute exactly one output field of data, the -derive command can compute multiple output fields in a single run. This command reads data from one or more input files and derives the output fields requested on the command line (sum, min, max, range, mean, stdev, vld_count).

Run the following command to derive several summary metrics for both the 10-meter U and V wind components:

In the above example, we used a wildcard to list multiple input file names.  And we used the -field command line option twice to specify two input fields.  For each input field, PCP-Combine loops over the input files, derives the requested metrics, and writes them to the output NetCDF file.  Run ncview to visualize this output:

This output file contains 8 variables: 2 input fields * 4 metrics. Note the output variable names the tool chose.  You can still override those names using the -name command line argument, but you would have to specify a comma-separated list of 8 names, one for each output variable.

Start by making an output directory for Gen-Vx-Mask and changing directories:

Since Gen-Vx-Mask performs a simple masking step, no configuration file is needed.

We'll run the Gen-Vx-Mask tool to apply a polyline for the CONUS (Contiguous United States) to our model domain. Run Gen-Vx-Mask on the command line using the following command:

Re-run using verbosity level 3 and look closely at the log messages. How many grid points were included in this mask?

Gen-Vx-Mask should run very quickly since the grid is coarse (185x129 points) and there are 244 lat/lon points in the CONUS polyline. The more you increase the grid resolution and number of polyline points, the longer it will take to run. View the NetCDF bitmap file generated by executing the following command:

Notice that the bitmap has a value of 1 inside the CONUS polyline and 0 everywhere else. We'll use the CONUS mask we just defined in the next step.

You could trying running plot_data_plane to create a PostScript image of this masking region. Can you remember how?

Notice that there are several ways that gen_vx_mask can be run to define regions of interest, including the use of Shapefiles!

Start by making an output directory for Grid-Stat and changing directories:

The behavior of Grid-Stat is controlled by the contents of the configuration file passed to it on the command line. The default Grid-Stat configuration file may be found in the data/config/GridStatConfig_default file. Prior to modifying the configuration file, users are advised to make a copy of the default:

Open up the GridStatConfig_tutorial file for editing with your preferred text editor.

The configurable items for Grid-Stat are used to specify how the verification is to be performed. The configurable items include specifications for the following:

• The forecast fields to be verified at the specified vertical level or accumulation interval
• The threshold values to be applied
• The areas over which to aggregate statistics - as predefined grids, configurable lat/lon polylines, or gridded data fields
• The confidence interval methods to be used
• The smoothing methods to be applied (as opposed to interpolation methods)
• The types of verification methods to be used

You may find a complete description of the configurable items in section 8.3.2 of the MET User's Guide. Please take some time to review them.

For this tutorial, we'll configure Grid-Stat to verify the 12-hour accumulated precipitation output of PCP-Combine. We'll be using Grid-Stat to verify a single field using NetCDF input for both the forecast and observation files. However, Grid-Stat may in general be used to verify an arbitrary number of fields. Edit the GridStatConfig_tutorial file as follows:

• Set:
  
  fcst = {
     wind_thresh = [ NA ];  
     field = [
        {
          name       = "APCP_12";
          level      = [ "(*,*)" ];
          cat_thresh = [ >0.0, >=5.0, >=10.0 ];
        }
     ];
  }
  obs = fcst;

  To verify the field of precipitation accumulated over 12 hours using the 3 thresholds specified.

• Set:
  
  mask = {
     grid = [];
     poly = [ "../gen_vx_mask/CONUS_mask.nc",
              "MET_BASE/poly/NWC.poly",
              "MET_BASE/poly/SWC.poly",
              "MET_BASE/poly/GRB.poly",
              "MET_BASE/poly/SWD.poly",
              "MET_BASE/poly/NMT.poly",
              "MET_BASE/poly/SMT.poly",
              "MET_BASE/poly/NPL.poly",
              "MET_BASE/poly/SPL.poly",
              "MET_BASE/poly/MDW.poly",
              "MET_BASE/poly/LMV.poly",
              "MET_BASE/poly/GMC.poly",
              "MET_BASE/poly/APL.poly",
              "MET_BASE/poly/NEC.poly",
              "MET_BASE/poly/SEC.poly" ];
  }

  To accumulate statistics over the Continental United States (CONUS) and the 14 NCEP verification regions in the United States defined by the polylines specified. To see a plot of these regions, execute the following command:

  gv ${MET_BUILD_BASE}/data/poly/ncep_vx_regions.pdf &

• In the boot dictionary, set:
  
  n_rep = 500;

  To turn on the computation of bootstrap confidence intervals using 500 replicates.

• In the nbrhd dictionary, set:
  
  width          = [ 3, 5 ];
  cov_thresh = [ >=0.5, >=0.75 ];

  To define two neighborhood sizes and two fractional coverage field thresholds.

• Set:
  
  output_flag = {
     fho     = NONE;
     ctc      = BOTH;
     cts      = BOTH;
     mctc   = NONE;
     mcts   = NONE;
     cnt     = BOTH;
     sl1l2   = BOTH;
     sal1l2 = NONE;
     vl1l2   = NONE;
     val1l2 = NONE;
     pct     = NONE;
     pstd   = NONE;
     pjc     = NONE;
     prc     = NONE;
     eclv    = NONE;
     nbrctc = BOTH;
     nbrcts = BOTH;
     nbrcnt = BOTH;
     grad    = BOTH;
  }

  To compute contingency table counts (CTC), contingency table statistics (CTS), continuous statistics (CNT), scalar partial sums (SL1L2), neighborhood contingency table counts (NBRCTC), neighborhood contingency table statistics (NBRCTS), and neighborhood continuous statistics (NBRCNT).

Next, run Grid-Stat on the command line using the following command:

Grid-Stat is now performing the verification tasks we requested in the configuration file. It should take a minute or two to run. The status messages written to the screen indicate progress.

In this example, Grid-Stat performs several verification tasks in evaluating the 12-hour accumulated precipiation field:

• For continuous statistics and partial sums (CNT and SL1L2), 15 output lines each:
  (1 field * 15 masking regions)
• For contingency table counts and statistics (CTC and CTS), 45 output lines each:
  (1 field * 3 raw thresholds * 15 masking regions)
• For neighborhood methods (NBRCNT, NBRCTC, and NBRCTS), 90 output lines each:
  (1 field * 3 raw thresholds * 2 neighborhood sizes * 15 masking regions)

To greatly increase the runtime performance of Grid-Stat, you could disable the computation of bootstrap confidence intervals in the configuration file. Edit the GridStatConfig_tutorial file as follows:

• In the boot dictionary, set:
  
  n_rep = 0;

  To disable the computation of bootstrap confidence intervals.

Now, try rerunning the Grid-Stat command listed above and notice how much faster it runs. While bootstrap confidence intervals are nice to have, they take a long time to compute, especially for gridded data.

The output of Grid-Stat is one or more ASCII files containing statistics summarizing the verification performed and a NetCDF file containing difference fields. In this example, the output is written to the current directory, as we requested on the command line. It should now contain 10 Grid-Stat output files beginning with the grid_stat_ prefix, one each for the CTC, CTS, CNT, SL1L2, GRAD, NBRCTC, NBRCTS, and NBRCNT ASCII files, a STAT file, and a NetCDF matched pairs file.

The format of the CTC, CTS, CNT, and SL1L2 ASCII files will be covered for the Point-Stat tool. The neighborhood method and gradient output are unique to the Grid-Stat tool.

• Rather than comparing forecast/observation values at individual grid points, the neighborhood method compares areas of forecast values to areas of observation values. At each grid box, a fractional coverage value is computed for each field as the number of grid points within the neighborhood (centered on the current grid point) that exceed the specified raw threshold value. The forecast/observation fractional coverage values are then compared rather than the raw values themselves.
• Gradient statistics are computed on the forecast and observation gradients in the X and Y directions.

Since the lines of data in these ASCII files are so long, we strongly recommend configuring your text editor to NOT use dynamic word wrapping. The files will be much easier to read that way.

Execute the following command to view the NetCDF output of Grid-Stat:

Click through the 2d vars variable names in the ncview window to see plots of the forecast, observation, and difference fields for each masking region. If you see a warning message about the min/max values being zero, just click OK.

Now dump the NetCDF header:

View the NetCDF header to see how the variable names are defined.

Notice how *MANY* variables there are, separate output for each of the masking regions defined. Try editing the config file again by setting apply_mask = FALSE; and gradient = TRUE; in the nc_pairs_flag dictionary. Re-run Grid-Stat and inspect the output NetCDF file. What affect did these changes have?

We have now successfully run the PCP-Combine and Grid-Stat tools to verify 12-hourly accumulated preciptation for a single output time. We did the following steps:

• Identified our forecast and observation datasets.
• Constructed PCP-Combine commands to put them into a common accumulation interval.
• Configured and ran Grid-Stat to compute our desired verification statistics.

Now that we've defined the logic for a single run, the next step would be writing a script to automate these steps for many model initializations and forecast lead times. Rather than every MET user rewriting the same type of scripts, use METplus to automate these steps in a use case!

Answers to Exercises from Session 1

These are the answers to the exercises from the previous page. Feel free to ask a MET representative if you have any questions!

ANSWER 1.1: add_rh - Add another field to grid_stat

Instructions: Modify the METplus configuration files to add relative humidity (RH) at pressure levels 500 and 250 (P500 and P250) to the output.

Answer: In the g2g.add_rh.conf param file, add the following variables to the [config] section.