MET Version 6.1

Release Date:

The MET Version 6.1 was released on December 4, 2017.

Release Notes

What's New?

  • Major Enhancements: generalized point observations, unit conversion, data censoring, economic cost/loss value, gradients, fourier decomposition, relative position, and binned climatologies
  • Please see slides describing recent changes: METv6.1_updates.pdf

Enhancements to Existing Tools:

  • Distribute the latest set of bugfixes, details not listed here.
  • Update "-version" command line option to include patch level info.
  • Grid library code
    • Add support for Gaussian grids.
    • Add support for Polar Stereographic CF-Compliant NetCDF files.
    • Re-orient NetCDF lat/lon grid.
    • Support northern and southern hemisphere Lambert Conformal grids.
  • Regridding
    • Add "shape" config option to the "regrid" and "interp" dictionaries to define a "square" (legacy) or "circle" area.
    • Automatically regrid masking regions to verification domain on the fly using nearest-neighbor interpolation.
  • Config file options
    • Add "GRIB2_stat_type" config file filtering option.
    • Fix "GRIB_lvl_typ" config file filtering option for GRIB2 data.
    • Add "convert(x)" config file option to define unit conversion functions.
      • For example, convert(x) = C_to_F(x);
    • Add "censor_thresh" and "censor_val" config file options to apply a filtering thresholds and replacement values.
      • For example, censor_thresh = [ <0 ]; censor_val = [ 0 ];
  • MODE
    • Remove "zero_border_size" config file option.
    • Remove "raw_thresh" config file option which is replaced by censoring.
    • Remove "AREA_FILTER" column from MODE output _obj.txt files.
    • Remove "FILTER" line from MODE output _cts.txt files.
    • Add output columns for "N_VALID", "GRID_RES", and "OBTYPE".
    • Speed up convex hull detection algorithm.
  • PB2NC
    • Add support for reading BUFR files directly, rather than just PREPBUFR.
    • Replace "obs_grib_code" config file option with "obs_bufr_var".
    • Add "obs_bufr_map", "obs_prepbufr_map", "message_type_map", and "time_summary" config file options.
    • Add "-index" command line option to list all available observations.
  • ASCII2NC
    • Enable input column 7 to contain an integer GRIB code (legacy) or string variable name.
  • Point-Stat and Ensemble-Stat
    • Generalize the specification of point observations using the variable name rather than mapping to a GRIB code.
  • Plot-Point-Obs
    • Add "-obs_var" command line option.
  • Grid-Stat and Point-Stat
    • Add Root-Mean Squared Forecast (RMSFA) and Observation (RMSOA) Anomalies to the CNT line type.
    • Add new Economic Cost-Loss Value (ECLV) STAT line type derived from CTC or PCT lines.
    • Add "eclv_points" config file option.
    • Add initial support for binned climatologies by adding "climo_stdev" and "climo_cdf_bins" config file options.
    • When computing BSS, use the "climo_mean" and "climo_stdev" fields to derive the climatological probability values to be verified.
  • Grid-Stat
    • Add new Gradient (GRAD) STAT line type for statistics derived from gradients, including the S1 score.
    • Add option to compute 1-dimensional Fourier Decompositions when generating CNT, SL1L2, SAL1L2, VL1L2, and VAL1L2 output lines.
    • Add "fourier" config file dictionary to specify wave numbers.
    • Append fourier wave number information to the "INTERP_MTHD" column.
    • Enhance "nc_pairs_flag" config file option to write derived "fourier" and "gradient" fields.
  • Ensemble-Stat
    • Add "skip_const" config file option to exclude points where all ensemble members and the observation value are identical.
    • Add "obs_thresh" config file option to only include points where the observation value meets the filtering threshold.
    • Add "BSS_SMPL" statistic to the PSTD line type.
    • Add new Relative Position (RELP) STAT line type.
  • Stat-Analysis
    • Enhance "-column_thresh" job command option to compute column difference with or without absolute values.
    • Add "RANGE" and "IQR" (Interquartile Range) columns for the summary job.
    • Allow for NA values when filtering data.
      • For example, -column_thresh CLIMO != NA
    • Add support for specifying derived columns, "INIT_HOUR" and "VALID_HOUR".
      • For example, -by INIT_HOUR
    • Add support for aggregating new ECLV, GRAD, and RELP line types.
    • Add support for reading CTC or PCT and deriving ECLV.
    • Add support for reading ORANK and deriving RELP.


Known Issues and Fixes

All Recommended Updates (Last Updated 04/03/2018)

Retrieve all of the recommended updates for METv6.1 in a tarfile:

  • Download the tarfile met-6.1_patches_20180403.tar.gz
  • Copy it into the top-level met-6.1 directory.
  • Uncompress and untar the file. Please note that the new version of the files in the tarball will overwrite the original version of those files. Any changes you may have made will be lost.
  • Rebuild MET, being sure to run make clean and configure first.

Summary of Recommended Updates

  • 04/03/2018: Fix ORANK header columns.
  • 02/14/2018: Fix plot_point_obs bug when the number of header locations exceeds 131072.
  • 02/14/2018: Fix ascii2nc bug when the number of header locations exceeds 131072.
  • 02/02/2018: Fix ensemble SSVAR output for gridded observations.
  • 02/02/2018: Fix error writing NetCDF output with empty unit string.
  • 02/02/2018: Update ascii2nc usage statement.
  • 02/02/2018: Update ascii2nc to interpret -9999 and NA as missing data.

Details

Fix ORANK header columns.
Posted 04/03/2018

Problem: The last two header columns of the ORANK output line type are incorrectly listed as "ENS_SPREAD ENS_MEAN" when they should be "CLIMO ENS_SPREAD".
Solution: Correct the header columns.
Update: met-6.1/src/libcode/vx_stat_out/stat_columns.h

Fix plot_point_obs bug when the number of header locations exceeds 131072.
Posted 02/14/2018

Problem: Attempting to plot a NetCDF point observation file containing more than 131072 (128*1024) header locations results in a segmentation fault.
Solution: Update the logic to correctly handle the buffer.
Update: met-6.1/src/tools/other/plot_point_obs/plot_point_obs.cc

Fix ascii2nc bug when the number of header locations exceeds 131072.
Posted 02/14/2018

Problem: The ascii2nc tool uses a buffer of size 131072 (128*1024). When the number of header locations exceeds this number, ascii2nc reuses that buffer but fails write its contents the last time through.
Solution: Update the logic to correctly flush the buffer.
Update: met-6.1/src/tools/other/ascii2nc/file_handler.cc

Fix ensemble SSVAR output for gridded observations.
Posted 02/02/2018

Problem: Major bug in Ensemble-Stat in the computation of SSVAR output using gridded observations. The ensemble mean data was not subset correctly for each verification mask causing incorrect variance and error computations. Existing SSVAR output which includes this bug should not be used. However, SSVAR output computed using point observations is correct.
Solution: The fix is correctly subsetting the ensemble mean data when processing gridded observations. Also, add error checks to the SSVAR computation to ensure all data arrays are of the same length.
Update: met-6.1/src/tools/core/ensemble_stat/ensemble_stat.cc
Update: met-6.1/src/libcode/vx_statistics/pair_data_ensemble.cc

Fix error writing NetCDF output with empty unit string.
Posted 02/02/2018

Problem: When creating NetCDF output files, the MET tools write the units as a variable attribute. However an empty units string causes the NetCDF library to abort.
Solution: The units for some variables in GRIB1 and GRIB2 tables are listed as empty strings. Update the MET library code to replace empty unit strings with NA to avoid this runtime error.
Update: met-6.1/src/libcode/vx_data2d/table_lookup.cc

Update ascii2nc usage statement.
Posted 02/02/2018

Problem: The ascii2nc usage statement fails to indicate that in met-6.1, column 7 of the 11-column point observation format may be set to a variable name string or an integer GRIB code, the previous behavior.
Solution: Update the usage statement.
Update: met-6.1/src/tools/other/ascii2nc/ascii2nc.cc

Update ascii2nc to interpret -9999 and NA as missing data.
Posted 02/02/2018

Problem: When processing the 11-column point observation format, ascii2nc fails to interpret NA as missing data. Instead, NA, and all other strings, are interpreted as a floating point value of 0.
Solution: Update the parsing logic to check for the NA string and correctly assign it the missing data value of -9999.
Update: met-6.1/src/tools/other/ascii2nc/met_handler.cc