MET Users Page

Release Notes for METv8.0

What's New?

  • Major Enhancements: python embedding, observation error logic, shapefiles for masking regions, point observation format overhaul, and rotated lat/lon projections
  • New Tools: Added three new utilities to dump shapefile information (gis_dump_dbf, gis_dump_shp, and gis_dump_shx)
  • Please see slides describing recent changes: met-8.0_updates.pdf

Changes to Output Formats:

  • Added new Ensemble Continuous Statistics line type (ECNT) for Ensemble-Stat and Point-Stat HiRA.
  • Removed CRPS, IGN, CRPSS, and SPREAD columns from the RHIST line type and move them to the new ECNT line type.
  • Added ENS_MEAN_OERR, SPREAD_OERR, SPREAD_PLUS_OERR columns to the ORANK line type.
  • Added ASPECT_DIFF and CURVATURE_RATIO columns to the MODE output.

Enhancements to Existing Tools:

  • Distributed the latest set of bugfixes (details not listed here).
  • Removed many compilation warnings.
  • Python Embedding
    • Enhanced the MET tools to read gridded data by calling a Python script which loads data and metadata into a NumPy object.
    • Note that users must configure MET using "--enable-python" to enable this functionality.
    • Example "scripts/python" and "data/python" directories are now included.
    • See Appendix F of the MET User's Guide for more information.
  • Common Libraries
    • Added support for Rotated Lat/Lon projections.
    • Updated the MRMS GRIB2 table.
  • Point Pre-Processing Tools
    • Made a major change to the NetCDF point observation format written by PB2NC, ASCII2NC, MADIS2NC, and LIDAR2NC. Data are now split out into multiple 1-dimensional arrays. Created "table" variables to store unique strings and, for each observation, to store the indices for those tables.
    • Updated the NetCDF point observation format by replacing the UNLIMITED dimension with a fixed dimension, resulting in smaller file sizes.
  • Multiple Tools
    • For the statistics tools, changed the default config file "output_flag" settings from BOTH to NONE.
    • For climatologies, added the "match_month" flag to the "climo_mean" and "climo_stdev" config file dictionaries. Setting "match_month" to FALSE now enable the use of persistence to define the climatology.
    • Added support for area-weighted (AW_MEAN) regridding logic.
    • Enhanced the ASCII2NC, MADIS2NC, PB2NC, STAT-Analysis, TC-Pairs, and TC-STAT to support the "mask.poly" config file option or "-mask_poly" command line option to process masking regions consistently across all the MET tools. They can now process the output of the Gen-Vx-Mask tool.
    • Enabled "vld_thresh = 0" in Ensemble-Stat and Series-Analysis config files.
    • Switched from using double precision to booleans for storing mask fields in Point-Stat, Grid-Stat, Ensemble-Stat, Wavelet-Stat, and Series-Analysis. When processing many masking regions, this will run faster and consume less memory.
    • Enhanced the logic for finding matching UGRD/VGRD verification tasks in Point-Stat and Grid-Stat to make it more robust.
  • Gen-Vx-Mask
    • Added support for the using shapefiles to define masking regions with the "-type shape" option.
  • Regrid-Data-Plane
    • Added the capability to read GOES-16 AOD data using the new "-coord" and "-qc" command line options.
  • ASCII2NC
    • Added support for the AERONET version 3 point observation format.
    • Added support for longer observation variable names.
  • PB2NC
    • Enhanced the output to include the PrepBufr values for report type, input report type, and instrument type for use in the observation error table lookup.
    • Fixed the computation of valid time for AIRNOW observations.
    • Added special handling for ANOWPM message types.
    • Rounded timestamps to the nearest second instead of truncating.
    • Enhanced the "-nmsg" command line option to be specified as the exact number of messages to be processed (e.g. -nmsg 25) or as a percentage of the messages (e.g. -nmsg 25%).
  • Time Summaries
    • Time summaries are now computed across multiple input files for ASCII2NC and PB2NC.
    • Enhanced the "time_summary" config file option by adding a "raw_data" flag to control whether the raw input values are written or only the time summary values.
    • Generalized the "time_summary.width" config file option to an integer for centered time windows (existing) or a dictionary with begin and end time offsets for uncentered windows (new).
  • Grid-Stat
    • Fixed the very slow computation of neighborhood methods fractional coverage fields introduced in met-7.0 when supporting circular neighborhoods. (This was also posted as a bugfix for met-7.0.)
    • Fixed a memory allocation issue, leading to faster run times.
  • Point-Stat
    • The ECNT line type is now computed when applying the HiRA methodology, in which nearby grid points are treated as ensemble member values.
  • Point-Stat and Ensemble-Stat
    • Added the "mask.llpnt" config file option to appply thresholds to filter point observations by applying thresholds to their lat/lon values directly.
  • Ensemble-Stat
    • Added support for observation error perturbations. (See "data/config/README" for details on these settings.)
    • Added a default lookup table in "data/table_files/obs_error_table.txt" but also support the MET_OBS_ERROR_TABLE environment variable for user -specified table.
    • Added the "obs_error" dictionary to the config file to specify how observation error perturbations should be applied.
    • Added the capability to write statistics for perturbed and unperturbed data to the new ECNT line type.
    • Removed the columns from the RHIST line type that were moved to ECNT.
    • Renamed the "-ssvar_mean" command line option as "-ens_mean", but the "-ssvar_mean" option is still quietly supported.
  • MODE
    • Replaced the "area_thresh" and "inten_perc_thresh" config file options with the more generic "filter_attr_name" and "filter_attr_thresh" options.
    • Added "aspect_diff" and "curvature_ratio" as new inputs to the fuzzy logic engine and added corresponding columns to the output file.
  • Series-Analysis
    • Added debug messages to inform the user about missing data.
  • STAT-Analysis
    • Added the capability to aggregate the ECNT line type and derive it from input ORANK lines.
    • Updated the computation of MSVE statistic in the VCNT line type to be done using partial sums rather than matched pair values. This enables it to be computed from VL1L2 lines in STAT-Analysis.
  • STAT-Analysis and TC-STAT
    • Added the "-column_union" job command option to summarize values across multiple columns.
  • Scripting
    • Added a sample Rscript (scripts/Rscripts/plot_ensemble.R) for plotting ensemble line types.
    • Updated the pntnc2ascii.R Rscript to handle the obs_var variable.