MET Version 9.0

Release Date:

The MET Version 9.0 was released on March 15, 2020.

Release Notes

When applicable, release notes are followed by the GitHub issue number which describes the bugfix, enhancement, or new feature:
   https://github.com/NCAR/MET/issues

  • Major Enhancements:
    • Python embedding: switch from Python 2 to 3, added to ASCII2NC and Stat-Analysis, support for multiple input files in Ensemble-Stat, Series-Analysis, and MTD, support for pandas, handle user's Python environment
    • Other: binned climatologies and new RPS and DMAP line types
  • New Tools:
    • Grid-Diag
    • Point2Grid
    • TC-Gen
    • TC-RMW
    • RMW-Analysis
  • Output Format Changes:
    • Add new ensemble Ranked Probability Score (RPS) line type to the output of Ensemble-Stat and Point-Stat (for HiRA) (#681).
    • Add MTD header columns for "FCST_CONV_TIME_BEG", "FCST_CONV_TIME_END", "OBS_CONV_TIME_BEG", and "OBS_CONV_TIME_END" (#1133).
    • Add MTD data column for a user-specified intensity percentile value INTENSITY_*, where * is the user-specified percentile (#1134).
  • Configuration File Changes:
    • Climatology Settings
      • Add the "climo_stdev" and "climo_cdf" dictionaries for binned climatology logic (#1224).
    • Replace the "climo_mean" dictionary options for "match_day" and "time_step" with "day_interval" and "hour_interval" (#1138).
    • Replace the "climo_cdf_bins" integer option with the "climo_cdf" dictionary (#545).
  • Ensemble-Stat
    • Add the "nbrhd_prob" and "nmep_smooth" dictionaries for computing neighborhood ensemble probability forecasts (#1089).
    • Add the "nep" and "nemp" entries to the "ensemble_flag" dictionary (#1089).
    • Add the "rps" entry to the "output_flag" dictionary (#681).
    • Add the "prob_cat_thresh" option to define probability thresholds for the RPS line type (#1262).
    • Add the "sid_inc" option to specify which stations should be included in the verification (#1235).
  • Grid-Stat
    • Replace the "nc_pairs_var_str" option with the "nc_pairs_var_suffix" and add the "nc_pairs_var_name" option (#1271).
    • Add the "climo_cdf" entry to the "nc_pairs_flag" dictionary (#545).
    • Add the "distance_map" dictionary to control output for the DMAP line type (#600).
    • Add the "dmap" entry to the "output_flag" dictionary (#600).
    • Add the "distance_map" entry to the "nc_pairs_flag" dictionary (#600).
  • Point-Stat
    • Add the "sid_inc" option to specify which stations should be included in the verification (#1235).
    • Add the "prob_cat_thresh" entry to the "hira" dictionary (#1262).
    • Add the "rps" entry to the "output_flag" dictionary (#681). 
  • Series-Analysis
    • Add the "climo_stdev" dictionary to support CDP thresholds (#1138).
  • MTD
    • Add the "conv_time_window" dictionary to the "fcst" and "obs" dictionaries to control the amount of temporal smoothing (#1084).
    • Add the "inten_perc_value" option to specify the desired intensity percentile to be reported (#1134).
  • Point2Grid, GridDiag, TC-Gen, TC-RMW, RMW-Analysis
    • Add default configuration files for these new tools.
  • Build Process Changes:
    • Transition MET source code and issue tracking from Subversion and Jira to GitHub (#805).
    • Enable the G2C library archive file name to be specified at configuration time by setting "GRIB2CLIB_NAME" (default is libgrib2c.a) (#1240).
    • Enable the BUFRLIB library archive file name to be specified at configuration time by setting "BUFRLIB_NAME" (default is libbufr.a) (#1185).
    • Update the copyright date to 2020 and switch to the Apache 2.0 license (#1230).
    • Integrate the Dockerfile into MET GitHub repository and automatically build the master_v8.1 branch, the develop branch, and all tagged releases on DockerHub (#1123).
    • Document the option to install MET into "exec" rather than "bin" (#1189).
    • Continued tracking and reduction of Fortify findings.
  • Enhancements to Existing Tools:
    • Changes for all bugs fixed by met-8.1.1 and met-8.1.2.
    • Grid Library
      • Add definitions for 51 missing pre-defined NCEP grids (#893).
      • Fix bug in the handling of some pre-defined NCEP grids (#1253).
      • Fix inconsistencies for many of the pre-defined NCEP grids (#1254).
      • Fix segfault when passing as input a thinned lat/lon grid (#1252).
      • Fix for Lambert Conformal grids crossing the international date line (#1276).
    • Python Embedding
      • Switch Python embedding from Python 2 to Python 3 (#1080).
      • Enhance Python embedding to support multiple input data types (#1056).
      • Restructure the Python embedding logic to check for the "MET_PYTHON_EXE" environment variable and run the user-specified instance of Python to write a temporary pickle file (#1205).
      • Refine and simplify the Python embedding pickle logic by testing on NOAA machines, Hera and WCOSS (#1264).
      • For Python embedding, support the use of the "MET_PYTHON_INPUT_ARG" constant (#1260).
    • NetCDF and GRIB Libraries
      • Fix bug in processing CF Compliant NetCDF valid time stamps (#1238).
      • Update the vx_data2d_nccf library to support all documented variants of time units (#1245).
      • Fix bug to allow for negative values of unixtime, prior to 1/1/1970 (#1239).
      • Add "GRIB1_tri" configuration file option to filter GRIB1 records based on the time range indicator value (#1263).
      • Bugfix for reporting the units for GRIB2 probabilities as "%" (#1212).
    • Common Libraries 
      • Print a warning message when a user specifies a config file entry as the wrong type (#1225).
      • Fix bug in the parsing of file lists and make this logic consistent across Series-Analysis, Ensemble-Stat, MTD, and TC-RMW (#1226).
      • When the climo mean and/or standard deviation fields contain bad data, exclude that matched pair from the verification (#1204).
      • Break out the Gaussian algorithm into "GAUSSIAN" and "MAXGAUSS" where "GAUSSIAN" applies a Gaussian filter using the "gaussian_dx" and "gaussian_radius" options while "MAXGAUSS" computes the maximum over the neighborhood prior to applying the Gaussian filter (#1234).
      • Report AW_MEAN regridding width at 1, not NA (#1186).
      • Add support for climatological distribution percentile thresholds, such as >CDP50 (#1138).
      • Fix MET-TC bug in the computation of initialization hour and valid hour (#1227).
    • PB2NC
      • Add the derivation of PBL and ensure consistency with VSDB (#1199).
      • Remove non-printable characters that are included in the output of the "-index" command line option (#1241).
    • ASCII2NC
      • Enhance ascii2nc to read point observations via Python embedding with the new "-format python" command line option (#1122).
    • Point2Grid
      • Initial release of the new point to grid tool (#1078).
      • Enhance to process GOES16/17 smoke and dust data from ADP files (#1194).
      • Update quality control processing logic (#1168).
      • Derive AOD at 550nm from 440 and 675 (#1121).
    • Regrid-Data-Plane
      • Remove GOES16/17 data processing since it was reimplemented in the new
    •  Point2Grid tool (#1243).
      • Add support for Gaussian regridding method to support the definition of surrogate-severe forecasts (#1136).
    • PCP-Combine
      • Support multiple arguments for the "-pcpdir" command line option (#1191).
      • Fix bug in the handling of bad data for the "-subtract" command (#1255).
    • Point-Stat
      • Enhance the HiRA logic to support CDP threshold types (#1251).
      • Add new ensemble Ranked Probability Score (RPS) output line type for HiRA (#681).
    • Grid-Stat
      • Add an "nc_pairs_var_name" configuration file option to explicitly define the NetCDF matched pairs output variable names (#1271).
      • Add new distance map (DMAP) output line type (#600).
    • Ensemble-Stat
      • Enhance to support Python embedding with "MET_PYTHON_INPUT_ARG" (#1140).
      • Add the computation of neighborhood probability forecasts (#1089).
      • Apply binned climatology logic using the "climo_cdf" config file option to the computation of ECNT statistics (#1224).
      • Fix logic for computing the lead time of a time-lagged ensemble to use the minimum lead time of the ensemble members (#1244).
      • Fix bug for initializing output variables when the first field processed contains missing data (#1242).
      • Add new ensemble Ranked Probability Score (RPS) output line type (#681).
    • Point-Stat and Ensemble-Stat
      • Add the new "sid_inc" configuration option to explicitly specify which stations should be included in the verification (#1235).
    • Point-Stat, Grid-Stat, and Ensemble-Stat
      • When applying climatology bins, report the mean of statistics across the bins for SL1L2, SAL1L2, CNT, PSTD, and ECNT line types (#1138).
    • Stat-Analysis
      • Add support for evaluating point forecasts by reading matched pairs via Python embedding (#1143).
    • MODE
      • Fix bug in the computation of the aspect ratio of objects with an area of 1 (#1215).
    • MTD
      • Enhance to support Python embedding with "MET_PYTHON_INPUT_ARG" (#1140).
      • Make the amount of temporal smoothing a configurable option (#1084).
      • Add a user-specified object intensity percentile to the output (#1134).
      • Fix bug for the centroid longitude being reported in degrees west rather than degrees east (#1214).
    • Series-Analysis
      • Enhance to support Python embedding with "MET_PYTHON_INPUT_ARG" (#1140).
      • Fix the memory allocation logic to dramatically reduce memory usage by up to a factor of 30 (#1267).
    • Grid-Diag
      • Initial release of the new grid diagnostics tool (#1106).
      • Fix bug in the application of the masking regions (#1261).
    • TC-Gen
      • Initial release of the new TC genesis tool (#1127).
      • Fix bug when checking the "min_duration", update log messages, and refine configuration file options (#1127).
    • TC-RMW
      • Initial version of the Tropical Cyclone, Radius of Maximum Winds tool (#1085).
    • RMW-Analysis
      • Initial version of the Radius of Maximum Winds Analysis tool (#1178).


Known Issues and Fixes

Typically, accumulated precipitation is stored in GRIB files using an accumulation interval with a "time range" indicator value of 4. Here is a description of the different time range indicator values and meanings: http://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html

For example, take a look at the APCP in the GRIB files included in the MET tar ball:

wgrib ${MET_BUILD_BASE}/data/sample_fcst/2005080700/wrfprs_ruc13_12.tm00_G212 | grep APCP

1:0:d=05080700:APCP:kpds5=61:kpds6=1:kpds7=0:TR=4:P1=0: \

P2=12:TimeU=1:sfc:0- 12hr acc:NAve=0

2:31408:d=05080700:APCP:kpds5=61:kpds6=1:kpds7=0:TR=4: \

P1=9:P2=12:TimeU=1:sfc:9- 12hr acc:NAve=0

The "TR=4" indicates that these records contain an accumulation between times P1 and P2. In the first record, the precip is accumulated between 0 and 12 hours. In the second record, the precip is accumulated between 9 and 12 hours.

However, the GRIB data uses a time range indicator of 5, not 4.

wgrib rmf_gra_2016040600.24 | grep APCP

291:28360360:d=16040600:APCP:kpds5=61:kpds6=1:kpds7=0: \

TR=5:P1=0:P2=24:TimeU=1:sfc:0-24hr diff:NAve=0

pcp_combine is looking in "rmf_gra_2016040600.24" for a 24 hour *accumulation*, but since the time range indicator is no 4, it doesn't find a match.

If possible switch the time range indicator to 4 on the GRIB files. If this is not possible, there is another workaround. Instead of telling pcp_combine to look for a particular accumulation interval, give it a more complete description of the chosen field to use from each file. Here is an example:

pcp_combine -add rmf_gra_2016040600.24 'name="APCP"; level="L0-24";' \

rmf_gra_2016040600_APCP_00_24.nc 

The resulting file should have the accumulation listed at 24h rather than 0-24.