Release Options
Component
Release Date:
The MET Version 6.0 was released on April 3, 2017.
Release Notes
What's New?
- METv6.0 includes new tools and enhancements:
- New Tools: lidar2nc
- Major Enhancements: NetCDF4 support, MET-TC E-Decks, HiRA verification, description header column
New Tools
- The lidar2nc reformats HDF4 Calipso data files into a NetCDF point observation format for use by Point-Stat and Ensemble-Stat.
Enhancements to Existing Tools
- Distribute the latest set of bugfixes, details not listed here.
- Enhance MET to handle southern polar stereographic grids.
- NetCDF
- Enhance MET tools to read/write NetCDF4 files and support a configurable compression level ("nc_compression" configuration option, ${MET_NC_COMPRESS} environment variable, and "-compress" command line option, listed in order of precedence).
- Support "add_offset" and "scale_factor" attributes to read NetCDF variables of type short, int, float, and double.
- Add logic to reorient latitudes to be south to north, if needed.
- GRIB1 and GRIB2 files
- Refine logic for defining custom GRIB tables using the ${MET_GRIB_TABLES} environment variable (previously named ${USER_GRIB_TABLES}).
- Support filtering GRIB2 data by product definition template, generating process, ensemble type, or derived product type using "GRIB2_pdt", "GRIB2_process", "GRIB2_ens_type", and "GRIB2_der_type" configuration options, respectively.
- Probabilities
- Support shorthand convention for reading/writing probabilistic thresholds where "==n" is interpreted as equally spaced thresholds (e.g. "==0.25" means ">=0.00,>=0.25,>=0.50,>=0.75,>=1.00").
- Add new "prob_as_scalar" configuration option to disable the application of probabilistic verification methods.
- Additional configuration options
- Add new description ("DESC") header column to MET ASCII output files. The description string may be set separately for each verification task defined in the configuration file.
- Add ${MET_TMP_DIR} environment variable to specify where temporary files should be written and override the "tmp_dir" configuration option.
- Remove restriction that UGRD must be immediately followed by VGRD at the same level to compute VL1L2 and VAL1L2 output.
- ASCII2NC and MADIS2NC
- Add "-mask_sid" command line option to define a group of station ID's.
- Reduce runtime by reading blocks of observations.
- PB2NC
- Enable "grid" configuration option in "mask" to be set to a named grid or gridded data file.
- Point-Stat
- Add "hira" configuration options to support High Resolution Assessment (HiRA) verification, a neighborhood verification method for point observations.
- Ensemble-Stat
- Add ensemble spread to the output: "SPREAD" column in RHIST line type and "ENS_SPREAD" in the ORANK line type.
- Dramatically reduce runtime with more efficient memory management and data access methods.
- Point-Stat and Ensemble-Stat
- Refine logic for the "duplicate_flag" configuration option to check for duplicate observations at the same station.
- Add "obs_summary" and "obs_perc_value" configuration options to derive summary metrics for multiple observations at the same station within the time window: "NONE", "MIN", "MAX", "UW_MEAN", "DW_MEAN", "MEDIAN", or "PERC" to use all observations, the minimum, maximum, average, time-weighted average, median, or a particular percentile, respectively.
- Add five new "method" options in the "interp" section: "UPPER_LEFT", "UPPER_RIGHT", "LOWER_RIGHT", and "LOWER_LEFT" to select the forecast value at a particular grid box corner and "BEST" to select the forecast value which most closely matches the observation value.
- Gen-Vx-Mask
- Add support for masking by the position of the sun using the solar altitude ("solar_alt") option azimuth ("solar_azi") "-type" options.
- Regrid-Data-Plane
- Change the "-name" command line option to a comma-separated list of output variable names for each field specified.
- PCP-Combine
- Switch error for subtracting data with different initialization times to a warning.
- Grid-Stat
- Add "shape" configuration option (SQUARE or CIRCLE) to the "interp" dictionary. Will be applied to other MET tools in future releases.
- Add support for masking by the position of the sun using the solar altitude ("solar_alt") option azimuth ("solar_azi") "-type" options.
- Allow Grid-Stat to run with only matched pairs output requested ("nc_pairs_flag") but no statistics computed ("output_flag").
- Dramatically reduce runtime with more efficient memory management.
- MODE
- Add support for arrays in the "merge_thresh" configuration option.
- MODE-Analysis
- Allow "-lookin" option to be set to a directory or filename.
- MTD
- Add cluster object centroid coordinates to the 2D attributes file.
- Add "DURATION_DIFF" column to the 3D pair attributes file.
- Add "CDIST_TRAVELLED" column to the 3D simple attributes file to indicate the total distance travelled for 2D object slices.
- GSID2MPR
- Reduce runtime by refining the logic of checking for duplicates.
- STAT-Analysis
- Add "-mask_sid" job command option to define a group of station ID's.
- WWMCA-Regrid
- Generalize to handle a wider variety of input naming conventions.
- Add "write_pixel_age" and "swap_endian" configuration options to process pixel age data rather than cloud data.
- MET-TC
- Enhance TC-Pairs to read E-Deck files using the "-edeck" command line option and write the new PROBRIRW line type containing probabilities of rapid intensity changes.
- Enhance TC-Stat to parse and filter PROBRIRW lines.
- Add new "probrirw" job type to TC-Stat to apply probabilistic methods to PROBRIRW lines and write PCT, PSTD, PRC, and PJC lines.
- Add "lead_req" configuration option to TC-Pairs and TC-Stat to specify required lead times that must be present for a track to be retained.
- Modify TC-Pairs to derive interp12, consensus, lag, and baseline tracks before applying track filtering logic.
- Distribute enhancements to the plot_tcmpr.R Rscript plotting utility.
- Enhance logic of STAT-Analysis, MODE-Analysis, and TC-Stat to process output from all previous versions of MET by reading version-specific column offset files rather than hard-coding those values.
Known Issues and Fixes
All Recommended Updates (Last Updated 11/13/2017)
Retrieve all of the recommended updates for METv6.0 in a tarfile:
- Download the tarfile met-6.0_patches_20171113.tar.gz
- Copy it into the top-level met-6.0 directory.
- Uncompress and untar the file. Please note that the new version of the files in the tarball will overwrite the original version of those files. Any changes you may have made will be lost.
- Rebuild MET, being sure to run make clean and configure first.
Summary of Recommended Updates
- 11/13/2017: Handle overlapping GRIB layers.
- 10/30/2017: Add mean sea level to the list of GRIB2 fixed level types.
- 09/20/2017: Fix madis2nc segfault.
- 09/20/2017: Add support for GRIB2 decimal level values.
- 09/20/2017: Add support for GRIB2_stat_type filtering option.
- 07/31/2017: Add support for Polar Stereographic CF-Compliant NetCDF files.
- 07/25/2017: Fix GRIB1 file support for fixed level types.
- 07/21/2017: Fix support for reorienting NetCDF lat/lon grids.
- 07/21/2017: Fix Mercator grid support.
- 07/21/2017: Fix GRIB2 file support for fixed level types.
- 06/23/2017: Fix Grid-Stat's handling of missing climatology data.
- 06/23/2017: Fix PCP-Combine to handle accumulation hours up to 5 digits in length.
- 06/16/2017: Fix MET NetCDF file support for more than 2 dimensions.
- 06/16/2017: Fix GRIB2 file support for probabilities.
- 04/19/2017: Print status message after running make test.
- 04/19/2017: Remove duplicate GRIB2 table entry for APCP.
- 04/19/2017: Change warnings for mutiple GRIB table matches to debug messages.
- 04/19/2017: Refine bilinear interpolation logic at the grid boundary.
- 04/19/2017: Fix buffer overflow bug in PB2NC.
- 04/19/2017: Fix dimension indexing bug in MET NetCDF library.
Details
Handle overlapping GRIB layers.
Posted 11/13/2017
Problem: Unable to select a GRIB layer that falls within a larger layer which preceeds it in the file.
Solution: Update logic to search for an exact match for the request layer.
Update: met-6.0/src/libcode/vx_data2d_grib/data2d_grib.cc
Add mean sea level to the list of GRIB2 fixed level types.
Posted 10/30/2017
Problem: GRIB2 level type of 101 for mean sea level pressure should also be considered a fixed level type.
Solution: Update the bugfix posted on 07/21/2017 to include 101.
Update: met-6.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc
Fix madis2nc segfault.
Posted 09/20/2017
Problem: The madis2nc tool produces a segfault when processing large numbers of observations, particularly from METAR files. The logic for resetting an interal buffer was incorrect.
Solution: Correct the logic for handling the buffer.
Update: met-6.0/src/tools/other/madis2nc/madis2nc.cc
Add support for GRIB2 decimal level values.
Posted 09/20/2017
Problem: MET was unable to recognize decimal level values, such as 0.35 meters below ground. The scaling factor is listed in the GRIB2 record but MET fails to process it.
Solution: Enhance MET's GRIB2 library to process the scaling factor when parsing level values.
Update: met-6.0/src/libcode/vx_data2d/var_info.cc
Update: met-6.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc
Add support for GRIB2_stat_type filtering option.
Posted 09/20/2017
Problem: MET was unable to filter GRIB2 records based on the statistical processing type specified by Table 4.10.
Solution: The GRIB2_stat_type option was added to filter records.
Update: met-6.0/src/basic/vx_config/config_constants.h
Update: met-6.0/src/libcode/vx_data2d_grib2/data2d_grib2.h
Update: met-6.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc
Update: met-6.0/src/libcode/vx_data2d_grib2/var_info_grib2.h
Update: met-6.0/src/libcode/vx_data2d_grib2/var_info_grib2.cc
Add support for Polar Stereographic CF-Compliant NetCDF files.
Posted 07/31/2017
Problem: MET fails to parse polar stereographic projections from CF-compliant NetCDF files.
Solution: Update the vx_data2d_nccf library to parse polar stereographic projections. Users encountering additional problems should contact met_help@ucar.edu.
Update: met-6.0/src/libcode/vx_data2d_nccf/nccf_file.cc
Fix GRIB1 file support for fixed level types.
Posted 07/25/2017
Problem: MET fails to apply the GRIB_lvl_typ configuration file filtering option when processing GRIB1 records.
Solution: Update the vx_data2d_grib library to only keep records matching the GRIB_lvl_typ value.
Update: met-6.0/src/libcode/vx_data2d_grib/data2d_grib_utils.cc
Fix support for reorienting NetCDF lat/lon grids.
Posted 07/21/2017
Problem: MET inspects lat,lon grids from NetCDF files which are written from north to south and reorients them from south to north. While it is flipping the grid definition, it fails to actually flip the data values south to north.
Solution: The fix is to update the vx_data_nccf to correctly flip the data along with the grid definition.
Update: met-6.0/src/libcode/vx_grid/grid_base.h
Update: met-6.0/src/libcode/vx_grid/grid_base.cc
Update: met-6.0/src/libcode/vx_data2d_nccf/nccf_file.cc
Fix Mercator grid support.
Posted 07/21/2017
Problem: MET does not properly handle Mercator grid which cover more than 180 degrees of longitude but do not cover the full globe.
Solution: Update the Mecator grid code in the vx_grid library to correctly convert between grid x,y and lat,lon.
Update: met-6.0/src/libcode/vx_grid/merc_grid.h
Update: met-6.0/src/libcode/vx_grid/merc_grid.cc
Fix GRIB2 file support for fixed level types.
Posted 07/21/2017
Problem: MET fails to apply the GRIB_lvl_typ configuration file filtering option when processing GRIB2 records. There is also an issue in specifying the level value for fixed level types, such as the surface or tropopause level.
Solution: Update the vx_data2d_grib2 library to only keep records matching the GRIB_lvl_typ value, if specfied. Also, for fixed GRIB level types 1 through 10 (http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-5.shtml) explicitly set the level value to 0. In GRIB2 files from the Air Force, for these fixed level types, the level values are not explicitly set and instead contain a fill value of -2147483647. This change effectively resets that fill value to 0.
Update: met-6.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc
Fix Grid-Stat's handling of missing climatology data.
Posted 06/23/2017
Problem: Passing to Grid-Stat a climatology field that contains missing data for probabilistic verification results in a runtime error. Missing climatology data may be the result of using the automated regridding feature.
Solution: Enhance Grid-Stat to skip points containing bad climatology values.
Update: met-6.0/src/tools/core/grid_stat/grid_stat.cc
Fix PCP-Combine to handle accumulation hours up to 5 digits in length.
Posted 06/23/2017
Problem: Requesting an accumulation interval greater than 999 results in a parsing error from PCP-Combine.
Solution: When PCP-Combine parses a timestring, interpret a number with 1 to 5 digits as hours and a number with 6 to 9 digits as hours followed by 2-digit minutes and 2-digit seconds.
Update: met-6.0/src/basic/vx_cal/time_strings.cc
Fix MET NetCDF file support for more than 2 dimensions.
Posted 06/16/2017
Problem: The MetNcFile::open() member function fails to store the number of dimensions defined in the file. This causes an error when processing NetCDF variables defined with more than 2 dimensions.
Solution: Storing the number of file dimensions enables the rest of the class to process the data correctly, regardless of the number of variable dimensions.
Update: met-6.0/src/libcode/vx_data2d_nc_met/met_file.cc
Fix GRIB2 file support for probabilities.
Posted 06/16/2017
Problem: Setting prob = TRUE; to indicate that a GRIB2 data field should be processed as probabilities does not work.
Solution: Update the GRIB2 logic for processing probabilities to make it consistent with the logic of GRIB1. Add a check for the presence of the prob setting as a boolean value.
Update: met-6.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc
Update: met-6.0/src/libcode/vx_data2d_grib2/var_info_grib2.cc
Print status message after running make test.
Posted 04/19/2017
Problem: Previous versions of MET printed a status message to indicate when the test scripts had completed but this is missing from the met-6.0 release.
Solution: Add the message back in
*** Finished Testing the Model Evaluation Tools Project ***
Update: met-6.0/scripts/Makefile
Remove duplicate GRIB2 table entry for APCP.
Posted 04/19/2017
Problem: The GRIB2 GALWEM table contains a duplicate entry for APCP.
Solution: Remove the duplicate.
Update: met-6.0/data/table_files/grib2_galwem.txt
Change warnings for mutiple GRIB table matches to debug messages.
Posted 04/19/2017
Problem: The logic for how/when to execute GRIB table lookups differs for GRIB1 and GRIB2. GRIB2 executes a table lookup prior to opening the data file which often leads to warnings about multiple matches.
Solution: In version 6.1 of MET, the GRIB1 and GRIB2 table lookup logic will be consistent. For now, simply change the multiple matches warnings to debug(3) message.
Update: met-6.0/src/libcode/vx_data2d/table_lookup.cc
Refine bilinear interpolation logic at the grid boundary.
Posted 04/19/2017
Problem: The bilinear interoplation logic in MET does not handle the edge of the grid the same way that copygb and wgrib2 do. MET creates a strip of bad data values where copygb and wgrib2 have valid data.
Solution: Update the bilinear interpolation algorithm to match the output of copygb and wgrib2. When at the end of the grid, only apply bound checks to corner points which have a non-zero weight.
Update: met-6.0/src/basic/vx_util/interp_util.cc
Fix buffer overflow bug in PB2NC.
Posted 04/19/2017
Problem: In version 6.0 of MET, PB2NC was enhanced to write blocks of data to the output NetCDF file rather than writing messages one at a time. While much more efficient, this change introduced a fixed upper limit (128*1024) for the number of messages it could process. This led to corrupted output for large (i.e. global) PREPBUFR files.
Solution: Update PB2NC to write and then reset the buffer when full.
Update: met-6.0/src/basic/vx_config/config_util.cc
Update: met-6.0/src/tools/other/pb2nc/pb2nc.cc
Fix dimension indexing bug in MET NetCDF library.
Posted 04/19/2017
Problem: Version 6.0 of MET uses the NetCDF4 library instead of the NetCDF3 library but introduces a bug in the processing of dimensions in MET NetCDF files. Processing MET NetCDF files with more than 2 dimensions may result in the following error message:
star found in bad slot
Solution: Fix the dimension indexing bug.
Update: met-6.0/src/libcode/vx_data2d_nc_met/met_file.cc
Addresses
mailing:
PO Box 3000
Boulder, CO 80307-3000
Shipping:
3090 Center Green Dr.
Boulder, CO 80301
Follow Us
Copyright © 2024. All rights reserved.