Use Case: Grid-to-Obs

Use Case: Grid-to-Obs cindyhg Tue, 06/25/2019 - 09:48

METplus Use Case: Grid to Obs Upper Air

The Grid to Obs Upper Air use case utilizes the MET PB2NC and Point-Stat tools. 

The default statistics created by this use case only dump the partial sums, so we will be also modifying the MET configuration file to add the continuous statistics to the output. 

There is a little more setup in this use case, which will be instructive and demonstrate the basic structure, flexibility and setup of METplus configuration.

Optional, Refer to the MET Users Guide for a description of the MET tools used in this use case. 
Optional, Refer to the METplus Users Guide for a reference to METplus variables used in this use case.

  1. Create Custom Configuration File

Define a unique directory under output that you will use for this use case. Create a configuration file to override OUTPUT_BASE to that directory.

vi ${METPLUS_PARM_BASE}/user_config/g2o.output.conf

Set OUTPUT_BASE to contain a subdirectory specific to the Grid to Obs use case. Make sure to put it under the [dir] section.

[dir]
OUTPUT_BASE = {ENV[METPLUS_TUTORIAL_DIR]}/output/grid_to_obs

Using this custom configuration file and the Grid to Obs use case configuration files that are distributed with METplus, you should be able to run the use case using the sample input data set without any other changes.

 

  1. Copy NetCDF data (output from PB2NC) to output directory

Since pb2nc takes 20 minutes to run on the sample data for this use case, we have pregenerated the netcdf output and will be copying this data to the OUTPUT_BASE directory.

mkdir -p ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs
cp -r ${METPLUS_DATA}/grid_to_obs/netcdf/* ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs/

 

  1. Change the METplus PROCESS_LIST configuration variable for this use case to only run PointStat.

Review the use case configuration file and find the PROCESS_LIST value. Take not of how it is formatted.

less ${METPLUS_PARM_BASE}/use_cases/grid_to_obs/examples/upper_air.conf

Edit g2o.output.conf to override the PROCESS_LIST. PUT IT IN THE [config] SECTION!

vi ${METPLUS_PARM_BASE}/user_config/g2o.output.conf
[config] 
PROCESS_LIST = PointStat

NOTE : PROCESS_LIST defines the processes that will be run for the use case.

 

  1. Review the settings in the Upper Air configuration file:
less ${METPLUS_PARM_BASE}/use_cases/grid_to_obs/examples/upper_air.conf

 

  1. Run the Grid to Obs Upper Air use case:
master_metplus.py -c use_cases/grid_to_obs/examples/upper_air.conf -c user_config/g2o.output.conf

 

  1. Review the output files:

The following is the statistical ouput and files are generated from the command:

ls ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs/gdas
  • point_stat_000000L_20170601_000000V.stat 
  • point_stat_000000L_20170602_000000V.stat 
  • point_stat_000000L_20170603_000000V.stat

METplus Use Case: Grid to Obs CONUS Surface

METplus Use Case: Grid to Obs CONUS Surface cindyhg Tue, 06/25/2019 - 09:53

METplus Use Case: Grid to Obs CONUS Surface

The Grid to Obs CONUS Surface use case utilizes the MET PB2NC and Point-Stat tools. 

The default statistics created by this use case only dump the partial sums, so we will be also modifying the MET configuration file to add the continuous statistics to the output. 

There is a little more setup in this use case, which will be instructive and demonstrate the basic structure, flexibility and setup of METplus configuration.

Optional, Refer to the MET Users Guide for a description of the MET tools used in this use case. 
Optional, Refer to the A-Z Config Glossary section of the METplus Users Guide for a reference to METplus variables used in this use case.

  1. Create Custom Configuration File

Define a unique directory under output that you will use for this use case. Create a configuration file to override OUTPUT_BASE to that directory.

vi ${METPLUS_PARM_BASE}/user_config/g2o.cs.conf

Set OUTPUT_BASE to contain a subdirectory specific to the Grid to Obs use case. Make sure to put it under the [dir] section.

[dir]
OUTPUT_BASE = {ENV[METPLUS_TUTORIAL_DIR]}/output/grid_to_obs_cs

Using this custom configuration file and the Grid to Obs use case configuration files that are distributed with METplus, you should be able to run the use case using the sample input data set without any other changes.

 

  1. Copy NetCDF data (output from PB2NC) to output directory

Since pb2nc takes 20 minutes to run on the sample data for this use case, we have pregenerated the netcdf output and will be copying this data to the OUTPUT_BASE directory.

mkdir -p ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs_cs
cp -r ${METPLUS_DATA}/grid_to_obs/netcdf/* ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs_cs/

 

  1. Change the METplus PROCESS_LIST configuration variable for this use case to only run PointStat. 

Review the use case configuration file and find the PROCESS_LIST value. Take note of how it is formatted.

less ${METPLUS_PARM_BASE}/use_cases/grid_to_obs/examples/conus_surface.conf

Edit g2o.output.conf to override the PROCESS_LIST. PUT IT IN THE [config] SECTION!

vi ${METPLUS_PARM_BASE}/user_config/g2o.cs.conf
[config] 
PROCESS_LIST = PointStat

NOTE : PROCESS_LIST defines the processes that will be run for the use case.

 

  1. Review the setting in the CONUS Surface configuration file:
less ${METPLUS_TUTORIAL_DIR}/METplus/parm/use_cases/grid_to_obs/examples/conus_surface.conf

 

  1. Run the Grid to Obs CONUS Surface use case:
master_metplus.py -c use_cases/grid_to_obs/grid_to_obs.conf -c use_cases/grid_to_obs/examples/conus_surface.conf -c user_config/g2o.cs.conf

You should see some ERROR messages output to the screen. The observation file for each run time could not be found.

 

  1. Debug Errors

Review the log file and read the ERROR messages.

less ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs_cs/logs/master_metplus.log.`date +%Y%m%d`

Look in the directory that was searched to see what is in there. Copy the path output to the screen and list the contents.

ls ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs_cs/nam/conus_sfc

You should see the following files

  • prepbufr.nam.20170601.t00z.tm00.nc
  • prepbufr.nam.20170602.t00z.tm00.nc
  • prepbufr.nam.20170603.t00z.tm00.nc

Recall the ERROR message and the template that is being used to search for data:

{valid?fmt=%Y%m%d}/nam.{valid?fmt=%Y%m%d%H}.nc

The template we are using to find data does not match the actual data. The current template setting expects the following files:

  • 20170601/nam.2017060100.nc
  • 20170602/nam.2017060200.nc
  • 20170603/nam.2017060300.nc

 

  1. Override the filename template for observation input to PointStat with the correct template.

Add it to g2o.cs.conf

vi ${METPLUS_PARM_BASE}/user_config/g2o.cs.conf
[filename_templates]
OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.nam.{valid?fmt=%Y%m%d}.t{valid?fmt=%H}z.tm00.nc

 

  1. Rerun the Grid to Obs CONUS Surface use case:
master_metplus.py -c use_cases/grid_to_obs/grid_to_obs.conf -c use_cases/grid_to_obs/examples/conus_surface.conf -c user_config/g2o.cs.conf

If you set the template correctly, it should take a few minutes to run.

 

  1. Review the output files:

The following is the statistical ouput and files are generated from the command:

ls ${METPLUS_TUTORIAL_DIR}/output/grid_to_obs_cs/nam
  • point_stat_000000L_20170601_000000V.stat 
  • point_stat_000000L_20170602_000000V.stat 
  • point_stat_000000L_20170603_000000V.stat