UFS Short-Range Weather (SRW) Practical Session Guide | Session 3 > 1. Customizing Unified Post Processor (UPP) Output

The following set of instructions will guide you through how to customize the UPP configuration file to add/remove fields/levels in the Grib2 output for SRW v1.0.1. This requires a pre-processing step to create the flat text configuration file that UPP reads. This guide assumes you have already completed Session 1: Setup and Run the Workflow.

1. Set environment variables as was done in Session 1:
   
   SCRATCH_DIR=/glade/scratch/$USER
   SR_WX_APP_TOP_DIR=$SCRATCH_DIR/ufs-srweather-app
   EXPT_SUBDIR=test_CONUS_25km_GFSv15p2
   EXPTDIR=${SR_WX_APP_TOP_DIR}/../expt_dirs/${EXPT_SUBDIR}
2. Move to the directory that contains the configuration files and scripts used for creating a custom flat text file read by UPP.
   
   cd ${SR_WX_APP_TOP_DIR}/src/EMC_post/parm
3. Create a copy of the original fv3lam.xml file that will be modified so you do not overwrite the original. This is the file that lists all variables you would like to output.
   
   cp fv3lam.xml fv3lam.xml.${USER}

   For a complete list of fields that can be output by UPP, please refer to the UPP Grib2 table in the online documentation. (Please note that some fields can only be output using certain models and/or physics options.)

4. Example 1: Adding a simple field
   This example describes how to add the field POT (potential temperature) on pressure levels to the fv3lam.xml.${USER} configuration file. This configuration file is split into two sections; BGDAWP and BGRD3D. Edit the configuration file by adding the following parameter to the BGRD3D section (search BGRD3D), but before the </paramset> tag at the bottom of the file.
   <param>
   <shortname>POT_ON_ISOBARIC_SFC</shortname>
   <pname>POT</pname>
   <level>25000. 50000. 70000. 85000.</level>
   <scale>4.0</scale>
   </param>

   You may also delete an unwanted field by completely removing the parameter block of that field from the XML file and then continuing with step 7.

   * Formatting is important; use other entries in the file as a guide.

   ** Adding fields requires knowledge on which fields can be output for your specific dynamic core and physics options. Not all fields available in UPP can be output with the UFS SRW application.

5. Example 2: Adding levels to a field
   This example will describe how to add levels to a field in the fv3lam.xml.${USER} configuration file. Edit the configuration file by adding the bold highlighted levels (in geopotential meters) to the field ‘Specific humidity on flight levels’.
   <param>
   <shortname>SPFH_ON_SPEC_HGT_LVL_ABOVE_GRND_FDHGT</shortname>
   <level>20. 30. 40. 50. 80. 100. 305. 610. 1524.</level>
   <scale>4.0</scale>
   </param>

   For the list of available levels for each level type, please reference the UPP documentation on ‘Controlling which levels UPP outputs’. You may also easily remove levels from the tag by simply deleting them from the list.

6. Example 3: Adding a satellite brightness temperature field
   This example describes how to add the field SBTA169 (GOES-16 Channel 9; IR mid-level troposphere water vapor) to the fv3lam.xml.${USER} configuration file. (Note that while other channels are available for GOES-16, we are only adding this one).
   Edit the configuration file by adding the following parameter to the BGDAWP section, but before the </paramset> tag (~L1820 of the original file).
   <param>
   <shortname>SBTA169_ON_TOP_OF_ATMOS</shortname>
   <scale>4.0</scale>
   </param>

   In order to output a satellite brightness temperature field, the CRTM coefficient files for each of the satellites are needed. Move to the top level UPP directory, pull the CRTM fix tar file, and unpack.

   cd ${SR_WX_APP_TOP_DIR}/src/EMC_post/
   mkdir crtm && cd crtm
   wget https://github.com/NOAA-EMC/EMC_post/releases/download/upp_v9.0.0/fix.tar.gz
   tar -xzf fix.tar.gz

   These coefficient files need to be copied to the UPP working directory and to do so requires modification of the UPP run script. Edit the file ${SR_WX_APP_TOP_DIR}/regional_workflow/scripts/exregional_run_post.sh to add the following lines in the same location where other files are being copied to the UPP run directory (~L189).

   # Copy coefficients for crtm2 (simulated synthetic satellites)
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/EmisCoeff/IR_Water/Big_Endian/Nalli.IRwater.EmisCoeff.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/EmisCoeff/MW_Water/Big_Endian/FAST*.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/EmisCoeff/IR_Land/SEcategory/Big_Endian/NPOESS.IRland.EmisCoeff.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/EmisCoeff/IR_Snow/SEcategory/Big_Endian/NPOESS.IRsnow.EmisCoeff.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/EmisCoeff/IR_Ice/SEcategory/Big_Endian/NPOESS.IRice.EmisCoeff.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/AerosolCoeff/Big_Endian/AerosolCoeff.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/CloudCoeff/Big_Endian/CloudCoeff.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/SpcCoeff/Big_Endian/*.bin ./
   cp_vrfy ${EMC_POST_DIR}/crtm/fix/TauCoeff/ODPS/Big_Endian/*.bin ./
7. In order to ensure there are no errors in your modified fv3lam.xml.${USER} file, you may validate it against the XML stylesheets provided in the UPP parm directory.
   
   cd ${SR_WX_APP_TOP_DIR}/src/EMC_post/parm
   xmllint --noout --schema EMC_POST_CTRL_Schema.xsd fv3lam.xml.${USER}

   If the XML is error free, then the command will return that the XML validates. (e.g. fv3lam.xml.${USER} validates)

8. Create the flat text file.
   
   Note:  The following is a one-line command.
   /usr/bin/perl PostXMLPreprocessor.pl fv3lam.xml.${USER} fv3lam_post_avblflds.xml postxconfig-NT-fv3lam.txt.${USER}

   The perl program PostXMLPreprocessor.pl processes the modified xml and creates the flat text configuration file with the file name provided. In this case the new flat text file is postxconfig-NT-fv3lam.txt.${USER} and should now exist in the parm directory if this step was successful.

9. To run the SRW post-processing with this new customized flat text file for the experiment from session 1, move back to the directory with the workflow configuration file.
   
   cd ${SR_WX_APP_TOP_DIR}/regional_workflow/ush

   Copy the config file from session 1 back to config.sh.

   cp config.sh.session_1 config.sh

   Then modify the config.sh, adding the following lines:

   USE_CUSTOM_POST_CONFIG_FILE="TRUE"
   CUSTOM_POST_CONFIG_FP="/glade/scratch/${USER}/ufs-srweather-app/src/EMC_post/parm/postxconfig-NT-fv3lam.txt.${USER}"

   Setting the USE_CUSTOM_POST_CONFIG_FILE to TRUE tells the workflow to use the custom file located in the user-defined path, CUSTOM_POST_CONFIG_FP. The path should include the filename. If this is set to true and the file path is not found, then an error will occur when trying to generate the SRW Application workflow using the generate_FV3LAM_wflow.sh script.

10. You can then proceed with running the workflow using the modified configuration file.

    Load the Python environment and generate the FV3LAM workflow.

    source ${SR_WX_APP_TOP_DIR}/env/wflow_cheyenne.env
    ./generate_FV3LAM_wflow.sh

    Start the entire case workflow following the instructions from Session 1: Step 4. Run the Workflow and UPP will use the new flat postxconfig-NT-fv3lam.txt.${USER} file. Once the workflow is complete, the new UPP output should include the added fields and levels.

Optional Downstream Application examples for UPP output

You can check the contents of the UPP output using a simple wgrib2 command. Move to the directory containing the UPP output.

Load the module for wgrib2 on Cheyenne and use it to list the fields within the Grib2 files.

Or for additional diagnostic output such as min/max values and grid specs:

If you want to seach for a specific variable rather than printing out the entire list:

You can produce a simple plot of the GOES-16, channel 9 field that we added in step 6 using the Model Evaluation Tools (MET) plot_data_plane function.

Load METv10.0.0 via a module on Cheyenne.

In the postprd directory, use the plot_data_plane tool to create a postscript file.

The GOES-16 Channel 9 (IR mid-level troposphere water vapor) image at F036 should look like this.

Regrid UPP output using wgrib2 and create a simple plot using MET. (Note that MET also has regrid capabilities; however, for those without the MET software, wgrib2 is a great tool)

UPP grib2 output is on the same grid as the model output, which for this SRW case is a 25 km Lambert Conformal grid. This example shows how to transform to a lat-lon grid. The specifications for a simple lat-lon projection are:

Example:

This will interpolate to a 0.125 degree latitude-longitude grid with earth-relative winds (U-wind goes east, V-wind goes north), zoomed in on the region of active convection.

Make a quick plot of GOES-16 Channel 9 from the newly regrid data using MET.

You can also view the contents of the new file with wgrib2 as before.

For more wgrib2 options/examples, you can refer to the wgrib2 site.

For more information on MET/METplus, refer to the MET Users Page/METplus Users Page.