Session 3: Post-processing.

Session 3: Post-processing.
arbetter Thu, 10/22/2020 - 11:43

The following set of instructions will guide you through how to customize the UPP configuration files to add/remove fields/levels in the Grib2 output for MRW v1.1.0. This requires a pre-processing step to create the flat text configuration file that UPP reads. In order to ensure there are no errors in your modified *.xml files, you may validate them against the XML stylesheets provided in the directory.

 

 

0. Rebuild and rerun the base case (if needed)

0. Rebuild and rerun the base case (if needed)
arbetter Tue, 11/03/2020 - 10:55

Exercises in session 2 may have changed variables or output.  To reset, follow steps 2-7 in Session 1 to rebuild and rerun the baseline case.  This is usually not needed, but added her for completeness.

1. Prepare the files

1. Prepare the files
arbetter Mon, 10/26/2020 - 11:32

Setup the environment for the case directory location.

For bash:

export CASEROOT=/glade/scratch/$USER/UFS_training/ufs-mrweather-app-workflow.c96

For csh:

setenv CASEROOT /glade/scratch/$USER/UFS_training/ufs-mrweather-app-workflow.c96

Load the wgrib2 module on Cheyenne.

module load grib-bins

Move to the directory that contains the configuration files and scripts used for creating a custom flat text file read by UPP.

cd $HOME/UFS_training/my_ufs_sandbox/src/post/parm

We recommend creating copies of the original *.xml and *.txt files that will be modified or created by this process so they are not overwritten.

cp postcntrl_gfs.xml postcntrl_gfs.xml.orig
cp postcntrl_gfs_f00.xml postcntrl_gfs_f00.xml.orig
cp postxconfig-NT-GFS.txt postxconfig-NT-GFS.txt.orig
cp postxconfig-NT-GFS-F00.txt postxconfig-NT-GFS-F00.txt.orig

2. Example: Adding a field

2. Example: Adding a field
arbetter Mon, 10/26/2020 - 11:33

This example describes how to add the field SFEXC (heat exchange coefficient at the surface) to the postcntrl_gfs.xml and postcntrl_gfs_f00.xml configuration files. Using your favorite text editor, edit each configuration file by adding the following parameter to the list at the end of the file, but before the </paramset> and </postxml> tags.

<param>
  <shortname>SFEXC_ON_SURFACE</shortname>
  <scale>3.0</scale>
</param>

You may also delete an unwanted field by completely removing the parameter block of that field from these files and then continuing with step 5.

* Formatting is important; use other entries in the file as a guide.

** Adding fields requires knowledge on which fields can be output for your specific dynamic core and physics options. Not all fields available in UPP can be output with the UFS MRW application.

 

3. Example: Adding levels to a field

3. Example: Adding levels to a field
arbetter Mon, 10/26/2020 - 11:34

This example will describe how to add height agl levels to the SPFH field in the postcntrl_gfs.xml and postcntrl_gfs_f00.xml configuration files. Using your favorite text editor, edit each configuration file by adding the bold highlighted levels to the field ‘Specific humidity on flight levels’.

<param>
<shortname>SPFH_ON_SPEC_HGT_LVL_ABOVE_GRND_FDHGT</shortname>
<level>20. 30. 40. 50. 80. 100.</level>
<scale>4.0</scale>
</param>

Leave no whitespace between the opening <level> and 20. and between 100. and the closing </level>

For the list of available levels for each level type, please reference the UPP documentation on ‘Controlling which levels UPP outputs’.

You may also easily remove levels from the <level> tag by simply deleting them from the list.

* Formatting is important; use other entries in the file as a guide.

 

4. Validate the XML files

4. Validate the XML files
arbetter Mon, 10/26/2020 - 11:35

In order to ensure there are no errors in your modified *.xml files, you may validate them against the XML stylesheets provided in the directory:

xmllint --noout --schema EMC_POST_CTRL_Schema.xsd postcntrl_gfs.xml
xmllint --noout --schema EMC_POST_CTRL_Schema.xsd postcntrl_gfs_f00.xml

If the xml’s are error free, then the commands will return that the xml validates. (e.g. postcntrl_gfs.xml validates)

 

5. make and run the files

5. make and run the files
arbetter Mon, 10/26/2020 - 11:47

In the $HOME/UFS_training/my_ufs_sandbox/src/post/parm directory, create the flat text files.

make

Running make calls the perl program PostXMLPreprocessor.pl, which checks for any modified xml files and re-creates the flat text configuration files where changes were found. The new files generated for this example include:

postxconfig-NT-GFS.txt
postxconfig-NT-GFS-F00.txt

In order to run the MRW post-processing with these new customized flat text files, you will need to copy or link them to the following location from the top level of your case directory.

cp postxconfig-NT-GFS.txt $CASEROOT/SourceMods/src.ufsatm/
cp postxconfig-NT-GFS-F00.txt $CASEROOT/SourceMods/src.ufsatm/

Check that the workflow recognizes the flat text files in the new directory location by previewing the namelist before re-running UPP.

cd $UFS_SCRATCH/ufs-mrweather-app-workflow.c96
./preview_namelists
Using $UFS_SCRATCH/ufs-mrweather-app-workflow.c96/SourceMods/src.ufsatm/postxconfig-NT-GFS.txt for post-processing
Using $UFS_SCRATCH/ufs-mrweather-app-workflow.c96/SourceMods/src.ufsatm/postxconfig-NT-GFS-F00.txt for post-processing

When running the workflow, CIME first looks in the ${CASEROOT}/SourceMods/src.ufsatm directory for the UPP parameter file(s) and uses them for the post-processing. If there are no flat text files present in this directory, then the default parameter file(s) in the pre-configured location are used.

UPP output from the initial case run (Session 1) is located in $CASEROOT/run with the naming convention:

GFSPRS.GrbFHH

If you would like to keep the previous UPP output to compare, copy it to a new name or directory so it does not get overwritten.

Rerun the UPP job to output the new field and/or levels.

cd $CASEROOT
./case.submit --only-job case.gfs_post

You can check the new UPP output using a simple wgrib2 command. For example, to check that the new SPFH height levels were added:

wgrib2 run/GFSPRS.GrbF12 | grep SPFH
...
773:38125949:d=2019082900:SPFH:20 m above ground:12 hour fcst:
776:38348160:d=2019082900:SPFH:30 m above ground:12 hour fcst:
779:38573297:d=2019082900:SPFH:40 m above ground:12 hour fcst:
782:38800697:d=2019082900:SPFH:50 m above ground:12 hour fcst:
786:39055090:d=2019082900:SPFH:80 m above ground:12 hour fcst:
791:39396135:d=2019082900:SPFH:100 m above ground:12 hour fcst:
...

You can also view additional diagnostic output such as min/max values. For example, to just look at the newly added variable SFEXC:

wgrib2 -V run/GFSPRS.GrbF12 | grep SFEXC -A6
717:34726623:vt=2019082912:surface:12 hour fcst:SFEXC Exchange Coefficient [kg/m^2/s]:
    ndata=73728:undef=0:mean=0.0138409:min=0:max=0.189
    grid_template=40:winds(N/S):
        Gaussian grid: (384 x 192) units 1e-06 input WE:NS output WE:SN
        number of latitudes between pole-equator=96 #points=73728
        lat 89.284225 to -89.284225
        lon 0.000000 to 359.062500 by 0.937500

If UPP fails, the UPP log files, $CASEROOT/run/oi.HHH, can be consulted for errors.

6. Optional: regrid UPP using wgrib2

6. Optional: regrid UPP using wgrib2
arbetter Mon, 10/26/2020 - 11:54

UPP grib2 output is on the same grid as the model output, which for the MRW is a Guassian grid. The specifications for a simple lat-lon projection are:

-new_grid latlon lon0:nlon:dlon lat0:nlat:dlat

Variable  Description
lon0/lat0 Longitude/Latitude of first grid point in degrees
nlon/nlat Number of Longitudes/Latitudes
dlon/dlat Grid resolution in degrees of Longitude/Latitude

 

Move to the run directory and run the following example.

cd $CASEROOT/run
wgrib2 GFSPRS.GrbF12 -new_grid_winds earth -new_grid latlon 0:1440:0.25 90:721:-0.25 GFSPRS.GrbF12.latlon

This would interpolate to a 0.25 degree latitude-longitude grid with earth-relative winds (U-wind goes east, V-wind goes north).

You can check the new grid by again using wgrib2. We will print out the info just for the SFEXC field rather than list all fields.

wgrib2 -V GFSPRS.GrbF12.latlon | grep SFEXC -A6
717:1042392327:vt=2019082912:surface:12 hour fcst:SFEXC Exchange Coefficient [kg/m^2/s]:
    ndata=1038240:undef=0:mean=0.013828:min=0:max=0.187
    grid_template=0:winds(N/S):
        lat-lon grid:(1440 x 721) units 1e-06 input WE:NS output WE:SN res 48
        lat 90.000000 to -90.000000 by 0.250000
        lon 0.000000 to 359.750000 by 0.250000 #points=1038240

For more examples, you can refer to the wgrib2 site.