Setup and Run

Setup and Run admin Fri, 03/01/2019 - 11:05

Once UPP has been built successfully, users will be ready to start the procedures of running UPP on their model data. While unipost.exe can be run interactively on the command line, it would require the user to link all necessary files to their working directory and create the itag file, which includes needed user-defined information read in by unipost, themselves. Instead, new users are encouraged to use one of the convenient run scripts provided in the UPPV4.1/scripts/ directory.

The unipost.exe program requires the following files:

  1. itag: 5(7) line text file that details WRF(FV3) model output to process
  2. Control file: wrf_cntrl.parm (grib1) or postxconfig-NT.txt (grib2): file specifying fields/levels to output
  3. Extra files: e.g. Data tables for microphysics or satellite fields.

*** The provided run scripts will create and/or link these required files automatically ***

itag

itag admin Fri, 03/01/2019 - 11:06

The itag is read in by unipost and is generated automatically in the run_unipost script based on user-defined options in the user edit section. The itag contains the following 5 (7) lines for processing WRF (FV3) data:

  1. Path and name of the WRF (FV3 dyn*) output file to be post-processed.
  2. Format of the WRF (FV3) model output (netcdf or binarynemsiompiio).
  3. Format of the UPP output (GRIB1 or GRIB2).
  4. Forecast valid time (not model start time) in YYYY-MM-DD_HH:MM:SS format (i.e. the forecast time desired to be post-processed).
  5. Dynamic core used (NCAR or GFS).
  6. FV3 only: Path and name of the FV3 phy* output file to be post-processed.
  7. FV3 only: Name of the Grib2 control file (postxconfig-NT-GFS.txt)

Example for WRF-ARW:

/path/to/file/wrfout_d01_2010-06-10_00:00:00
netcdf
grib2
2010-06-10_00:00:00
NCAR

Example for FV3GFS

/path/to/file/dynf006.nemsio
binarynemsiompiio
grib2
2016-10-03_06:00:00
GFS
/path/to/file/phyf006.nemsio
postxconfig-NT-GFS.txt

Control File for GRIB1

Control File for GRIB1 admin Fri, 03/01/2019 - 11:08

This section describes the parameter file, which can be modified to control how the post processing is performed, for outputting GRIB1 format available for WRF-ARW only. A default control file for users to practice with can be found in /UPPV4.1/parm/wrf_cntrl.parm for WRF runs.

Components of the WRF control file

The control file is composed of a header and a body. The header specifies the output file information. The body allows you to select which fields and levels to process.

The header of the wrf_cntrl.parm file contains the following variables:

  • KGTYPE: defines the output grid type, which should always be 255
  • IMDLTY: identifies the process ID for AWIPS
  • DATSET: defines the prefix used for the output file name (i.e. WRFPRS for WRF runs)

The body of the wrf_cntrl.parm file is composed of a series of line pairs similar to:

     (PRESS ON MDL SFCS ) SCAL=( 3.0)
     L=(11000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000 00000)

The first line specifies the:

  • variable (e.g. PRESS)
  • level type (e.g. ON MDL SFCS)
  • precision of the data written out to the GRIB1 file (e.g. SCAL=3.0)

The second line specifies the levels on which the variable is to be output. In this example, a "0" indicates no output on this level and a "1" indicates for that variable to be output at the level specified by the position of the digit and the level type defined.

For those wishing to modify the wrf_cntrl.parm file, please refer to How to control which GRIB1 variables and levels are output.

GRIB1: Variable and level modifications

GRIB1: Variable and level modifications admin Wed, 03/06/2019 - 17:14

Controlling which variables unipost outputs

To output a field, the body of the control file needs to:

  • contain an entry for the appropriate variable
  • have output turned on for at least one level

Controlling which levels unipost outputs

To control which levels to output, the second line is used, where a "1" turns a level on and a "0" turns a level off.

  • For isobaric output, 47 levels are possible from 2 to 1000 hPa (2, 5, 7, 10, 20, 30, 50, 70 mb and then every 25 from 75 mb to 1000 mb). Modifications to which isobaric levels are possible can be made in /UPPV4.1/src/unipost/CTLBLK.f.
    • Modify the variable LSMDEF to change the number of pressure levels in your array (e.g. LSMDEF=47)
    • Modify the SPLDEF array to change the values of pressure levels, i.e. (/200.,500....97500.,100000./)
  • For model-level output, all model levels are possible, from the highest to lowest
  • For soil levels, layers or levels depend on your LSM:
    • Noah LSM soil layers are 0-10, 10-40, 40-100, and 100-200 cm.
    • RUC LSM soil levels are 0, 1, 4, 10, 30, 60, 100, 160, and 300 cm. For the RUC LSM, it is necessary to turn on 5 additional levels in the wrf_cntrl.parm to output 9 instead of 4. (For the old RUC LSM, there are only 6 layers, and if using this, you will need to change RUC LSM from 9 to 6 in the WRFPOST.f routine).
    • Pliem-Xiu LSM layers are 0-1 and 1-100 cm.
  • For low, mid, and high cloud layers, the layers are ≥642 hPa, ≥350 hPa, and <350 hPa, respectively.
  • For PBL layer averages, the levels correspond to 6 layers with a thickness of 30 hPa each.
  • For flight levels, the levels are 30, 50, 80, 100, 305, 457, 610, 914, 1524, 1829, 2134, 2743, 3658, 4572, 6000 and 7010 m.
  • For AGL radar reflectivity, the levels are 4000 and 1000 m.
  • For surface or shelter-level output, only the first position of the line needs to be turned on.

For a list of available fields for GRIB1, see the GRIB1 Table

Control Files for GRIB2

Control Files for GRIB2 admin Fri, 03/01/2019 - 11:09

For outputting in GRIB2 format, a number of parameter files are utilized and can be found in the /UPPV4.1/parm/ directory:

  • post_avblflds.xml: parameter file listing all possible fields available for output in UPP
  • postcntrl.xml: sample configurable parameter file which declares which fields will be output from UPP
  • postxconfig-NT.txt: text file created from the postcntrl.xml that is read by unipost

For new users, the sample text file can be used to run UPP.

The following sections describe the configurable postcntrl.xml parameter file and the process for creating the postxconfig-NT.txt file required by unipost for outputting GRIB2 format.

Components of the post control xml

The header of the postcntrl.xml file contains a number of variables that specify the output file information, most of which will not need modification. The first variable is datset which defines the prefix used for the output file name (e.g. WRFPRS for WRF runs or GFSPRS for FV3 runs), and will be the only one described here.

The body of the postcntrl.xml file contains information for each variable similar to the following examples:

Example: 1

<param>
<shortname>PRES_ON_HYBRID_LVL</shortname>
<pname>PRES</pname>
<level>1. 2. 3. 4. 5.</level>
<scale>6.0</scale>
</param>

Where,

  • shortname declares the variable (e.g. PRES) and level type (e.g. ON_HYBRID_LVL)
  • pname is the standard GRIB2 abbreviation for that type of variable (e.g. PRES)
  • level lists the levels to output the variable on (e.g. hybrid levels 1. 2. 3. 4. 5.)
  • scale is the precision of the data written out to the GRIB2 file (e.g. a scale of 6)

Example 2:

<param>
<shortname>LFTX_ON_ISOBARIC_SFC_500-1000hpa</shortname>
<pname>LFTX</pname>
<table_info>NCEP</table_info>
<level>50000.</level>
<level2>100000.</level2>
<scale>3.0</scale>
</param>

Where,

  • shortname declares the variable (e.g. LFTX) and level type (e.g. ON_ISOBARIC_SFC_500-1000hpa)
  • pname is the standard GRIB2 abbreviation for that type of variable (e.g. LFTX)
  • table_info designates what table information to use if not standard (e.g. NCEP)
  • level lists the first level(s) of a layer, comma separated if outputting multiple layers (e.g. pressure level 50000. Pa)
  • level2 lists the second level(s) of a layer, comma separated if outputting multiple layers (e.g. pressure level 100000. Pa)
  • scale is the precision of the data written out to the GRIB2 file (e.g. a scale of 3)

arrow While the postcntrl.xml is not actually read directly by unipost, it is required for creating the postxconfig-NT.txt.

For those wishing to modify the postcntrl.xml file, please refer to How to control which GRIB2 variables and levels are output.

Creating the postxconfig text file

When outputting GRIB2 format, if any modifications have been made to the postcntrl.xml, a preprocessing step is required by the user to convert the postcntrl.xml to a flat text file postxconfig-NT.txt.

First, in order to ensure the xml files are error free, xml stylesheets are available to validate them and are located in the /UPPV4.1/parm/directory:

  • EMC_POST_CTRL_Schema.xsd: used to validate the postcntrl.xml
  • EMC_POST_Avblflds_Schema.xsd: used to validate the post_avblflds.xml

To run the validation, type:

     xmllint -noout --schema EMC_POST_CTRL_Schema.xsd postcntrl.xml
     xmllint -noout --schema EMC_POST_Avblflds_Schema.xsd post_avblflds.xml

Confirmation of validation will be given (e.g. postcntrl.xml validates) or will otherwise return errors.

Once the xmls are validated, the postxconfig-NT.txt file can be created. Edit the /UPPV4.1/parm/makefile so that it points to the correct directory locations. The makefile will call the perl program /UPPV4.1/parm/POSTXMLPreprocessor.pl to generate the new text file postxconfig-NT.txt.

To run the makefile, type:

     make

This will create the new postxconfig-NT.txt text file necessary for outputting GRIB2 format, which will include all fields from the postcntrl.xml.

GRIB2: Variable and level modifications

GRIB2: Variable and level modifications admin Wed, 03/06/2019 - 17:43

To output a field, the body of the postcntrl.xml file needs to contain an entry for the appropriate variable and include at least the shortname, pname, level, and scale, with additional tags as deemed necessary. If a field is not in your postcntrl.xml but it is available for output, you may simply copy that parameter from the post_avblflds.xml.

Controlling which levels unipost outputs

To modify which levels to output, the values in the level tag will need to be edited to reflect the desired output levels for that variable. If the variable is defined over a specific layer, then both the level (first level of the layer) and level2 (second level of the layer) tags will need to be edited.

The list below details which variables are available for output for a variety of fields.

  • For isobaric output, 47 levels are possible from 2 to 1000 hPa (2, 5, 7, 10, 20, 30, 50, 70 mb and then every 25 from 75 mb to 1000 mb). Modifications to which isobaric levels are possible can be made in /UPPV4.1/src/unipost/CTLBLK.f.
    • Modify the variable LSMDEF to change the number of pressure levels in your array (e.g. LSMDEF=47)
    • Modify the SPLDEF array to change the values of pressure levels, i.e. (/200.,500....97500.,100000./)
  • For model-level output, all model levels are possible, from the highest to lowest
  • For soil levels, layers or levels depend on your LSM:
    • Noah LSM soil layers are 0-10, 10-40, 40-100, and 100-200 cm.
    • RUC LSM soil levels are 0, 1, 4, 10, 30, 60, 100, 160, and 300 cm. For the RUC LSM, it is necessary to turn on 5 additional levels in the wrf_cntrl.parm to output 9 instead of 4. (For the old RUC LSM, there are only 6 layers, and if using this, you will need to change RUC LSM from 9 to 6 in the WRFPOST.f routine).
    • Pliem-Xiu LSM layers are 0-1 and 1-100 cm.
  • For low, mid, and high cloud layers, the layers are ≥642 hPa, ≥350 hPa, and <350 hPa, respectively.
  • For PBL layer averages, the levels correspond to 6 layers with a thickness of 30 hPa each.
  • For flight levels, the levels are 30, 50, 80, 100, 305, 457, 610, 914, 1524, 1829, 2134, 2743, 3658, 4572, 6000 and 7010 m.
  • For AGL radar reflectivity, the levels are 4000 and 1000 m.
  • For surface or shelter-level output, specify the surface level for the particular variable. 

For a list of available fields for GRIB2, see the GRIB2 Table

Scripts

Scripts admin Fri, 03/01/2019 - 11:12

The UPP package includes a number of sample scripts for running UPP. These can be found in /UPPV4.1/scripts and includes the following:

  • run_unipost: basic run script for running UPP
  • run_unipostandgrads: basic run script with additional sample code for plotting using the GrADS package
  • run_unipostandgempak: basic run script with additional sample code for plotting using the GEMPAK package
  • run_unipost_frames: run script for model output with more than one time per file (WRF only)
  • run_unipost_gracet: run script for model output with non-zero minutes/seconds (WRF only)
  • run_unipost_minutes: run script for subhourly model output (i.e. 15 minute output) (WRF only)

Run Script User Edits

Each of the run scripts contain a section at the top for all user modified variables, including a description. For this tutorial, we will look at just the basic run script.

1. Set up the basic path variables:

  • TOP_DIR: Top level directory for the UPPV4.1 source code
  • DOMAINPATH: Working directory for the run
  • UNIPOST_HOME: Path to your UPP build directory
  • POSTEXEC: Location of the UPP executables
  • SCRIPTS: Location of the UPP scripts directory (e.g. UPPV4.1/scripts)
  • modelDataPath: Location of your model data to be processed (e.g. wrfprd/)
  • paramFile: Name and location of your wrf_cntrl.parm file which lists desired fields for GRIB1 output
  • txtCntrlFile: Name and location of the postxconfig-NT.txt which is generated from the postcntrl.xml for GRIB2 output (postxconfig-NT-WRF.txt for WRF or postxconfig-GFS.txt for FV3)

2. Specify the dynamic core being run:

  • dyncore: The model core being used (i.e. ARW or FV3 )

3. Specify the format for the input model files and output UPP files:

  • inFormat: Format of the input model data (i.e. netcdf or binarynemsiompiio)
  • outFormat: Format of the output from UPP (grib or grib2)

4. Specify the forecast cycles to be post-processed:

  • startdate: Forecast startdate in YYYYMMDDHH
  • fhr: First forecast hour to be post-processed
  • lastfhr: Last forecast hour to be post-processed
  • incrementhr: Increment (in hours) between forecast files

5. Specify the domains to post-process (used for WRF only):

  • domain_list: List of domains to post-process (e.g. "d01 d02")

6. Set the run command for you system:

  • RUN_COMMAND: System run command for parallel runs (serial builds not available for UPPV4.1).
    • Parallel: mpirun -np N unipost.exe, where N is the number of processors

Run UPP

Run UPP admin Fri, 03/01/2019 - 11:13

Before running any of the run scripts, perform the following instructions:

1. cd to the DOMAINPATH directory specified in your run script

2. Make a directory to put the UPP results in:

mkdir postprd

3. Make a directory to put a copy of your control parameter file in:

mkdir parm

Note: For first time users it's advised to run with the default parameter file included with the code release

4. Copy over the relevant control file from UPPV4.1/parm/ to the newly created parm directory:

  • wrf_cntrl.parm for GRIB1
  • postxconfig-NT-WRF.txt (postxconfig-NT-GFS.txt) for GRIB2

5. Edit your control file to reflect the fields and levels you want unipost to output (see details on editing the GRIB1 and GRIB2 control files)

6. Copy over your desired run script from UPPV4.1/scripts/ to the postprd/ directory

7. Edit the run script (see the scripts page for details)

Once these directories are setup and the run script edits are complete, your script can be run interactively from the postprd/ directory by simply typing the the script name on the command line:

./run_unipost

UPP Output

UPP Output admin Fri, 03/01/2019 - 11:14

Upon successful completion of a run, the generated output files will be located in the postprd/ working directory and will include WRFPRS_dnn.hh or GFSPRS.hh, where "nn" refers to the domain id and "hh" denotes the forecast hour.

To view GRIB1 output files, use the wgrib utility (if available on your machine). To view details of all variables in the file, type for example:

wgrib -V wrfprs_d01.12

To view GRIB2 output files, use the wgrib2 utility (if available on your machine). To view details of all variables in the file, type for example:

wgrib2 -V wrfprs_d01.12

If you chose to run either the run_unipostandgrads or run_unipostandgempak scripts, then an additional suite of images will also be produced named variablehh_GrADS.png or variablehh.gif for GrADS or GEMPAK, respectively.

If the run did not complete successfully, a log file in the post-processor working directory called unipost_dnn.hh.out (WRF) or unipost.hh.out (FV3GFS), where "nn" refers to the domain id and "hh" denotes the forecast hour, may be consulted for further information.