In this part, we will demonstrate how to define a custom regional ESG (Extended Schmidt Gnomonic) grid instead of using one of the predefined ones. Note that in addition to this grid, we must define a corresponding write-component grid. This is the grid on which the fields are provided in the output files. (This remapping from the native ESG grid to the write-component grid is done because tasks downstream of the forecast may not be able to work with the native grid.)
To define the custom ESG grid and the corresponding write-component grid, three groups of parameters must be added to the experiment configuration file config.sh:
- ESG grid parameters
- Computational parameters
- Write-component grid parameters
Below, we will demonstrate how to add these parameters for a new 3 km sub-CONUS ESG grid. For reference, note that for the predefined grids, these parameters are set in the script $USHDIR/set_predef_grid_params.sh.
First, change location to USHDIR and make a backup copy of your config.sh from the experiment in Session 2.2:
cp config.sh config.sh.session_2p2
For this experiment, we will start with the configuration file used in Session 2.1 and modify it. Thus, first copy it into config.sh:
Now edit config.sh as follows:
- Change the name of the experiment directory to something that properly represents the purpose of this experiment:
EXPT_SUBDIR="test_custom_ESGgrid" - Comment out the line that defines the predefined grid, i.e. change the line
PREDEF_GRID_NAME="RRFS_CONUS_3km"to
#PREDEF_GRID_NAME="RRFS_CONUS_3km" - Ensure that the grid generation method is set to "ESGgrid", i.e. you should see the line
GRID_GEN_METHOD="ESGgrid"right after the line for PREDEF_GRID_NAME that you commented out in the previous step. If not, make sure to include it. The other valid value for this parameter is "GFDLgrid", but we will not demonstrate adding that type of grid here; ESG grids are preferred because they provide more uniform grid size distributions.
- Ensure that QUILTING is set to "TRUE". This tells the experiment to remap output fields from the native ESG grid onto a new orthogonal grid known as the write-component grid and then write the remapped grids to a file using a dedicated set of MPI processes.
- Add the definitions of the ESG grid parameters after the line for QUILTING:
- Define the longitude and latitude (in degrees) of the center of the grid:
ESGgrid_LON_CTR="-114.0"
ESGgrid_LAT_CTR="37.0" - Define the grid cell size (in meters) in the x (west-to-east) and y (south-to-north) directions:
ESGgrid_DELX="3000.0"
ESGgrid_DELY="3000.0" - Define the number of grid cells in the x and y directions:
ESGgrid_NX="420"
ESGgrid_NY="300" - Define the number of halo points around the grid with a “wide” halo
ESGgrid_WIDE_HALO_WIDTH="6"This parameter can always be set to "6" regardless of the other grid parameters.
- Define the longitude and latitude (in degrees) of the center of the grid:
- Add the definitions of computational parameters after the definitions of the ESG grid parameters:
- Define the physics time step (in seconds) to use in the forecast model with this grid:
DT_ATMOS="45"This is the largest time step in the model. It is the time step on which the physics parameterizations are called. In general, DT_ATMOS depends on the horizontal resolution of the grid. The finer the grid, the smaller it needs to be to avoid numerical instabilities.
- Define the MPI layout:
LAYOUT_X="16"
LAYOUT_Y="10"These are the number of MPI processes into which to decompose the grid.
- Define the block size. This is a machine-dependent parameter that does not have a default value. For Cheyenne, set it to "32":
BLOCKSIZE="32"
- Define the physics time step (in seconds) to use in the forecast model with this grid:
- Add the definitions of the write-component grid parameters right after the computational parameters. The workflow supports three types of write-component grids: regional lat-lon, rotated lat-lon, and Lambert conformal. Here, we demonstrate how to set up a Lambert conformal grid. (Note that there are scripts to calculate the write-component grid parameters from the ESG grid parameters, but they are not yet in user-friendly form. Thus, here, we use an approximate method to obtain the write-component parameters.) The steps are:
- Define the number of write-component groups (WRTCMP_write_groups) and the number of MPI processes per group (WRTCMP_write_tasks_per_group):
WRTCMP_write_groups="1"
WRTCMP_write_tasks_per_group=$(( 1*LAYOUT_Y ))Each write group consists of a set of dedicated MPI processes that writes the fields on the write-component grid to a file on disk while the forecast continues to run on a separate set of processes. These parameters may have to be increased for grids having more grid points.
- Define the type of write-component grid. Here, we will consider only a Lambert conformal grid:
WRTCMP_output_grid="lambert_conformal" - Define the longitude and latitude (in degrees) of the center of the write-component grid. The most straightforward way to define these is to set them to the coordinates of the center of the native ESG grid:
WRTCMP_cen_lon="${ESGgrid_LON_CTR}"
WRTCMP_cen_lat="${ESGgrid_LAT_CTR}" - Define the first and second standard latitudes associated with a Lambert conformal coordinate system. For simplicity, we set these to the latitude of the center of the ESG grid:
WRTCMP_stdlat1="${ESGgrid_LAT_CTR}"
WRTCMP_stdlat2="${ESGgrid_LAT_CTR}" - Define the grid cell size (in meters) in the x (west-to-east) and y (south-to-north) directions on the write-component grid. We simply set these to the corresponding quantities for the ESG grid:
WRTCMP_dx="${ESGgrid_DELX}"
WRTCMP_dy="${ESGgrid_DELY}" - Define the number of grid points in the x and y directions on the write-component grid. To ensure that the write-component grid lies completely within the ESG grid, we set these to 95% of ESGgrid_NX and ESGgrid_NY, respectively. This gives
WRTCMP_nx="399"
WRTCMP_ny="285" - Define the longitude and latitude (in degrees) of the lower left (southwest) corner of the write-component grid. Approximate values (from a linearization that is more accurate the smaller the horizontal extent of the grid) for these quantities can be obtained using the formulas (for reference):
WRTCMP_lon_lwr_left = WRTCMP_cen_lon - (degs_per_meter/(2*cos_phi_ctr))*WRTCMP_nx*WRTCMP_dx
WRTCMP_lat_lwr_left = WRTCMP_cen_lat - (degs_per_meter/2)*WRTCMP_ny*WRTCMP_dywhere
cos_phi_ctr = cos((180/pi_geom)*WRTCMP_lat_lwr_left)
degs_per_meter = 180/(pi_geom*radius_Earth)Here, pi_geom ≈ 3.14 and radius_Earth ≈ 6371e+3 m. Substituting in these values along with the write-component grid parameters set above, we get
WRTCMP_lon_lwr_left="-120.74"
WRTCMP_lat_lwr_left="33.16"
- Define the number of write-component groups (WRTCMP_write_groups) and the number of MPI processes per group (WRTCMP_write_tasks_per_group):
For simplicity, we leave all other parameters in config.sh the same as in the experiment in Session 2.1. We can now generate a new experiment using this configuration file as follows:
- Ensure that the proper workflow environment is loaded and generate the experiment directory and corresponding rocoto workflow:
source ../../env/wflow_cheyenne.env
cd $USHDIR
./generate_FV3LAM_wflow.sh - After this completes, for convenience, in your shell redefine the variable EXPTDIR to the directory of the current experiment:
EXPT_SUBDIR="test_custom_ESGgrid"
EXPTDIR="${SR_WX_APP_TOP_DIR}/../expt_dirs/${EXPT_SUBDIR}" - Relaunch the workflow using the launch_FV3LAM_wflow.sh script and monitor its status:
cd $EXPTDIR
./launch_FV3LAM_wflow.sh
tail -n 80 log.launch_FV3LAM_wflow - Once all workflow tasks have successfully completed, plot and display the output using a procedure analogous to the one in Session 2.1. (Before doing so, make sure that the shell variable EXPTDIR is set to the directory for this experiment; otherwise, you might end up generating plots for one of the other experiments!)
