An AMI (Amazon Machine Image) is provided to you with:

• All prerequisites installed (compiler, netCDF, NCEPlibs, Python packages)

• A sandpit directory with the following subdirectories:

  • A data directory with 

    • Lookup tables for the Thompson microphysics (qr_acr_qg.dat, qr_acr_qs.dat, and freezeH2O.dat)

    • File source_mods.tar with modified code for one of the exercises

  • A source directory with 

    • Single column mode code (public_release branch of gmtb-scm repository) and its CCPP submodules (ufs_public_release branches of ccpp-framework and ccpp-physics repositories)

    • Single Column Model executable gmtb-scm previously created in the bin directory

Setup and configure the AWS connection:

In order to setup the AMI, download and untar the following file on your local machine, and configure the connection to AWS.

Download the following file to your local machine:

https://dtcenter.org/GMTB/AWS/aws.tar

This file will be downloaded into a local directory, depending on your system and browser.  Downloads, Documents, Desktop, etc.

Open a terminal window on your system (PuTTy, iTerm, Terminal, etc)

Source one of the scripts:

This script will prompt you to enter a Group Number. This has been provided to you today.

Next, connect to the AWS instance, using the following commands:

At this point, you will be logged into a Linux AWS instance, with the SCM pre-installed and configured for this short course.

Thanks to the Joint Center for Satellite Data Assimilation (JCSDA) for providing the infrastructure to run the instances in the Amazon Web Services (AWS) cloud.

Review the physics suites

We’d like to run several experiments with the SCM, showing differences among 3 physics suites for two different meteorological regimes. Let’s first take a look at the composition of the physics suites and then how the two cases are set up.

Navigate to the directory where available suite definition files are stored and see what is available:

We’re interested in the following suites: suite_SCM_GFS_v15p2.xml, suite_SCM_GSD_v1.xml, and suite_SCM_csawmg.xml. 

Notice the XML format, paying attention to the key elements: suite (with name attribute), group, subcycle, scheme.

You will find primary and interstitial schemes.  All suites share many common elements, particularly interstitial schemes that are required to prepare data for schemes and calculate diagnostics expected in a GFS run, since they were constructed by modifications to the original GFS operational suite. 

Navigate to the directory where case setup namelists are stored to see what we’re going to run.

Look at twpice.nml to see what kinds of information are required to set up a case. The variables most likely to differ among cases are: case_name, dt, runtime, [thermo,mom]_forcing_type, sfc_flux_spec, year, month, day, hour.

Each case also has a netCDF file associated with it (determined by the case_name variable) that contains the initial conditions and forcing data for the case. Take a look at one of the files to see what kind of information is expected:

There are groups for scalars (single values relevant for the simulation), initial conditions (function of vertical level only), and forcing (function of time and/or vertical level). Note that not all forcing variables are required to be non-zero.

Run the SCM

Now that we have an idea what the physics suites contain and how to set up cases, let’s use the run script called multi_run_gmtb_scm.py which will run several instances of the SCM serially.

If invoked by itself, this script will run through all permutations of supported suites and cases. If you provide only a case, it will run all supported suites for that case. If you provide only a suite, it will run all supported cases for that suite. Alternatively, you can provide the script with a file that defines which cases, suites, and associated namelists to use. We will use this final option.

Navigate to the run directory, and look at the multi_run setup file:

Exit the less command with q.

This file just defines a few Python lists. We’ll be running the TWP-ICE maritime deep convection case and the LASSO continental shallow convection case from May 18, 2016 with the suites discussed previously. The namelists variable is optional if the suites have dictionary entries in ../src/default_namelists.py. This would also be the place to specify non-default namelists to change parameters or physics options.

Before running the SCM, copy the lookup tables used by the Thompson microphysics so they are available at runtime.

Run:

Running should take a couple minutes with progress displayed on the console.

Let’s analyze the results

The output directory for each case is controlled by its case configuration namelist. For this exercise, a directory containing the output for each integration is placed in the bin directory. You may now inspect that each of these output directories (e.g. bin/output_$CASE_$SUITENAME) now contains a populated output.nc file.

At this point, one could use whatever plotting tools they wish to examine the output. For this course, we will use the basic Python scripts that are included in the repository. The script is called gmtb_scm_analysis.py and expects a configuration file as an argument. The configuration file specifies which output files to read, which variables to plot, and how to do so.

Run the following to generate plots for the LASSO case:

The script creates a new directory within the bin directory called plots_LASSO_2016051812/comp/full. 

Open plot files for variables, such as mean profiles of water vapor specific humidity, cloud water mixing ratio, temperature, and temperature tendencies due to forcing and physics processes.

Despite using very different physics suites, there is relatively little difference in the results. 

TWP-ICE maritime deep convection case

Let’s move on to the TWP-ICE maritime deep convection case. Run the following to generate plots for this case:

While this case spans periods of clear skies, shallow convection, and deep convection, the plots represent time means over the actively deep convection period of the simulation. Let’s focus on the mean temperature profile, its bias (difference from observations), and the temperature tendencies due to forcing and physics processes.

profiles_mean_T.png
profiles_bias_T.png
profiles_mean_multi_T_forcing.png

The mean temperature profile by itself doesn’t provide much information since the differences among the suites is much smaller than the magnitude of the quantity being plotted, but the profiles begin to hint at differences. The bias profile is more instructive in this case. One suite clearly has a cool bias throughout most of the atmosphere, while the remaining two suites have a similar magnitude cool bias, but are closer to zero bias at different heights in the column.

Modify a scheme

Let’s modify a scheme and re-run an experiment to see if we can obtain a reduction in the temperature bias profile. For this exercise, we will concentrate on the suite with the largest temperature bias (csawmg) and the most obvious culprit, the deep convection scheme. Further, assume that we are the laziest physics developer in the world and we will “solve” the temperature bias problem through brute force.

NOTE: the following steps have been provided for you in the modified source code instructions at the end of this page, but are described here for your information.

Looking at the suite definition file for the SCM_csawmg suite (file gmtb-scm/ccpp/suites/suite_SCM_csawmg.xml), we see that the deep convection scheme is called “cs_conv” (Chikira-Sugiyama convection). Its top-level CCPP entry-point subroutine (cs_conv_run) is found in the file gmtb-scm/ccpp/physics/physics/cs_conv.F90. The call to CS_CUMLUS produces tendencies of the state variables, including temperature, and they are applied in the code shortly thereafter.

We will define a new variable used to multiply the temperature tendency before it is applied to the temperature state. Rather than defining a local variable, we’ll turn it into a “tuning knob” by making it a namelist variable and pass it in to the scheme through the CCPP framework. Hint: We can assume that this variable does not already exist in any supported CCPP scheme or in the list of variables supplied by the host SCM for this course, although in practice, one should check to see if an appropriate variable already exists for our use.

The implementation begins with straight Fortran code. The variable is added to the argument list, declared as in INTENT(IN) scalar real, multiplied by the temperature tendency variable at the appropriate place in the code.

Since we modified the interface by adding an argument, we will need to add CCPP metadata and “advertise” that the scheme requires the new variable. Notice that the new variable has been added (in the right order) in the metadata for the subroutine cs_conv_run in cs_conv.meta.

At this point, IF the new variable already existed in the SCM’s list of CCPP-available variables, we would be done! After rerunning cmake to regenerate the software caps and make to compile, the modified code could be run. In this case, we will need to add the variable to the SCM and “advertise” that it is available for CCPP schemes to use. For the SCM, model control variables are contained within the GFS_control_type derived datatype found in the gmtb-scm/scm/src/GFS_typedefs.F90, so we will add the new variable (global_ttend_mult) there. Note that it is not necessary for the local variable names to match between the host and the physics scheme -- the CCPP standard name and other metadata needs to match, however. This derived datatype is initialized with a default value (1.0) in the control_intialize subroutine in the same file, and read in from the same physics namelist as other physics variables, potentially overriding the default.

Once the Fortran has been added, we will need to edit the SCM’s metadata by adding the new variable’s information in the appropriate place in gmtb-scm/scm/src/GFS_typedefs.meta (note that each derived datatype has its own section beginning with [ccpp-arg-table] in this file).

This completes the addition of a new variable on the physics side and the SCM host side. 

Using the default namelist for the SCM_csawmg suite would produce identical results at this point, so we’ll replicate the existing namelist for this suite with a non-default value for the new variable (global_ttend_mult = 1.15). See gmtb-scm/ccpp/physics_namelists/input_csawmg_short_course.nml.

Rather than having you implement these changes yourself, they have been provided to you in a tar file. Using the instructions below, extracting the tar file will put the modified files in the correct directories and update their timestamps so that cmake/make will know to recompile them and their dependencies. Note: This can be undone by running git checkout . on the top-level gmtb-scm directory and the ccpp/physics subdirectory.

Re-run the updated code and case

Finally, re-run the TWP-ICE case with the modified code and namelist (NOTE: this command is all one line, your browser may split the line, be sure to copy/paste it into one command!):

Let’s see how we did!

Re-run the plotting script using a new plot configuration file to plot all 4 runs of the TWP-ICE case.

Look in plots_twpice_short_course/comp/active. The temperature bias profile plot  (profiles_bias_T.png; also supplemental slide 32)  shows the 3 original suites alongside the modified csawmg suite (labeled csawmg+) on the plot. For this case, we have definitely improved the performance of this suite for this metric by brute force!

Open the temperature tendencies plot (profiles_mean_multi_T_forcing.png; also supplemental slide 33).

Run another case

Physics suites need to be general enough to work everywhere. Let’s try a different deep convection case to see whether our modification shows promise. The arm_sgp_summer_1997_A case features continental deep convection over the central US plains. We’ll run the same 3 suites through this case plus the modified suite using the same method as above (from the bin directory):

Once completed, run the plotting script on the new output:

The output is placed in bin/plots_arm_short_course. 

Open the temperature bias plot (profiles_bias_T.png; also supplemental slide 35).

Source code and compilation

The source code for the Single Column Model and the CCPP Physics are hosted on GitHub in public repositories.  For this Short Course, these have been pre-installled for you.  You can download the code yourself on your own machine as follows:

The Users Guide for the SCM describes the required pre-requisite compilers and libraries.  These MUST be installed (or already available) on your system BEFORE attempting to compile the SCM.  Install any necessary pre-requisite compilers, MPI libraries, etc, as described in the Users Guide, then compile the SCM as follows:

This will create an executable, gmtb_scm, in the current working directory. 

CCPP website

SCM v3+ User's Guide

CCPP v3 Technical Documentation

CCPP v3 Scientific Documentation

CCPP user forum

Selected AMS 2020 presentations involving CCPP and the Single-Column Model

• Tuesday 01/14, 03:15 PM - 03:30 PM. Room 257AB. 
  • 6A.2 Combining the Common Community Physics Package with a Single-Column Model to Drive NWP Physics Advancements. Grant J. Firl, NCAR and the Developmental Testbed Center, Boulder, CO; and D. Heinzeller, L. Xue, and L. Bernardet. 
• Tuesday 01/14, 04:00 PM - 06:00 PM. Hall B.
  • 646 An Evaluation of Common Community Physics Package (CCPP) Physics Suites across Scales. Kathryn M. Newman, NCAR, Boulder, CO; and T. J. Hertneky, E. A. Kalina, M. Harrold, L. Pan, G. Firl, E. D. Grell, L. Carson, and M. Ek.
  • 647 One-Stop Shopping for Physics across Scales: From a Single-Column Model to Three-Dimensional Configurations for Weather and S2S. Linlin Pan, NOAA/GSD, Univ. of Colorado/CIRES, and Developmental Testbed Center, Boulder, CO; NOAA, Boulder, CO; and L. Bernardet, D. Heinzeller, E. Kalina, G. Firl, E. Grell, K. Newman, L. Carson, and G. Grell
• Wednesday 01/15, 03:15 PM - 03:30 PM. Room 251.
  • 11B.2 The Common Community Physics Package (CCPP): Unifying Physics across NOAA and NCAR Models Using a Common Software Framework. Dom Heinzeller, NOAA/ESRL/GSD, and Univ. of Colorado/CIRES, and Developmental Testbed Center, Boulder, CO; and G. J. Firl, L. Bernardet, L. Carson, M. Zhang, S. Goldhaber, C. Craig, D. Gill, M. Duda, and F. M. Vitt.
• Thursday 01/16 11:00 AM - 11:15 AM. Room 257AB.
  • 12A.3 Physics Interoperability as a Strategy for Advancing NOAA’s Unified Forecast System Physics Suites Ligia Bernardet NOAA/GSD, and Developmental Testbed Center, Boulder, CO; and G. J. Firl, D. Heinzeller, L. Carson, M. Zhang, J. Schramm, and L. Nance.