Download and Build

carson Mon, 03/29/2021 - 15:01

Download and Build

Defining your workspace

The first step is to create an environment variable that contains the path to the directory in which you will build and run the SCM.

Go to the directory in which you will build and run the SCM and run the command below substituting /path/to/my/build/and/run/directory with your actual directory name

cd /path/to/my/build/and/run/directory

Next, define an environment variable that contains the path

For C-Shell:

setenv SCM_WORK `pwd`

For Bourne Shell:

export SCM_WORK=`pwd`

Determining your platform

In order to build the SCM, you need to have either a computational platform that meets certain system requirements and has certain pre-installed libraries and tools or use a Docker container. Please follow one of the options below to build the SCM.

Option 1 Using a pre-configured platform

Option 1. Using a computational platform with the required software stack

In order to build and run the SCM, your platform needs to have the system software listed below. The versions we have tested with are listed below

GNU compilers 12.1.0, cmake 3.23.2, NetCDF 4.7.4, Python 3.9.12
GNU compilers 10.1.0, cmake 3.16.4, NetCDF 4.8.1, Python 3.7.12
GNU compilers 11.1.0, cmake 3.18.2, NetCDF 4.8.1, Python 3.8.5
Intel compilers 2022.0.2, cmake 3.20.1, NetCDF 4.7.4, Python 3.7.11
Intel compilers 2022.1.0, cmake 3.22.0, NetCDF 4.8.1, Python 3.7.12

The following Python modules are required: xarray, dask, netCDF4, bottleneck, matplotlib, f90nml, configobj, shapely, pandas, and scipy.

In addition, there are several utility libraries as part of the HPC-Stack package that must be installed.

bacio - Binary I/O Library
sp - Spectral Transformation Library
w3nco - GRIB decoder and encoder library

Obtaining the code

The source code for the CCPP and SCM is provided through GitHub.com. This tutorial accompanies the v6.0.0 release tag, which contains the tested and supported version for general use.

Clone the the v6.0.0 release code using

cd $SCM_WORK

git clone --recursive -b release/public-v6 https://github.com/NCAR/ccpp-scm

The recursive option in this command clones the release/public-v6 branch of the NCAR authoritative SCM repository (ccpp-scm) and all subrepositories (ccpp-physics, ccpp-framework, and CMakeModules).

Setting up the environment in a preconfigured platform

Computational platforms that meet the system requirements and in which the HPS-Stack are prebuilt and installed in a central location are referred to as preconfigured platforms. Examples of preconfigured platforms are the Hera NOAA high-performance computing machine (using the Intel compiler) and the NCAR Cheyenne system (using the Intel and GNU compilers).

Platform-specific scripts are provided to load modules and set the user environment for preconfigured platforms. These scripts load compiler modules, the NetCDF module, Python environment, etc. and set compiler and HPC-Stack environment variables.

Go to the top-level code directory (ccpp-scm)

cd $SCM_WORK/ccpp-scm

Now source the correct script for your platform and shell.

For t/csh shells,

source scm/etc/Hera_setup_intel.csh
source scm/etc/Cheyenne_setup_gnu.csh
source scm/etc/Cheyenne_setup_intel.csh

For bourne/bash shells,

. scm/etc/Hera_setup_intel.sh
. scm/etc/Cheyenne_setup_gnu.sh
. scm/etc/Cheyenne_setup_intel.sh

Setting up the environment in a non-preconfigured platform

If you are not using a preconfigured platform, you need to install HPC-Stack yourself following the instructions found in Section 2.2.2 of the CCPP SCM User and Technical Guide v6-0-0.

After performing the installation and setting environment variables bacio_ROOT, sp_ROOT, and w3nco_ROOT to the location where HPC-Stack is installed, continue following the instructions in this tutorial.

Staging additional datasets

You need to download the lookup tables (large binaries, 324 MB) for the Thompson microphysics package and place them in the correct directory.

cd $SCM_WORK/ccpp-scm

./contrib/get_all_static_data.sh

./contrib/get_thompson_tables.sh 

Building the code

Following the commands below, you will run cmake to query system parameters, execute the CCPP pre-build script to match the physics variables (between what the host model – SCM – can provide and what is needed by physics schemes in the CCPP), and build the physics caps needed to use them. Subsequently, you will run make to build the SCM executable.

cd $SCM_WORK/ccpp-scm/scm

mkdir -p bin && cd bin

cmake ../src

make -j4

A successful build will produce executable file $SCM_WORK/ccpp-scm/scm/bin/scm

carson Mon, 03/29/2021 - 15:06

Option 2 Using a Docker Container

Option 2. Using a Docker container

In order to run a precompiled version of the CCPP SCM in a container, Docker needs to be available on your machine. Please visit https://www.docker.com to download and install the version compatible with your system. Docker frequently releases updates to the software; it is recommended to apply all available updates. NOTE: In order to install Docker on your machine, you need to have root access privileges. More information about getting started can be found at https://docs.docker.com/get-started and in Section 2.5 of the CCPP SCM User and Technical Guide v6-0-0.

In the first exercise you will use a prebuilt Docker image available on Docker Hub. In subsequent exercises you will use the same image to rebuild your own SCM executable with various modifications.

Using a prebuilt Docker image

A prebuilt Docker image is available on Docker Hub. In order to use this, execute the following from the terminal where Docker is run:

docker pull dtcenter/ccpp-scm:v6.0.0

To verify that it exists afterward, run the command:

docker images

Proceed to “Set up the Docker image”.

Set up the Docker image

Next, you will set up an output directory so that output generated by the SCM and its scripts within the container can be used outside of the container using the following steps:

Set up a directory that will be shared between the host machine and the Docker container. When set up correctly, it will contain output generated by the SCM within the container for manipulation by the host machine. For Mac/Linux,

mkdir -p /path/to/output

For Windows, you can try to create a directory of your choice to mount to the container, but it may not work or may require more configuration, depending on your particular Docker installation. We have found that Docker volume mounting in Windows can be difficult to set up correctly. One method that worked for us was to create a new directory under our local user space, and specify the volume mount as below.

/c/Users/my_username/path/to/directory/to/mount

In addition, with Docker Toolbox, double check that the mounted directory has correct permissions. For example, open VirtualBox, right click on the

running virtual machine, and choose “Settings”. In the dialog that appears, make sure that the directory you’re trying to share shows up in “Shared Folders" (and add it if it does not) and make sure that the “auto-mount" and “permanent" options are checked.

Set an environment variable to point to the directory that was set up in the previous step. For t/csh shells:

setenv OUT_DIR /path/to/output

For bourne/bash shells:

export OUT_DIR=/path/to/output

To use the SCM in the container interactively, run non-default configurations, create plots, or even develop code, issue the following command:

docker run --rm -it -v ${OUT_DIR}:/home --name run-ccpp-scm dtcenter/ccpp-scm:v6.0.0 /bin/bash

Note that this command will not actually run the SCM, but will put you within the container space and within the bin directory of the SCM with a pre-compiled executable. At this point, you can run the scripts as described in the following sections.

A couple things to note when using a container:

The run scripts should be used with the -d option if output is to be shared with the host machine
Since the container is ephemeral, if you do development you should push your changes to a remote git repository to save them (i.e. a fork on GitHub.com).

carson Mon, 03/29/2021 - 15:30