Software Containers
Software ContainersEach component of the end-to-end NWP container system has its own software container. There are three options for establishing each image from which the software container will be instantiated:
1. Pull the image from Docker Hub
2. Build the image from scratch
3. Pull the Docker Hub image and convert it to a Singularity image
Instructions are provided for all three options. Please follow the appropriate section that fits your needs.
WPS and WRF NWP Components
WPS and WRF NWP ComponentsThe wps_wrf software container consists of two components:
- WRF Preprocessing System (WPS)
- Advanced Weather Research and Forecasting (WRF-ARW) model
There are three options for establishing the image from which the software container will be instantiated. Please follow the appropriate section below that fits your needs.
Option 1: Pull the dtcenter/wps_wrf image from Docker Hub
If you do not want to build the image from scratch, simply use the prebuilt image by pulling it from Docker Hub.
docker pull dtcenter/wps_wrf:${PROJ_VERSION}
To see what images you have available on your system, type:
Option 2: Build the dtcenter/wps_wrf image from scratch
To access the preprocessing (WPS) and model (WRF) code from the Git repository and build the image from scratch, first go to your project space where you cloned the repository:
and then build an image called dtcenter/wps_wrf:
This command goes into the wps_wrf directory and reads the Dockerfile directives. Please review the contents of the wps_wrf/Dockerfile for additional information.
To see what images you have available on your system, type:
Option 3: Pull the dtcenter/wps_wrf Docker Hub image and convert it to a Singularity image
If you are using Singularity rather than Docker, the commands are similar:
singularity pull docker://dtcenter/wps_wrf:${PROJ_VERSION}_for_singularity
Unlike Docker, Singularity does not keep track of available images in a global environment; images are stored in image files with the .sif extension. Use the `ls` command to see the image file you just downloaded
GSI data assimilation
GSI data assimilationThe GSI container will be used to perform data assimilation. There are three options for establishing the image from which the software container will be instantiated. Please follow the appropriate section below that fits your needs.
Option 1: Pull the dtcenter/gsi image from Docker Hub
If you do not want to build the image from scratch, simply use the prebuilt image by pulling it from Docker Hub.
docker pull dtcenter/gsi:${PROJ_VERSION}
To see what images you have available on your system, type:
Option 2: Build the dtcenter/gsi image from scratch
To access the GSI container from the Git repository and build the image from scratch, first go to your project space where you cloned the repository:
and then build an image called dtcenter/gsi:
This command goes into the gsi directory and reads the Dockerfile directives. Please review the contents of the gsi/Dockerfile for additional information.
To see what images you have available on your system, type:
OPTION 3: Pull the dtcenter/gsi Docker Hub image and convert it to a Singularity image
If you are using Singularity rather than Docker, the commands are similar:
singularity pull docker://dtcenter/gsi:${PROJ_VERSION}
Unlike Docker, Singularity does not keep track of available images in a global environment; images are stored in image files with the .sif extension. Use the `ls` command to see the image file you just downloaded
-rwxr-xr-x 1 ec2-user ec2-user 879349760 Sep 29 20:49 wps_wrf_${PROJ_VERSION}.sif
Unified Post Processor (UPP)
Unified Post Processor (UPP)The UPP container will be used to post process the model output. There are three options for establishing the image from which the software container will be instantiated. Please follow the appropriate section below that fits your needs.
Option 1: Pull the dtcenter/upp image from Docker Hub
If you do not want to build the image from scratch, simply use the prebuilt image by pulling it from Docker Hub.
docker pull dtcenter/upp:${PROJ_VERSION}
To see what images you have available on your system, type:
Option 2: Build the dtcenter/upp image from scratch
To access the UPP container from the Git repository and build the image from scratch, first go to your project space where you cloned the repository:
and then build an image called dtcenter/upp:
This command goes into the upp directory and reads the Dockerfile directives. Please review the contents of the upp/Dockerfile for additional information.
To see what images you have available on your system, type:
Option 3: Pull the dtcenter/upp Docker Hub image and convert it to a Singularity image
If you are using Singularity rather than Docker, the commands are similar:
singularity pull docker://dtcenter/upp:${PROJ_VERSION}
Unlike Docker, Singularity does not keep track of available images in a global environment; images are stored in image files with the .sif extension. Use the `ls` command to see the image file you just downloaded
-rwxr-xr-x 1 ec2-user ec2-user 1224183808 Sep 29 21:04 upp_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 879349760 Sep 29 20:49 wps_wrf_${PROJ_VERSION}.sif
Python Graphics
Python GraphicsThe Python container will be used to plot model forecast fields. There are three options for establishing the image from which the software container will be instantiated. Please follow the appropriate section below that fits your needs.
Option 1: Pull the dtcenter/python image from Docker Hub
If you do not want to build the image from scratch, simply use the prebuilt image by pulling it from Docker Hub.
docker pull dtcenter/python:${PROJ_VERSION}
To see what images you have available on your system, type:
Option 2: Build the dtcenter/python image from scratch
To access the Python scripts from the Git repository and build the image from scratch, first go to your project space where you cloned the repository:
and then build an image called dtcenter/python:
This command goes into the python directory and reads the Dockerfile directives. Please review the contents of the python/Dockerfile for additional information.
To see what images you have available on your system, type:
Option 3: Pull the dtcenter/python Docker Hub image and convert it to a Singularity image
If you are using Singularity rather than Docker, the commands are similar:
singularity pull docker://dtcenter/python:${PROJ_VERSION}
Unlike Docker, Singularity does not keep track of available images in a global environment; images are stored in image files with the .sif extension. Use the `ls` command to see the image file you just downloaded
-rwxr-xr-x 1 ec2-user ec2-user 646881280 Sep 29 21:28 python_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 1224183808 Sep 29 21:04 upp_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 879349760 Sep 29 20:49 wps_wrf_${PROJ_VERSION}.sif
MET Verification
MET VerificationVerification of the model fields will be performed using the Model Evaluation Tools (MET) container. There are three options for establishing the image from which the software container will be instantiated. Please follow the appropriate section below that fits your needs.
Option 1: Pull the dtcenter/nwp-container-met image from Docker Hub
If you do not want to build the image from scratch, simply use the prebuilt image by pulling it from Docker Hub.
docker pull dtcenter/nwp-container-met:${PROJ_VERSION}
To see what images you have available on your system, type:
Option 2: Build the dtcenter/nwp-container-met image from scratch
To access the Model Evaluation Tools (MET) code from the Git repository and build the verification software from scratch, first go to your project space where you cloned the repository:
and then build an image called dtcenter/nwp-container-met:
This command goes into the met/MET directory and reads the Dockerfile directives. Please review the contents of the met/MET/Dockerfile for additional information.
To see what images you have available on your system, type:
Option 3: Pull the dtcenter/met Docker Hub image and convert it to a Singularity image
If you are using Singularity rather than Docker, the commands are similar:
singularity pull docker://dtcenter/nwp-container-met:${PROJ_VERSION}
Unlike Docker, Singularity does not keep track of available images in a global environment; images are stored in image files with the .sif extension. Use the `ls` command to see the image file you just downloaded
-rwxr-xr-x 1 ec2-user ec2-user 1045712896 Sep 29 21:34 nwp-container-met_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 646881280 Sep 29 21:28 python_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 1224183808 Sep 29 21:04 upp_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 879349760 Sep 29 20:49 wps_wrf_${PROJ_VERSION}.sif
METviewer Database and Display
METviewer Database and DisplayOnce verification scores for each valid time have been computed in the MET container, the METviewer database can be loaded and the user interface can be launched in your web-browser to compute summary statistics and display them. There are two options for establishing the image from which the software container will be instantiated. Please follow the appropriate section below that fits your needs.
Option 1: Pull the dtcenter/metviewer image from Docker Hub
If you do not want to build the image from scratch, simply use the prebuilt image by obtaining the tar file and loading it.
docker pull dtcenter/nwp-container-metviewer:${PROJ_VERSION}
If you followed all the instructions up to this point in the tutorial, the output should look similar to this:
REPOSITORY | TAG | IMAGE ID | CREATED | SIZE |
dtcenter/wps_wrf | 3.5.1 | 9b2f58336cf9 | 3 hours ago | 3.22GB |
dtcenter/nwp-container-met | 3.5.1 | 9e3d0e924a30 | 3 hours ago | 5.82GB |
dtcenter/upp | 3.5.1 | 82276dfccbc3 | 4 hours ago | 3.36GB |
dtcenter/nwp-container-metviewer | 3.5.1 | f2095355127f | 4 hours ago | 2.8GB |
dtcenter/python | 3.5.1 | 7ea61e625dff | 4 hours ago | 1.28GB |
dtcenter/gsi | 3.5.1 | c6bebdbfb674 | 5 hours ago | 2.88GB |
Option 2: Build the dtcenter/metviewer image from scratch
To access the METviewer database and display system from the Git repository and build the verification software from scratch, first go to your project space where you cloned the repository:
and then build an image called dtcenter/metviewer:
This command goes into the metviewer/METviewer directory and reads the Dockerfile directives. Please review the contents of the metviewer/METviewer/Dockerfile for additional information.
To see what images you have available on your system, type:
Option 3: Pull the dtcenter/metviewer Docker Hub image and convert it to a Singularity image
If you are using Singularity rather than Docker, the commands are similar:
singularity pull docker://dtcenter/nwp-container-metviewer-for-singularity:${PROJ_VERSION}
Unlike Docker, Singularity does not keep track of available images in a global environment; images are stored in image files with the .sif extension. Use the `ls` command to see the image file you just downloaded
-rwxr-xr-x 1 ec2-user ec2-user 1045712896 Sep 29 21:34 nwp-container-met_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 1045712896 Sep 29 21:34 nwp-container-metviewer-for-singularity_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 646881280 Sep 29 21:28 python_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 1224183808 Sep 29 21:04 upp_${PROJ_VERSION}.sif
-rwxr-xr-x 1 ec2-user ec2-user 879349760 Sep 29 20:49 wps_wrf_${PROJ_VERSION}.sif