METplus: Initial setup

Prerequisites: Software

The Requirements section in the Software Installation chapter of the METplus User's Guide lists the software and Python packages that are required to run the METplus wrappers. Note that there is a core set of requirements needed to run the METplus wrappers and additional requirements needed to utilize some of the more advanced features.

Prerequisites: Environment

This tutorial assumes MET 11.0.1 and METplus 5.0.1 have been installed on the machine used to run the exercises and that the sample input data is also available. If this is not the case, please navigate to the Software Installation chapter of the METplus User's Guide.

The following instructions are required so the commands in this tutorial can be copied and run without modification. The steps involve creating a working directory to store files used/generated by the tutorial and configuring a simple script that can be run to set up the shell environment. Once configured correctly, the script can be run upon returning to the tutorial content to easily resume progress.

Click on the link that corresponds to the environment you are setting up.

If you are running the tutorial instructions on your own computer, select the bash or csh instructions depending on which shell you prefer. We recommend using bash if you do not have a preference.

Pre-Configured Environments

Setting up the Tutorial Environment on Hera (NOAA)

Setting up the Tutorial Environment on Jet (NOAA)

Setting up the Tutorial Environment on Cheyenne (NCAR)

Setting up the Tutorial Environment on Seneca (NCAR)

User Configured Environments

Setting up the Tutorial Environment (bash)

Setting up the Tutorial Environment (csh)

admin Mon, 06/24/2019 - 15:59

Setting up the Tutorial Environment on Cheyenne (NCAR)

The following instructions should be run if configuring the shell environment to run the METplus Tutorial on Cheyenne (NCAR). If you are running on your own computer or a NOAA machine that has been set up to run the tutorial, please go back and click the appropriate link for those instructions.

Create a Working Directory

Create a directory called 'METplus-5.0.0_Tutorial' to hold all of the files you will create during the tutorial. This can be any directory that you have write permission.

In the following instructions, change "/path/to" to the directory you chose: EDIT AFTER COPYING and BEFORE HITTING RETURN!

Change directory to the location where you want to put your tutorial files.

cd /path/to

Create a directory called METplus-5.0.0_Tutorial.

This directory will contain all of your tutorial work, including configuration files, output data, and any other notes you'd like to keep.

mkdir METplus-5.0.0_Tutorial

Change directory into the tutorial directory.

cd METplus-5.0.0_Tutorial

Create Directories for Configuration Files and Output Data

mkdir user_config

mkdir output

Obtain the Tutorial Setup Script

Copy the tutorial setup shell script to configure your runtime environment for use with METplus. You will also copy over a METplus .conf file that you will use to run:

cp /glade/p/ral/jntp/MET/METplus/Tutorial_Files/METplus-5.0.0_Tutorial_Files/tutorial-5.0.0.conf ./tutorial.conf

Run the command that corresponds your default shell.
DO NOT RUN BOTH SETS OF COMMANDS.
If you are unsure which shell you are using, run the command "echo $SHELL" to check.

If you are using bash:

cp /glade/p/ral/jntp/MET/METplus/Tutorial_Files/METplus-5.0.0_Tutorial_Files/METplus-5.0.0_TutorialSetup.cheyenne.bash ./METplus-5.0.0_TutorialSetup.sh

If you are using csh:

cp /glade/p/ral/jntp/MET/METplus/Tutorial_Files/METplus-5.0.0_Tutorial_Files/METplus-5.0.0_TutorialSetup.cheyenne.csh ./METplus-5.0.0_TutorialSetup.sh

Next navigate to the Verify Environment is Set Correctly page.

mccabe Thu, 07/29/2021 - 09:50

Setting up the Tutorial Environment on Jet (NOAA)

The following instructions should be run if configuring the shell environment to run the METplus Tutorial on Jet (NOAA). If you are running on your own computer or an NCAR machine that has been set up to run the tutorial, please go back and click the appropriate link for those instructions.

Create a Working Directory

Create a directory called 'METplus-5.0.0_Tutorial' to hold all of the files you will create during the tutorial. This can be any directory that you have write permission.

In the following instructions, change "/path/to" to the directory you chose: EDIT AFTER COPYING and BEFORE HITTING RETURN!

Change directory to the location where you want to put your tutorial files.

cd /path/to

Create a directory called METplus-5.0.0_Tutorial. This directory will contain all of your tutorial work, including configuration files, output data, and any other notes you'd like to keep.

mkdir METplus-5.0.0_Tutorial

Change directory into the tutorial directory.

cd METplus-5.0.0_Tutorial

Create Directories for Configuration Files and Output Data

mkdir user_config

mkdir output

Obtain the Tutorial Setup Script

Copy the tutorial setup shell script to configure your runtime environment for use with METplus. You will also copy over a METplus .conf file that you will use to run:

cp /lfs1/HFIP/dtc-hurr/METplus/METplus-5.0.0_Tutorial_Files/METplus-5.0.0_TutorialSetup.jet.`basename $SHELL` ./METplus-5.0.0_TutorialSetup.sh

cp /lfs1/HFIP/dtc-hurr/METplus/METplus-5.0.0_Tutorial_Files/tutorial-5.0.0.conf ./tutorial.conf

Next navigate to the Verify Environment is Set Correctly page.

mccabe Wed, 11/17/2021 - 13:07

Setting up the Tutorial Environment on Seneca (NCAR)

The following instructions should be run if configuring the shell environment to run the METplus Tutorial on Seneca (NCAR). If you are running on your own computer or a NOAA machine that has been set up to run the tutorial, please go back and click the appropriate link for those instructions.

Create a Working Directory

Create a directory called 'METplus-5.0.0_Tutorial' to hold all of the files you will create during the tutorial. This can be any directory that you have write permission.

In the following instructions, change "/path/to" to the directory you chose: EDIT AFTER COPYING and BEFORE HITTING RETURN!

Change directory to the location where you want to put your tutorial files.

cd /path/to

Create a directory called METplus-5.0.0_Tutorial.

This directory will contain all of your tutorial work, including configuration files, output data, and any other notes you'd like to keep.

mkdir METplus-5.0.0_Tutorial

Change directory into the tutorial directory.

cd METplus-5.0.0_Tutorial

Create Directories for Configuration Files and Output Data

mkdir user_config

mkdir output

Obtain the Tutorial Setup Script

Copy the tutorial setup shell script to configure your runtime environment for use with METplus. You will also copy over a METplus .conf file that you will use to run:

cp /d1/projects/METplus/METplus-5.0.0_Tutorial_Files/tutorial-5.0.0.conf ./tutorial.conf

cp /d1/projects/METplus/METplus-5.0.0_Tutorial_Files/METplus-5.0.0_TutorialSetup.seneca.`basename $SHELL` ./METplus-5.0.0_TutorialSetup.sh

(OPTIONAL) Set up conda Environment for METplus Analysis and s2s Tutorial Sections

Due to the need for external package dependencies for the METplus Analysis and s2s sections, the following instructions are also necessary. If you do not plan on executing the commands of those sections, you can skip these steps.

First, ensure that you are running the bash environment:

bash

Now, verify that you have a .condarc file in your /home/user directory. If the file does not exist, create one. In that file, you will need the following entries, in the exact same format (dashes and colons):

channels:
- defaults
- conda-forge
- anaconda

Verify that you are using the most recent version of anaconda3 in your .bashrc file. Verify that your conda_setup in your .bashrc file is using /usr/local/anaconda3/bin/conda. You should see a code block similar to following:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/usr/local/anaconda3/bin/conda' 'shell.bash' 'hook'2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/usr/local/anaconda3/etc/profile.d/conda.sh" ]; then
. "/usr/local/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/usr/local/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

If you do have to make any changes to your .bashrc file, it would be best at this point to sign out, then sign back into your Seneca connection from the terminal. Errors have been found to happen even after sourcing your .bashrc file for the updated content.

From the command line, enter the following:

/usr/local/anaconda3/bin/conda create -n metplus_analysis_tutorial_env python=3.8

mamba -y

If this was successful, you will see the following at the end of the output streamed to the screen:

Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate metplus_analysis_tutorial_env
#
# To deactivate an active environment, use
#
# $ conda deactivate

This creates a basic Python 3.8 conda environment named ‘metplus_analysis_tutorial_env’ that contains mamba (to accelerate installing the remaining Python packages).

Activate this environment by running the following from the command line:

conda activate metplus_analysis_tutorial_env

Install the following packages to the conda environment by running the following from the command line (using mamba to speed up the installation):

mamba install imageio=2.19.3 imutils=0.5.4 matplotlib=3.5.2 metpy=1.3.1 netcdf4=1.6.2 numpy=1.24.2 pandas=1.5.1 pint=0.19.2 plotly=5.9.0 pytest=7.2.1 python-kaleido=0.2.1 pyyaml=6.0 scikit-image=0.19.3 scipy=1.8.1 lxml=4.9.1 eofs=1.4.0 scikit-learn=1.2.0 -y

You will see many lines of information streaming to your screen. You will see following output to your screen if everything was successful:

Preparing transaction: done
Verifying transaction: done
Executing transaction: /
Installed package of scikit-learn can be accelerated using scikit-learn-intelex.
More details are available here: https://intel.github.io/scikit-learn-intelex

For example:
$ conda install scikit-learn-intelex
$ python -m sklearnex my_application.py
done

Now you are ready to run the METplus Analysis tutorials.

Next navigate to the Verify Environment is Set Correctly page.

mccabe Wed, 06/30/2021 - 12:24

Setting up the Tutorial Environment on Hera (NOAA)

The following instructions should be run if configuring the shell environment to run the METplus Tutorial on Hera (NOAA). If you are running on your own computer or an NCAR machine that has been set up to run the tutorial, please go back and click the appropriate link for those instructions.

Create a Working Directory

Create a directory called 'METplus-5.0.0_Tutorial' to hold all of the files you will create during the tutorial. This can be any directory that you have write permission.

In the following instructions, change "/path/to" to the directory you chose: EDIT AFTER COPYING and BEFORE HITTING RETURN!

Change directory to the location where you want to put your tutorial files.

cd /path/to

Create a directory called METplus-5.0.0_Tutorial. This directory will contain all of your tutorial work, including configuration files, output data, and any other notes you'd like to keep.

mkdir METplus-5.0.0_Tutorial

Change directory into the tutorial directory.

cd METplus-5.0.0_Tutorial

Create Directories for Configuration Files and Output Data

mkdir user_config

mkdir output

Obtain the Tutorial Setup Script

Copy the tutorial setup shell script to configure your runtime environment for use with METplus. You will also copy over a METplus .conf file that you will use to run:

cp /scratch1/BMC/dtc/METplus/METplus-5.0.0_Tutorial_Files/METplus-5.0.0_TutorialSetup.hera.`basename $SHELL` ./METplus-5.0.0_TutorialSetup.sh

cp /scratch1/BMC/dtc/METplus/METplus-5.0.0_Tutorial_Files/tutorial-5.0.0.conf ./tutorial.conf

Next navigate to the Verify Environment is Set Correctly page.

mccabe Wed, 06/30/2021 - 12:22

Setting up the Tutorial Environment (bash)

The following instructions should be run if configuring the bash environment to run the METplus Tutorial on your own computer. If you are running on an NCAR or NOAA machine that has been set up to run the tutorial, please go back and click the appropriate link for those instructions.

Create a Working Directory

Create a directory called 'METplus-5.0.0_Tutorial' to hold all of the files you will create during the tutorial. This can be any directory that you have write permission.

In the following instructions, change "/path/to" to the directory you chose: EDIT AFTER COPYING and BEFORE HITTING RETURN!

Change directory to the location where you want to put your tutorial files.

cd /path/to

Create a directory called METplus-5.0.0_Tutorial.

This directory will contain all of your tutorial work, including configuration files, output data, and any other notes you'd like to keep.

mkdir METplus-5.0.0_Tutorial

Change directory into the tutorial directory.

cd METplus-5.0.0_Tutorial

Create Directories for Configuration Files and Output Data

mkdir user_config

mkdir output

Obtain the Tutorial Setup Script

Copy the tutorial setup shell script to configure your runtime environment for use with METplus. You will also copy over a METplus .conf file that you will use to run:

wget https://dtcenter.org/sites/default/files/community-code/metplus/tutorial-data/METplus-5.0.0_TutorialSetup.bash.sh.txt -O ./METplus-5.0.0_TutorialSetup.sh
wget https://dtcenter.org/sites/default/files/community-code/metplus/tutorial-data/tutorial-5.0.0.conf -O ./tutorial.conf

Configure the Tutorial Setup Script

Open the tutorial setup script with the editor of your choice and modify the values for METPLUS_BUILD_BASE, MET_BUILD_BASE, and METPLUS_DATA.

vi METplus-5.0.0_TutorialSetup.sh

METPLUS_BUILD_BASE is the full path to the METplus installation (/path/to/METplus-X.Y)
MET_BUILD_BASE is the full path to the MET installation (/path/to/met-X.Y)
METPLUS_DATA is the location of the sample test data directory

Next navigate to the Verify Environment is Set Correctly page.

mccabe Wed, 06/30/2021 - 12:06

Setting up the Tutorial Environment (csh)

The following instructions should be run if configuring a csh environment to run the METplus Tutorial on your own computer. If you are running on an NCAR or NOAA machine that has been set up to run the tutorial, please go back and click the appropriate link for those instructions.

Create a Working Directory

Create a directory called 'METplus-5.0.0_Tutorial' to hold all of the files you will create during the tutorial. This can be any directory that you have write permission.

In the following instructions, change "/path/to" to the directory you chose: EDIT AFTER COPYING and BEFORE HITTING RETURN!

Change directory to the location where you want to put your tutorial files.

cd /path/to

Create a directory called METplus-5.0.0_Tutorial.

This directory will contain all of your tutorial work, including configuration files, output data, and any other notes you'd like to keep.

mkdir METplus-5.0.0_Tutorial

Change directory into the tutorial directory.

cd METplus-5.0.0_Tutorial

Create Directories for Configuration Files and Output Data

mkdir user_config

mkdir output

Obtain the Tutorial Setup Script

Copy the tutorial setup shell script to configure your runtime environment for use with METplus. You will also copy over a METplus .conf file that you will use to run:

wget https://dtcenter.org/sites/default/files/community-code/metplus/tutorial-data/METplus-5.0.0_TutorialSetup.csh.sh.txt -O ./METplus-5.0.0_TutorialSetup.sh
wget https://dtcenter.org/sites/default/files/community-code/metplus/tutorial-data/tutorial-5.0.0.conf -O ./tutorial.conf

Configure the Tutorial Setup Script

Open the tutorial setup script with the editor of your choice and modify the values for METPLUS_BUILD_BASE, MET_BUILD_BASE, and METPLUS_DATA.

vi METplus-5.0.0_TutorialSetup.sh

Next navigate to the Verify Environment is Set Correctly page.

mccabe Wed, 06/30/2021 - 12:21