Logging In and Submitting Jobs

Logging in and submitting jobs on NCAR's Cheyenne

Logging in to cheyenne.ucar.eduThe HWRF tutorial practical session exercises are run on NCAR's Cheyenne computing platform.

You will want to open at least two login windows on Cheyenne, one for editing files and a second for running jobs. This can be done by:

• On Linux, select open terminal from the right mouse button menu.
• On Apple OS X, start the Terminal and XQuartz applications from the /Applications/Utilities folder.
• On Windows, start the application XSession from the Start menu and then Putty.

Start by opening an X-enabled window to a login node:

1. Plug your Key card into your laptop's USB slot.
2. Log in to the NCAR supercomputer Cheyenne from your laptop:
   ssh -Y USER@cheyenne.ucar.edu, where USER is for Cheyenne
3. At the prompt, enter your 4-digit pin.
4. Press the gold disk on your Key card until the prompt returns.
5. Remove the Key card and place it in a safe place.

This proceedure logs you into Cheyenne with an X-windows enabled window. If you want to use a visual editor such as emacs or nedit launch it from this Cheyenne window.

Editing files with vi

Basic commands for using vi to edit a file:

• vi filename ... Open a file named filename (creates new file if filename does not exist)

The vi editor has two modes: navigation mode and insert mode. To enter insert mode, type "i" in the prompt. "--INSERT--" will appear at the bottom of the screen.

To exit INSERT mode, types "Esc"

To save your work, type :w while in navigation mode.

To exit the file and save your work, type ZZ while in navigation mode.

Configuring your environment on cheyenne.ucar.edu

The default shell on Cheyenne is the C-Shell (tcsh). In this tutorial, instructions are provided only for tcsh.

Configure your runtime environment by setting environment variables and loading the modules required for this tutorial.
Edit the ~/.cshrc file to set:

• module purge
  module load ncarenv
  module load intel/16.0.3
  module load impi/5.1.3.210
  module load mkl/11.3.3
  module load netcdf
  module load ncarcompilers
  module load pnetcdf/1.8.0
  module load ncview
  module load grads
  module load ncl
  module load nco
  module load matlab

  setenv SCRATCH /glade/scratch/${USER}/HWRF_v3.9a

  setenv LIB_Z_PATH /usr/lib64
  setenv LIB_PNG_PATH /usr/lib64
  setenv LIB_JASPER_PATH /usr/lib64
  setenv LIB_G2_PATH ${SCRATCH}/hwrfrun/sorc/hwrf-utilities/libs
  setenv JASPERLIB /glade/u/apps/ch/os/usr/lib64
  setenv JASPERINC /glade/u/apps/ch/os/usr/include

  setenv HOMEhwrf /glade/scratch/${USER}/HWRF_v3.9a/hwrfrun
  setenv WORKhwrf /glade/scratch/${USER}/pytmp/hwrfrun/2016100400/14L
  setenv COMIN /glade/scratch/${USER}/pytmp/hwrfrun/com/2016100400/14L

  set path = (/glade/p/ral/jnt/tools/grads-2.0.2/bin /glade/p/ral/jnt/tools/wgrib2 $path)

These settings may be copied and included from the following file:
/glade/p/ral/jnt/HWRF/HWRF_v3.9a_tut_codes/hwrf.envars.csh

Save your changes, close the ~/.cshrc file, and run the following command:

• source ~/.cshrc

Any future shells you launch will automatically apply these settings to your environment.

Running jobs on cheyenne.ucar.edu

This is an example to be followed later, when you start conducting each of the exercises.

Do not edit any scripts or submit any jobs at this time.

To run each of the wrapper scripts associated with the first and second case studies of the practical session, copy template file /glade/p/ral/jnt/HWRF/HWRF_v3.9a_tut_codes/qsub_Cheyenne_wrapper.csh to your working directory with a new name, run.job_name.csh.

Edit the template file to change the wall clock time and the number of cores needed for your job. Also change the job name and the name of the wrapper script. To submit the script, use qsub run.job_name.csh.

Example for init_gfs:

• cd ${SCRATCH}/hwrfrun/wrappers
  cp /glade/p/ral/jnt/HWRF/HWRF_v3.9a_tut_codes/qsub_Cheyenne_wrapper.csh run.init_gfs.csh

• #PBS -N init_gfs
  #PBS -A NJNT0006
  #PBS -l walltime=00:20:00
  #PBS -q premium
  #PBS -l select=4:ncpus=24:mpiprocs=24
  #
  ${SCRATCH}/hwrfrun/wrappers/init_gfs_wrapper

Submit the script by running:

• qsub run.init_gfs.csh

The number of processors and wall clock required for individual tasks are included in the comment section at the end of the template. Please change them accordingly.

Checking status of jobs on cheyenne.ucar.edu

To monitor jobs on Cheyenne type qstat -u ${USER} in the login prompt. This will give you status of your own jobs.

CISL

The Computation and Information Systems Lab CISL group provides user support and services for Cheyenne. Please refer to their website for more information or to open a help ticket.