Community GSIV3.7 Online Tutorial

Community GSIV3.7 Online Tutorial
admin Mon, 06/15/2020 - 13:50

Note: These instructions pertain to the 2018 community GSI release V3.7 only.

Community GSIV3.7 Online Tutorial

Note: These instructions pertain to the 2018 community GSI release V3.7 only. See the other instructions if you are using another release.

Navigating the GSI Online Tutorial

Throughout this tutorial, you will have to type several commands on the command line and edit settings in several files.

Several exercises are available to target various dynamical cores and data assimilation techniques. Select any of the following practice cases below to get started.

Getting Started

Getting Started cindyhg Mon, 07/15/2019 - 14:19

Getting Started

Download and Build GSI System

Download and Build GSI System cindyhg Mon, 07/15/2019 - 14:20

BUILDING GSI FROM SCRATCH

Obtaining the source code & setting up the directory

This exercise is intended to give you practice building the GSI system. This version of GSI system includes a WRF I/O library. There are no need to build the WRF model for this version GSI building.

The community GSI resources, including source code, build system, utilities, and fix files, are available for download from the DTC community GSI users website: https://dtcenter.org/community-code/gridpoint-statistical-interpolation-gsi/download.

The code downloads will result in the following tar file:

comGSIv3.7_EnKFv1.3.tar.gz

GSI requires use of CRTM coefficients to analyze satellite radiance observations. Due to their large size, these are available as a separate tarfile available for download by selecting the link: CRTM 2.3.0 Big_Endian coefficients tarball

The tars file may be unpacked using the UNIX commands:

tar -zxvf comGSIv3.7_EnKFv1.3.tar.gz

 

This creates the top level GSI directory: comGSIv3.7_EnKFv1.3

 

Setting up the machine environment

Set the necessary paths for using your selected compiler, such as loading the appropriate modules or modifying your path variable.

Before configuring the GSI code to be built, at least one, and no more than two environment variables must be set.

  • NETCDF: the path to the NETCDF libraries
  • LAPACK_PATH: the path to the LAPACK math libraries

For more detail, see GSIv3.7 Users' Guide Chapter 2.

Configure and Compile the GSI code

This version GSI can ONLY be compiled with Cmake. 

First create a building directory (any name works) outside the GSI directory:

mkdir build

Then, get into this building directory

cd build

Now run cmake to configure makefile by typing:

cmake ${Path To GSI Home Directory}/comGSIv3.7_EnKFv1.3

 

Cmake will check your computer environment and generate the makefile. A successful cmake run should end with:

-- Configuring done 
-- Generating done 
-- Build files have been written to: (Path_of_this_directory)/build

The building directory includes many new file and directories generated from Cmake:

bin cmake_install.cmake include Makefile Testing 
CMakeCache.txt CTestTestfile.cmake lib regression_var.out util 
CMakeFiles DartConfiguration.tcl libsrc src 

Now, we can simply run make to compile the GSI and its untilities:

make

If the compilation is successful, the executable gsi.x, will be created in the ./bin directory. If the compilation is not successful, run make with make VERBOSE=1 to collect more information on errors.

After a successful compilation:

Once GSI compiled correctly, return to the online tutorial page to set up and run the case(s) of your choice.

Download Practice Data

Download Practice Data cindyhg Mon, 07/15/2019 - 14:24

DOWNLOAD DATA FOR PRACTICE

The links to download data for practice cases

Please see the following linkes to download the corresponding data for cases you want to practice:

Prepare Base Run Script

Prepare Base Run Script cindyhg Mon, 07/15/2019 - 14:25

PREPARE RUN SCRIPT FOR BASIC CASES

Setting up the Run Script

The GSI run script ush/comgsi_run_regional.ksh, which is provided with the source code, must be customized to the local environment. These changes include things such as the analysis time and date, the path to files, and MPI specific information, which are necessary for running all of tutorial cases.

For this tutorial, start by copying the original run script to a working copy
cd ${PATH of working directory}/run 
cp ${PATH}/comGSIv3.7_EnKFv1.3/ush/comgsi_run_regional.ksh run_gsi_regional.ksh_basic 
This will be used as the template for all the later scripts used in the regional practice.

Make the following modifications to the script run_gsi_regional.ksh_basic

#PBS -A PXXXXXXXX #PBS -l walltime=00:20:00 #PBS -N basic #PBS -l select=1:ncpus=4:mpiprocs=4 #PBS -q premium #PBS -o out.basic #PBS -j oe
  • On top of the scripts, add an appropriate batch script. For example on Cheyenne:
  • Set the variable GSIPROC to 4
  • Set the variable ARCH to LINUX_PBS, as an example value appropriate for Linux with PBS.
  • In the case set up section of the script, set the environment variables to values appropriate for your environment on your machine. For example:
ANAL_TIME=2018081212
JOB_DIR=/gpfs/fs1/p/ral/jntp/mhu/gsi/v37/run
RUN_NAME=basic
OBS_ROOT=/glade/p/ral/jntp/DAtask/case_data/2018081212/obs
BK_ROOT=/glade/p/ral/jntp/DAtask/case_data/2018081212/bkg
GSI_ROOT=/gpfs/fs1/p/ral/jntp/mhu/gsi/v37/comGSIv3.7_EnKFv1.3
CRTM_ROOT=/glade/p/ral/jntp/DAtask/case_data/CRTM_2.2.3
ENS_ROOT=/glade/p/ral/jntp/DAtask/case_data/2018081212/gfsens
HH=`echo $ANAL_TIME | cut -c9-10`
GSI_EXE=${JOB_DIR}/gsi.x #assume you have a copy of gsi.x here
WORK_ROOT=${JOB_DIR}/${RUN_NAME}
FIX_ROOT=${GSI_ROOT}/fix
GSI_NAMELIST=${GSI_ROOT}/ush/comgsi_namelist.sh
PREPBUFR=${OBS_ROOT}/rap.t${HH}z.prepbufr.tm00
BK_FILE=${BK_ROOT}/wrfout_d01_2018-08-12_12:00:00
  • This example is only for ARW core background files.
if_hybrid=No     # Yes, or, No -- case sensitive !
if_4DEnVar=No    # Yes, or, No -- case sensitive (set if_hybrid=Yes first)!
if_observer=No   # Yes, or, No -- case sensitive !
if_nemsio=No     # Yes, or, No -- case sensitive !
if_oneob=No      # Yes, or, No -- case sensitive !

bk_core=ARW
bkcv_option=NAM
if_clean=clean

  • Still in case set up setion, leave the following settings as default:
  • Search Section # Link to the radiance data, look for the list of observations and comment the links to the BUFR file under this section:
# ln -s ${srcobsfile[$ii]} ${gsiobsfile[$ii]}

PREPARE RUN SCRIPT AND ANAVINFO FILE FOR BASIC CASES

submit run script

The basic run script can be submitted by if use Cheyenne:

qsub run_gsi_regional.ksh_basic

An example of this basic run script is available from the link run_gsi_regional.ksh_basic

check the results

After job finished, a new directory ./basic will appear under current directory. The files in directory ./basic is listed here and the stdout from this basic run is available here

ARW Practice Cases

ARW Practice Cases cindyhg Mon, 07/15/2019 - 14:37

Exercises

ARW Practice Cases

Case 1: Single observation test with GLOBAL BE

Case 1: Single observation test with GLOBAL BE cindyhg Mon, 07/15/2019 - 14:38

ARW SINGLE OBSERVATION TEST W/ GLOBAL BE

Introduction

This exercise consists of running the GSI analysis with a single pseudo observation at a specified location, to illustrate how that observation influences the analysis. 

Further information on setting up a single observation test is available in section 4.2 of the GSI User's Guide. 

The ARW background field is provided in netcdf format, and the global BE is employed as the background error covariance.

Setting up the Run Script

Setting up the Run Script cindyhg Mon, 07/15/2019 - 14:40

ARW SINGLE OBSERVATION TEST W/ GLOBAL BE

Setting up the Run Script

For this exercise, make a copy of the prepared basic run script from the section Prepare run script for basic cases: 

cp run_gsi_regional.ksh run_gsi_regional.ksh_psot. 

Make the following additional modifications to the script run_gsi_regional.ksh_psot:

  • Set the name/path for the analysis run directory
  • Set up the one observation test option to true
  • if_oneob=Yes
  • Select the background field format
  • bk_core=ARW
  • Select the global background error covariance
  • bkcv_option=GLOBAL

An example of this run script is available from the link run_gsi_regional.ksh

Check the GSI namelist

Users can check the namelist to see how to set up a single observation test. The related namelist options include :

  • oneobtest=.true.
  • Set the location of the pseudo observation: &SINGLEOB_TEST
    maginnov=1.0,magoberr=0.8,oneob_type='t',
    oblat=38.,oblon=279.,obpres=500.,obdattim=${ANAL_TIME},
    obhourset=0.,
    /

An example namelist is available here.

Running the Script

Running the Script cindyhg Mon, 07/15/2019 - 14:40

ARW SINGLE OBSERVATION TEST W/ GLOBAL BE

Running the Script

In this example, GSI is run as a 4-core MPI job. If you named your run script run_gsi_regional.ksh_psot and run on PBS system (Cheyenne), type: 

qsub run_gsi_regional.ksh_psot 

to launch the job. The progress of the job can be monitored by examining the tail of the standard out file in the run directory as set in the variable WORK_ROOT: 

tail stdout

When completed,the contents of this run directory are provided in the following list .

Case 2: Single observation test with NAM BE

Case 2: Single observation test with NAM BE cindyhg Mon, 07/15/2019 - 14:42

ARW SINGLE OBSERVATION TEST W/ NAM BE

Introduction

This exercise consists of running the GSI analysis with a single pseudo observation at a specified location, to illustrate how that observation influences the analysis. 

Further information on setting up a single observation test is available in section 4.2 of the GSI User's Guide. 

The ARW background field is provided in netcdf format, and the NAM (North American Mesoscale Model) BE is employed as the background error covariance.

Setting up the Run Script

Setting up the Run Script cindyhg Mon, 07/15/2019 - 14:43

ARW SINGLE OBSERVATION TEST W/ NAM BE

Setting up the Run Script

For this exercise, make a copy of the prepared basic run script: 
cp run_gsi_regional.ksh_basic run_gsi_regional.ksh_psot. 

Make the following additional modifications to the script run_gsi_regional.ksh_psot:

  • Set the name/path for the analysis run directory
  • Set up the one observation test option to true
  • if_oneob=Yes
  • Select the background field format
  • bk_core=ARW
  • Select the NAM background error covariance
  • bkcv_option=NAM

An example of this run script is available from the link run_gsi_regional.ksh

Check the GSI namelist

Users can check the namelist to see how to set up a single observation test. The related namelist options include :

  • oneobtest=.true.
  • Set the location of the pseudo observation: &SINGLEOB_TEST
    maginnov=1.0,magoberr=0.8,oneob_type='t',
    oblat=38.,oblon=279.,obpres=500.,obdattim=${ANAL_TIME},
    obhourset=0.,
    /

An example namelist is available here.

Running the Script

Running the Script cindyhg Mon, 07/15/2019 - 14:44

ARW SINGLE OBSERVATION TEST W/ NAM BE

Running the Script

In this example, GSI is run as a 4-core MPI job. If you named your run script run_gsi_regional.ksh and run on PBS system (Cheyenne), type: 

qsub run_gsi_regional.ksh 

to launch the job. 

The progress of the job can be monitored by examining the tail of the standard out file in the run directory as specified in the variable WORK_ROOT: 

tail stdout

When completed, the contents of this run directory are provided in the following list .

Results

Results cindyhg Mon, 07/15/2019 - 14:45

ARW SINGLE OBSERVATION TEST W/ NAM BE

Results

The standard output file stdout contains the run diagnostics, such as convergence information and observation distribution from the GSI run. Details of the standard output file are available in section 4.1 of the GSI User's Guide.

Information about the use of observations by the analysis, and the corresponding statistics are available from the fit files (namedfort.2*). The fit files located in the run directory should agree with the following fit files for temperature (fit_t1); wind (fit_w1);moisture (fit_q1); surface pressure (fit_p1).

Visualizing the Analysis

The model analysis may be visualized through use of the ncl script GSI_singleobs_arw.ncl provided with the community GSI under ./util/Analysis_Utilities/plots_ncl. It plots the XY (left column) and XZ (right column) cross sections of the analysis increment fields through the grid point that has the maximum temperature increment.

To visualize your output, copy the ncl script to run directory and change lines:

  • change cdf_analysis = addfile("wrf_inout.cdf","r") to point to analysis results.
  • change cdf_bk = addfile("${DATA_ROOT}/wrfout_d01_2018-08-12_12:00:00.cdf","r") to point to background.

A sample script can be found at GSI_singleobs_arw.ncl

 

Once you have customized the script, run the script with the command:
ncl GSI_singleobs_arw.ncl 

The script will generate a file: GSI_singleObse_T_arw.pdf. Use display GSI_singleObse_T_arw.pdf to show the image. Compare this image with the reference solution [PDF] for this configuration.

Case 3: 3DVAR with conventional data (PrepBUFR)

Case 3: 3DVAR with conventional data (PrepBUFR) cindyhg Tue, 07/16/2019 - 10:47

3DVAR GSI USING ARW BACKGROUND (PREPBUFR)

Introduction

This exercise consists of running the GSI analysis with an ARW netcdf formatted background field, conventional data from prepbufr. 

Further information on setting up the run is available in chapter 3 of the GSI User's Guide. 

The ARW background field is provided in netcdf format, and the regional NAM BE is employed as the background error covariance.

Setting up the Run Script

Setting up the Run Script cindyhg Tue, 07/16/2019 - 10:48

3DVAR GSI USING ARW BACKGROUND (PREPBUFR)

Setting up the Run Script

For this exercise, make a copy of the previously prepared run script: 
cp run_gsi_regional.ksh_basic run_gsi_regional.ksh. 

Make the following additional modifications to the script run_gsi_regional.ksh:

  • Set the name/path for the analysis run directory to
  • WORK_ROOT=${run directory}
  • Select the background field format bk_core=ARW
  • Select the NAM regional background error covariance bkcv_option=NAM
  • Comment out the links to the radiance data, gpsro and radar data:
  • # ln -s ${srcobsfile[$ii]} ${gsiobsfile[$ii]} 

    •  

An example of this run script is available from the link run_gsi_regional.ksh

Running the Script

Running the Script cindyhg Tue, 07/16/2019 - 10:49

3DVAR GSI USING ARW BACKGROUND (PREPBUFR)

Running the Script

For this example, GSI is run as a 4-core MPI job. If you run on PBS system (Cheyenne), type: 

qsub run_gsi_regional.ksh 

to launch the job.

The progress of the job can be monitored by examining the tail of the standard out file in the run directory as specfied in the variable WORK_ROOT: 

tail stdout

The contents of this run directory are provided in the following list.

Results

Results cindyhg Tue, 07/16/2019 - 10:49

3DVAR GSI USING ARW BACKGROUND (PREPBUFR)

Results

The standard output file stdout contains the run diagnostics, such as convergence information, and observation distribution from the GSI run. Details of the standard output file are available in section 4.1 of the GSI User's Guide.

Information about the use of observations by the analysis, and the corresponding innovations are available from the fit files (named fort.2*). The fit files located in the run directory should agree with the following fit files for temperature (fit_t1); wind(fit_w1); moisture (fit_q1); surface pressure (fit_p1); and radiance (fit_rad1); and GPS (fort.212); and radar radial velocity(fort.209).

Convergence information is available in the file: fort.220

Visualizing the Analysis

The model analysis may be visualized through modifying the ncl script Analysis_increment.ncl provided with the community GSI under ./util/Analysis_Utilities/plots_ncl. This script plots the analysis increments from conventional observations at at level 1 and 20.

To visualize your output, copy the ncl script to run directory and make the following changes:

  • Set: cdf_analysis = addfile("wrf_inout.cdf","r") to point to analysis results
  • Set: cdf_bk = addfile("${PATH}/wrf_inout.cdf","r") to point to background file.
  • Set: kmax=1 for plot at level 2 or 20 for plot at level 21

The sample scripts for these plots can be found at GSI_Analysis_increment.ncl

 

Once you have customized the script for your output directory, run the script with the command:
ncl GSI_Analysis_increment.ncl 
Once done a pdf file GSI_Analysis_increment_20.pdf will be generated for 21 level analysis increment in the run directory. Compare these images with the reference solution [PDF]. Here is reference for the 2nd level [PDF].

Case 4: 3DVAR with conventional data (PrepBUFR) plus other data

Case 4: 3DVAR with conventional data (PrepBUFR) plus other data cindyhg Tue, 07/16/2019 - 10:50

3DVAR GSI USING ARW BACKGROUND (PREPBUFR AND OTHER OBS)

Introduction

This exercise consists of running the GSI analysis with an ARW netcdf formatted background field, conventional data from prepbufr, satellite radiances, gpsro and radar data. 

Further information on setting up the run is available in chapter 3 of the GSI User's Guide. 

The ARW background field is provided in netcdf format, and the regional NAM BE is employed as the background error covariance.

Setting up the Run Script

Setting up the Run Script cindyhg Tue, 07/16/2019 - 10:51

3DVAR GSI USING ARW BACKGROUND (PREPBUFR AND OTHER OBS)

Setting up the Run Script

For this exercise, make a copy of the previously prepared run script: 
cp run_gsi_regional.ksh_basic run_gsi_regional.ksh. 

Make the following additional modifications to the script run_gsi_regional.ksh:

  • Set the name/path for the analysis run directory to
  • WORK_ROOT=${run directory}
  • Select the background field format bk_core=ARW
  • Select the NAM regional background error covariance bkcv_option=NAM
  • Open the links to the radiance data, gpsro and radar data by un-commenting the following line:
  • ln -s ${srcobsfile[$ii]} ${gsiobsfile[$ii]} 

An example of this run script is available from the link run_gsi_regional.ksh

Running the Script

Running the Script cindyhg Tue, 07/16/2019 - 10:51

3DVAR GSI USING ARW BACKGROUND (PREPBUFR AND OHTER OBS)

Running the Script

For this example, GSI is run as a 4-core MPI job. If you run on PBS system (Cheyenne), type: 

qsub run_gsi_regional.ksh 

to launch the job.

The progress of the job can be monitored by examining the tail of the standard out file in the run directory as specfied in the variable WORK_ROOT: 

tail stdout

The contents of this run directory are provided in the following list.

Results

Results cindyhg Tue, 07/16/2019 - 10:52

3DVAR GSI USING ARW BACKGROUND (PREPBUFR AND OTHER OBS)

Results

The standard output file stdout contains the run diagnostics, such as convergence information, and observation distribution from the GSI run. Details of the standard output file are available in section 4.1 of the GSI User's Guide.

Information about the use of observations by the analysis, and the corresponding innovations are available from the fit files (named fort.2*). The fit files located in the run directory should agree with the following fit files for temperature (fit_t1); wind(fit_w1); moisture (fit_q1); surface pressure (fit_p1); and radiance (fit_rad1); and GPS (fort.212); and radar radial velocity(fort.209).

Convergence information is available in the file: fort.220

Visualizing the Analysis

The model analysis may be visualized through modifying the ncl script Analysis_increment.ncl provided with the community GSI under ./util/Analysis_Utilities/plots_ncl. This script plots the data impact from additional satellite radiance, gpsro and radar data (analysis with conventional, satellite radiance, gpsro and radar data minus analysis with conventional data only) at level 31.

To visualize your output, copy the ncl script to run directory and make the following changes:

  • Set: cdf_analysis = addfile("wrf_inout.cdf","r") to point to analysis results
  • Set: cdf_bk = addfile("${Path to case 3 result}/wrf_inout.cdf","r") point to case 3 analysis file.
  • Set: kmax=20 for plot at level 21

The sample scripts for these plots can be found at GSI_Analysis_increment.ncl

 

Once you have customized the script for your output directory, run the script with the command:
ncl Analysis_increment.ncl 
Once done a pdf file GSI_Analysis_increment_20.pdf will be generated in the run directory. Compare these images with the reference solution [PDF].

Case 5: 3D Hybrid EnVar

Case 5: 3D Hybrid EnVar cindyhg Tue, 07/16/2019 - 10:53

GSI 3D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Introduction

This exercise runs the GSI 3 Dimensional Ensemble-Variational (EnVar) hybrid analysis with the ARW background, conventional data at 12z August 12, 2018. 

Please note the ARW background field is provided in netcdf format, and the NAM BE is employed as the background error covariance in this experiment. The global ensemble forecasts are linked to run this GSI hybrid test.

Setting up the Run Script for GSI hybrid analysis

Setting up the Run Script for GSI hybrid analysis cindyhg Tue, 07/16/2019 - 10:53

GSI 3D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Setting up the Run Script for GSI hybrid analysis

Copy the sample run script run_gsi_regional.ksh from the practical case 3 (ARW 3DVAR with PrepBUFR) to a working directory and make the following modifications to run GSI hybrid analysis:

  • Set the name/path for the analysis run directory to
  • WORK_ROOT=${run directory}
  • Set the location of the ensemble files in the variable ENS_ROOT=...
  • Set to run GSI hybrid analysis: if_hybrid=Yes
  • Set not to run GSI 4D hybrid analysis: if_4DEnVar=No

Please note that the sample script provides links to the GFS ensemble data for this tutorial case only. If you are running your own GSI hybrid case with a different date, please make any necessary modifications to specify the variable ENS_ROOT and ENSEMBLE_FILE_mem in the run script.

Running the Script

Running the Script cindyhg Tue, 07/16/2019 - 10:54

SETUP GSI 3D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Running the Script

If you run on PBS system (Cheyenne), type: 

qsub run_gsi_regional.ksh 

to launch the job. 

The progress of the job can be monitored by examining the tail of the standard out file in the run directory as specified in the variable WORK_ROOT: 

tail stdout

When completed, the contents of this run directory are provided in the following list .

Results

Results cindyhg Tue, 07/16/2019 - 10:55

SETUP GSI 3D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Results

The standard output file stdout contains the run diagnostics, such as convergence information, and observation distribution from the GSI run. Details of the standard output file are available in section 4.1 of the GSI User's Guide.

Information about the use of observations by the analysis, and the corresponding innovations are available from the fit files (named fort.2*). The fit files located in the run directory should agree with the following fit files for temperature (fit_t1); wind(fit_w1); moisture (fit_q1); surface pressure (fit_p1); and radiance (fit_rad1); and GPS (fort.212); and radar radial velocity(fort.209).

Convergence information is available in the file: fort.220

Visualizing the Analysis

Use the same method as the practical case 3 (ARW 3DVAR) to make plots of the analysis increments. This time, plots will be made for the 2nd level (kmax=1) and level 21 (kmax=20). Once done pdf files GSI_Analysis_increment_1.pdf and GSI_Analysis_increment_20.pdf will be generated in the run directory. Compare these images with the reference solution [level 2 ] and [level 21].

Case 6: Cycling Case

Case 6: Cycling Case cindyhg Tue, 07/16/2019 - 10:56

GSI CYCLING RUN ARW BACKGROUND

Introduction

This exercise illustrate the basic structure and flow of a cycling data assimilation system, as shown in this chart (time doesn't match with the new case). It consists of running the GSI analysis with an ARW netcdf background field, and then the GSI analysis provides the initial fields for running a WRF-ARW forecast. The forecast output can be used as the GSI background for next GSI analysis.

The regional NAM BE is employed as the background error covariance and only conventional observations are assimilated in this example.

There are 4 steps in this GSI-ARW cycling data assimilation exercise:

  • Step 1: GSI Data Analysis for 12Z of August 12, 2018. This step is similar to the online exercise ARW 3DVAR with conventional data (PrepBUFR)
  • Step 2: WRF-ARW model forecast at 12Z of August 12, 2018, using the GSI analysis from step 1.
  • Step 3: GSI Data Analysis for 18Z of August 12, 2018, using the 6-hour forecast output from step 2.
  • Step 4: WRF-ARW model forecast at 18Z of August 12, 2018, using the GSI analysis from step 3.

GSI analysis at 12Z

GSI analysis at 12Z cindyhg Tue, 07/16/2019 - 10:57

GSI CYCLING RUN ARW BACKGROUND

GSI analysis at 12Z

For this step of GSI analysis at 12Z of August 12, 2018, we will use the GSI analysis output wrf_inout from the online exercise 03 ARW 3DVAR with conventional data (PrepBUFR) .

If you haven't practiced case 03, simply follow the steps in the above link to perform the GSI data assimilation and get the analysis.

Set up WRF-ARW run at 12Z

Set up WRF-ARW run at 12Z cindyhg Tue, 07/16/2019 - 10:58

GSI CYCLING RUN ARW BACKGROUND

Set up WRF-ARW run at 12Z

For this step of WRF-ARW run at 12Z of August 12, 2018, we will use the GSI analysis output wrf_inout from the online exercise 05 as the initial fields to launch 6-hour WRF forecast.

First, make sure you have a compiled code of the latest WRF-ARW code (V4.0). You can download the boundary condition for WRF from link . 

Please follow the WRF tutorial and documents for the details of the WRF system application. 

Here we provide the namelist file for reference of WRF run: 
namelist.input

Running the ARW and checking the forecast results

The ARW can be run by creating a run script run_wrf.ksh and submitting it in the run directory: 

bsub < run_wrf.ksh

It will take a few minutes to finish. Once done, users should see the forecast files in each forecast hour from 12z to 20z like 
wrfout_d01_2018-08-12_HH_00:00:00.

The contents of this run directory are provided in the following list .

The ARW standard output file rsl.out.0000 is povided for reference.

GSI analysis at 18Z

GSI analysis at 18Z cindyhg Tue, 07/16/2019 - 10:58

GSI CYCLING RUN ARW BACKGROUND

GSI analysis at 18Z

For this step of GSI analysis at 18Z of August 12, 2018, we will use the 6-hour forecast output from the WRF run at 12Z as the background and conventional observations at 18Z. The steps to set up the GSI analysis is very similar to case 03, except for the background field and observations. The example run script can be found here.

Running the GSI run Script and checking the results

After GSI runs, a run directory will be created according to the path set in the variable WORK_ROOT. The contents of this run directory are provided in the following list .

The standard output file stdout and the fit files for this GSI run: temperature (fit_t1); wind (fit_w1); moisture (fit_q1).

Convergence information (section 4.6 of the GSI User's Guide) is available in the file: fort.220

WRF-ARW run at 18Z

As a continued data assimilation run, the 18Z GSI analysis from the above step is then used as the initial field, together with the WRF background conditions at 18Z, to launch the WRF forecast at 18Z. The steps are very similar to the WRF run at 12Z and therefore not repeated here.

Case 7: 4D Hybrid EnVar Case

Case 7: 4D Hybrid EnVar Case cindyhg Tue, 07/16/2019 - 10:59

GSI 4D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Introduction

This exercise runs the GSI 4 Dimensional Ensemble-Variational (EnVar) hybrid analysis with the ARW background and conventional data at 18z August 12, 2018. 

Please note the ARW background field is provided at three time levels in netcdf format from case 6, and the NAM BE is employed as the background error covariance in this experiment. The global ensemble files at three time levels are linked to run this GSI 4DEnVar hybrid test.

Please check the Download Practice Data section if need to obtain the background, observation, and ensemble forecast files.

Setting up the Run Script for GSI 4D hybrid analysis

Setting up the Run Script for GSI 4D hybrid analysis cindyhg Tue, 07/16/2019 - 10:59

GSI 4D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Setting up the Run Script for GSI 4D hybrid analysis

Copy the sample run script run_gsi_regional.ksh from the practical case 3 (ARW 3DVAR with PrepBUFR) to a working directory and make the following modifications to run GSI 4D hybrid analysis:

  • set the analysis time to 
    ANAL_TIME=2017051318
  • Set the name/path for the analysis run directory to
  • WORK_ROOT=${run directory}
  • Set the location of the ensemble files in the variable ENS_ROOT=...
  • set the BK_ROOT to where you store the background files, WRF forecast at 3 time levels
  • set the path to the background file at the analysis time 
    BK_FILE=${BK_ROOT}/wrfout_d01_2018-08-12_18:00:00
  • set to run 4D hybrid analysis 
    if_hybrid=Yes 
    if_4DEnVar=Yes

As can be seen in the sample run script, for ${if_4DEnVar} = Yes, there are two additional background files at 1 hour before and after the analysis time, as specified in the variable BK_FILE_M1 and BK_FILE_P1; there are also two additional sets of ensemble files before and after the analysis time, as specified in the variable ENSEMBLE_FILE_mem_m1 and ENSEMBLE_FILE_mem_p1. Please make sure those variables setup right.

An example of this run script is available from the link run_gsi_regional.ksh

Running the Script

Running the Script cindyhg Tue, 07/16/2019 - 11:00

GSI 4D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Running the Script

If you run on PBS system (Cheyenne), type: 

qsub run_gsi_regional.ksh 

to launch the job. 

The progress of the job can be monitored by examining the tail of the standard out file in the run directory as specified in the variable WORK_ROOT: 

tail stdout

When completed, the contents of this run directory are provided in the following list .

Results

Results cindyhg Tue, 07/16/2019 - 11:00

GSI 4D HYBRID FOR ARW USING GLOBAL ENSEMBLE FORECAST

Results

The standard output file stdout contains the run diagnostics, such as convergence information, and observation distribution from the GSI run. Details of the standard output file are available in section 4.1 of the GSI User's Guide.

Information about the use of observations by the analysis, and the corresponding innovations are available from the fit files (named fort.2*). The fit files located in the run directory should agree with the following fit files for temperature (fit_t1); wind(fit_w1); moisture (fit_q1); surface pressure (fit_p1); and radiance (fit_rad1); and GPS (fort.212); and radar radial velocity(fort.209).

Convergence information is available in the file: fort.220

Visualizing the Analysis

Use the same method as the practical case 3 (ARW 3DVAR) to make plots of the analysis increments. This time, plots will be made for the 2nd level (kmax=1) and level 21 (kmax=20). Once done pdf files GSI_Analysis_increment_1.pdf and GSI_Analysis_increment_20.pdf will be genera ted in the run directory. Compare these images with the reference solution [level 2 ] and [level 21].

Other Practical Cases

Other Practical Cases cindyhg Tue, 07/16/2019 - 11:02

Exercises

Other Practical Cases

Case 8: GSI analysis for HWRF

Case 8: GSI analysis for HWRF cindyhg Tue, 07/16/2019 - 11:13

3D HYBRID GSI USING NMM - HURRICANE WRF CASE

Introduction

The GSI is used for 3D Hybrid Data Assimilation within the Hurricane WRF (HWRF) opertional system, which employes the NMM dynamical core. To run the full HWRF system, see: [HWRF Online Tutorial]

Case 9: Chem case: GSI analysis for WRF-Chem

Case 9: Chem case: GSI analysis for WRF-Chem cindyhg Tue, 07/16/2019 - 11:14

GSI CHEMICAL ANALYSIS FOR WRF-CHEM GOCART

Introduction, Background and Data

The GSI has been developed to analyze chemical observations, such as MODIS AOD or PM2.5, to improve the pollution forecast with chemical models. 

This exercise introduces running the GSI analysis with WRF-Chem GOCART background and PM2.5 observations.

Please check the Download Practice Data section if need to obtain the background and observation files.

Setup GSI run scripts for chemical analysis

Setup GSI run scripts for chemical analysis cindyhg Tue, 07/16/2019 - 11:15

GSI CHEMICAL ANALYSIS FOR WRF-CHEM GOCART

Setup GSI run scripts for chemical analysis

The script run_gsi_chem.ksh was built based on regional GSI run scripts and has a similar structure to the regional run scriptrun_gsi_regional.ksh, but include a couple of different details.

The first part of the run script sets up the computer environment and case configuration. This is the similar to the regional analysis run scripts, except for the namelist for the chemical application, the specification of the chemical cases (bk_core and obs_type):

  • Set the analysis date: ANAL_TIME=2012060318
  • Set the observation file: PREPBUFR=${OBS_ROOT}/anow.2012060318.bufr
  • Set the background file: BK_FILE=${BK_ROOT}/wrfinput_d01_2012-06-03_18:00:00
  • Set to use the namelist for chemical analysis: 
    GSI_NAMELIST=${GSI_ROOT}/ush/comgsi_namelist_chem.sh
  • Set the core for the background file: BK_CORE=WRFCHEM_GOCART
  • Set the observation type: obs_type=PM2.5

Similar to the regional run script, this chemical run script will also double check the needed parameters. Then it creates a run directory and generates the namelist in the directory and copies the background, observations, and fixed files into the run directory

An example of the run script and namelist is available from the link run_gsi_chem.ksh namelist for chem

Running the GSI Chemical Run script

Running the GSI Chemical Run script cindyhg Tue, 07/16/2019 - 11:15

GSI CHEMICAL ANALYSIS FOR WRF-CHEM GOCART

Running the GSI Chemical Run script

If you run on PBS system (Cheyenne), submit the GSI chemical analysis run script: 
qsub run_gsi_chem.ksh

When completed, the contents of this run directory are provided in the following list .

Results

The standard output file stdout contains the run diagnostics, details of the standard output file are available in section 6.2.2 of the GSI User's Guide.

Analysis Increments should be checked after successfully running the chemical analysis to see if the data impact are reasonable.

Case 10: Chem case: GSI analysis for CMAQ

Case 10: Chem case: GSI analysis for CMAQ cindyhg Tue, 07/16/2019 - 11:17

GSI CHEMICAL ANALYSIS FOR CMAQ

Introduction, Background and Data

The GSI has been developed to analyze chemical observations, such as MODIS AOD or PM2.5, to improve the pollution forecast with chemical models. 

This exercise introduces running the GSI analysis with CMAQ background and PM2.5 observations.

Please check the Download Practice Data section if need to obtain the background and observation files.

Setup GSI run scripts for chemical analysis

Setup GSI run scripts for chemical analysis cindyhg Tue, 07/16/2019 - 11:18

GSI CHEMICAL ANALYSIS FOR CMAQ

Setup GSI run scripts for chemical analysis

The script run_gsi_chem.ksh was built based on regional GSI run scripts and has a similar structure to the regional run scriptrun_gsi_regional.ksh, but include a couple of different details.

The first part of the run script sets up the computer environment and case configuration. This is the similar to the regional analysis run scripts, except for the namelist for the chemical application, the specification of the chemical cases (bk_core and obs_type):

  • Set the analysis date: ANAL_TIME=2013062112
  • Set the observation file: PREPBUFR=${OBS_ROOT}/anow.2013062112.bufr
  • Set the background file: BK_FILE=${BK_ROOT}/cmaq2gsi_4.7_20130621_120000.bin
  • Set to use the namelist for chemical analysis: 
    GSI_NAMELIST=${GSI_ROOT}/ush/comgsi_namelist_chem.sh
  • Set the core for the background file: BK_CORE=CMAQ
  • Set the observation type: obs_type=PM2.5

Similar to the regional run script, this chemical run script will also double check the needed parameters. Then it creates a run directory and generates the namelist in the directory and copies the background, observations, and fixed files into the run directory

An example of the run script and namelist is available from the link run_gsi_chem.ksh namelist for chem

Running the GSI Chemical Run script and Results

Running the GSI Chemical Run script and Results cindyhg Tue, 07/16/2019 - 11:19

GSI CHEMICAL ANALYSIS FOR CMAQ

Running the GSI Chemical Run script

If you run on PBS system (Cheyenne), submit the GSI chemical analysis run script: 
qsub run_gsi_chem.ksh

When completed, the contents of this run directory are provided in the following list .

Results

The standard output file stdout contains the run diagnostics, details of the standard output file are available in section 6.2.2 of the GSI User's Guide.

Case 11: GFS case: GSI Analysis for GFS

Case 11: GFS case: GSI Analysis for GFS cindyhg Tue, 07/16/2019 - 11:22

SET UP GSI GLOBAL ANALYSIS

Introduction

This exercise consists of running the GSI 3DVar analysis with T62 global (GFS) background field, conventional data from prepbufr, satellite radiances, and gpsro data.

Background and Data

The global background files are:

surface forecast files at 3 time levels: 3, 6, 9 hours

sfcf03 sfcf06 sfcf09

atmosphere forecast files at 3 time levels: 3, 6, 9 hours

sigf03 sigf06 sigf09

The observation files are:

  • prepbufr   (conventional data)
  • airsbufr   (gdas1.t06z.airsev.tm00.bufr_d)
  • gomebufr   (gdas1.t06z.gome.tm00.bufr_d)
  • gsnd1bufr  (gdas1.t06z.goesfv.tm00.bufr_d)
  • iasibufr   (gdas1.t06z.mtiasi.tm00.bufr_d)
  • sbuvbufr   (gdas1.t06z.osbuv8.tm00.bufr_d)
  • ssmisbufr  (gdas1.t06z.spssmi.tm00.bufr_d)
  • amsuabufr  (gdas1.t06z.1bamua.tm00.bufr_d)
  • gpsrobufr  (gdas1.t06z.gpsro.tm00.bufr_d)
  • hirs4bufr  (gdas1.t06z.1bhrs4.tm00.bufr_d)
  • mhsbufr    (gdas1.t06z.1bmhs.tm00.bufr_d)
  • seviribufr (gdas1.t06z.sevcsr.tm00.bufr_d)
  • tmirrbufr  (gdas1.t06z.sptrmm.tm00.bufr_d)

Please check the Downloading Practice Data section if need to obtain the background and observation files.

Set up GSI Global Analysis

Set up GSI Global Analysis
admin Mon, 06/29/2020 - 21:11

Setting up the Run Script

For this exercise, we will use run script run_gsi_global.ksh provided with the release package under ./ush directory.

Based on an example environment, make the following modifications to the script run_gsi_global.ksh:

  • In section "case set up (users should change this part)":

    An example of this run script is available from the link run_gsi_global.ksh

    • specify the analysis date:
      ANAL_TIME=2014080400
    • specify the global case:
      GFSCASE=T62
    • specify the run directory:
      WORK_ROOT=...
    • specify the location of the background files:
      BK_ROOT=.../T62.gfs/bkg
    • specify the location of the observations:
      OBS_ROOT=.../T62.gfs/obs

Run Script and Results

Run Script and Results
admin Mon, 06/29/2020 - 21:13

Running the Script

Here, GSI global is run as a 4-core MPI job. If you run on PBS system (Cheyenne), Use:

qsub run_gsi_global.ksh

to launch the job.

The run script will create an output or run directory according to the path set in the variable WORK_ROOT. The contents of this run directory are provided in the following list.

Results

The standard output file stdout contains the run diagnostics, such as convergence information, and observation distribution from the GSI run. Details of the standard output file are available in section 4.1 of the GSI User's Guide.

Information about the use of observations by the analysis, and the corresponding innovations are available from the fit files (named fort.2*). The fit files located in the run directory should agree with the following fit files for temperature (fit_t1); wind (fit_w1); moisture (fit_q1); surface pressure (fit_p1); and radiance (fit_rad1); and GPS (fort.212).

Convergence information (section 4.6 of the GSI User's Guide) is available in the file: fort.220

Docker for GSI

Docker for GSI cindyhg Tue, 07/16/2019 - 11:26

DOCKER FOR GSI/ENKF PRACTICE

Docker image for GSI/EnKF

A docker container image is available in GSI/EnKF Download page for users to practice GSI and EnKF on-line tutorial cases under Docker container. This image includes all necessory software environments for running GSI/EnKF and is provided for practice running on-line cases only. 

Note: Users need to have docker correctly installed and know how to run docker. We don't have resources to support docker and its usage, including the image we released.

Run GSI/EnKF within Docker

Please check README.GSI_Docker in the release tarball for an introduction on using this docker container.