MET Tool: TC-Stat

MET Tool: TC-Stat cindyhg Mon, 06/24/2019 - 12:01

TC-Stat Tool: General

TC-Stat Functionality

The TC-Stat tool reads the .tcst output file(s) of the TC-Pairs tool. This tools provides the ability to further filter the TCST output files as well as summarize the statistical information. The TC-Stat tool reads .tcst files and runs one or more analysis jobs on the data. TC-Stat can be run by specifying a single job on the command line or multiple jobs using a configuration file. The TC-Stat tool is very similar to the Stat-Analysis tool. The two analysis job types are summarized below:

The filter job simply filters out lines from one or more TCST files that meet the filtering options specified.
The summary job operates on one column of data from TCST file. It produces summary information for that column of data: mean, standard deviation, min, max, and the 10th, 25th, 50th, 75th, and 90th percentiles, independence time, and frequency of superior performance.
The rirw job identifies rapid intensification or weakening events in the forecast and analysis tracks and applies categorical verification methods.
The probrirw job applies probabilistic verification methods to evaluate probability of rapid inensification forecasts found in edeck's.

TC-Stat Usage

View the usage statement for TC-Stat by simply typing the following:

tc_stat

Usage: tc_stat
	-lookin path	TCST file or top-level directory containing TCST files (where TC-Pairs output resides). It allows the use of wildcards (at least one tcst file is required).
	[-out file]	Output path or specific filename to which output should be written rather than the screen (optional).
	[-log file]	Outputs log messages to the specified file
	[-v level]	Level of logging
	[-config config_file] \| [JOB COMMAND LINE] (Note: "\|" means "or")
	*[-config config_file]*	TCStat config file containing TC-Stat jobs to be run.
	*[JOB COMMAND LINE]*	Arguments necessary to perform a TC-Stat job.

At a minimum, you must specify at least one directory or file in which to find TCST data (using the -lookin path command line option) and either a configuration file (using the -config config_file command line option) or a job command on the command line.

TC-Stat Tool: Configure

TC-Stat Tool: Configure cindyhg Mon, 06/24/2019 - 12:02

TC-Stat Tool: Configure

Start by making an output directory for TC-Stat and changing directories:

mkdir -p ${METPLUS_TUTORIAL_DIR}/output/met_output/tc_stat

cd ${METPLUS_TUTORIAL_DIR}/output/met_output/tc_stat

The behavior of TC-Stat is controlled by the contents of the configuration file or the job command passed to it on the command line. The default TC-Stat configuration may be found in the data/config/TCStatConfig_default file. Make a copy of the default configuration file and make following modifications:

cp ${MET_BUILD_BASE}/share/met/config/TCStatConfig_default TCStatConfig_tutorial

Open up the TCStatConfig_tutorial file for editing with your preferred text editor.

vi TCStatConfig_tutorial

Set:

amodel = [ "HWRF", "GFDL" ];
bmodel = [ "BEST" ];
event_equal = TRUE;

To only parse over two of the three model names in the tc_pairs.tcst file. The event_equal=TRUE flag will keep pairs that are in both HWRF and GFDL pairs.

Many of the filter options are left blank, indicating TC-Stat should parse over all available fields. You will notice many more available filter options beyond what was available with the TCPairsConfig.

Now, scroll all the way to the bottom of the TCStatConfig, and you will find the jobs[] section. Edit this section as follows:

jobs = [ "-job filter -dump_row tc_stat.tcst" ];

TC-Stat: Run on TC-Pairs output

TC-Stat: Run on TC-Pairs output cindyhg Mon, 06/24/2019 - 12:03

TC-Stat: Run on TC-Pairs output

Run the TC-Stat using the following command:

tc_stat \

-lookin ../tc_pairs/tc_pairs.tcst \

-config TCStatConfig_tutorial -v 3

Open the output file tc_stat.tcst. We can see that this filter job simply event equalized the two models specified in amodel.

vi tc_stat.tcst

Let's try to filter further, this time using the command line rather than the configuration file:

tc_stat \

-job filter -lookin ../tc_pairs/tc_pairs.tcst \

-dump_row tc_stat2.tcst \

-water_only TRUE \

-column_str LEVEL HU,TS,TD,SS,SD \

-event_equal TRUE \

-match_points TRUE -v 3

Here, we ran a filter job at the command line: only keeping tracks over water (not encountering land) and with categories Hurricane (HU), Tropical Storm (TS), Tropical Depression (TD), Subtropical Storm (SS), and Subtropical Depression (SD).

Open the output file tc_stat2.tcst: notice fewer lines have been kept. Look at the "LEVEL" column ... note all the non-tropical and subtropical level classifications have been filtered out of the sample.

vi tc_stat2.tcst

Also, find the columns ADLAND and BDLAND. All these values are now positive, meaning the tracks over land (negative values) have been filtered.

With the filtering jobs mastered, lets give the second type of job - summary jobs - a try!

TC-Stat: Run on TC-Pairs output

TC-Stat: Run on TC-Pairs output cindyhg Mon, 06/24/2019 - 12:03

TC-Stat: Run on TC-Pairs output

Now, we will run a summary job using TC-Stat on the command line using the following command:

tc_stat \

-job summary -lookin ../tc_pairs/tc_pairs.tcst \

-amodel HWRF,GFDL \

-by LEAD,AMODEL \

-column TK_ERR \

-event_equal TRUE \

-out tc_stat_summary.tcst

Open up the file tc_stat_summary.tcst. Notice this output is much different from the filter jobs.

vi tc_stat_summary.tcst

The track data is event equalized for the HWRF and GFDL models, and summary statistics are produced for the TK_ERR column for each model by lead time.

TC-Stat: Plotting with R

TC-Stat: Plotting with R cindyhg Mon, 06/24/2019 - 12:03

TC-Stat: Plotting with R

In this section, you will use the R statistics software package to produce a plot of a few results. R was introduced in practical session 1.

The MET release includes a number of plotting tools for TC graphics. All of the Rscripts are included with the MET distribution in the Rscripts directory. The script for TC graphics is plot_tcmpr.R, which uses the TCST output files from TC-Pairs as input. To get the usage statement, type:

export MET_INSTALL_DIR=${MET_BUILD_BASE}

export MET_BASE=${MET_INSTALL_DIR}/share/met

Rscript ${MET_BASE}/Rscripts/plot_tcmpr.R

The TC-Stat tool can be called from the Rscript to do additional filter jobs on the TCST output from TC-Pairs. This can be done on the command line by calling a filter job (following tc-stat), or a configuraton file can be used to select filtering criteria. A default configuration file can be found in Rscripts/include/plot_tcmpr_config_default.R.

The configuration file also includes various plot types to generate: MEAN, MEDIAN, SCATTER, REFPERF, BOXPLOT, and RANK. All of the plot commands can be called on the command line as well as with the configuration file. Plots are configurable (title, colors, axis labels, etc) either by modifying the configuration file or setting options on the command line. To run from the command line:

Rscript ${MET_BASE}/Rscripts/plot_tcmpr.R \

-lookin ../tc_pairs/tc_pairs.tcst \

-filter "-amodel HWRF,CONS" \

-dep "TK_ERR" \

-series AMODEL HWRF,CONS \

-plot MEAN,BOXPLOT,RANK \

-outdir .

This plots the track error for two models: HWRF and CONS. CONS is the user defined consensus you generated in TC-Pairs.

Next, open up the output *png files:

display TK_ERR_boxplot.png &

display TK_ERR_mean.png &

display TK_ERR_rank.png &

The script produces the three plot types called in the configuration file:

Boxplot showing distribution of errors for a homogeneous sample of the two models (TK_ERR_boxplot.png).
Mean Errors with 95% CI for the same sample (TK_ERR_mean.png).
Rank plot indicating performance of HWRF model relative to CONS (TK_ERR_rank.png).