METviewer: Time Series of Categorical Statistics Plot

The following list shows the setting name and the setting value for each of the controls on the web application. Please go through them in order, starting at the top, setting and checking the controls as suggested. The settings that you need to change are marked in blue. The settings are grouped according to the areas on the web app. If you need more information about the controls, please click the little i in a circle to the right of METviewer 2.9 at the very top of the page. This will take you to the METviewer documentation.

• Database: Click on the box named Select databases and from the Verification group, select mv_met_tutorial
  To make a plot with sample tutorial data.
• Tab: Series - Create a time-series (as we will do here) or threshold-series line plot. The different tabs correspond to the types of plots that METviewer can generate, such as box plots, bar plots, histograms, and ensemble plots.
• Plot Data: Stat - Plot traditional statistics as opposed to MODE object attributes.
• Y1 Axis variables - These controls specify the statistics and curves for the Y1 axis of the plot.

  Y1 Dependent (Forecast) Variables - pairs of forecast variables and statistics to be plotted

  • Click dropdown list and select: APCP_03 - to select 3-hourly Accumulated Precipitation
  • Click Select attribute stat dropdown list and select: ACC and CSI - to plot Accuracy and Critical Success Index

  Y1 Series Variables - specifies the lines on the plot for each dependent variable

  • Click the dropdown list and select: MODEL
  • Click the Select value dropdown list and select: QPF - output for only one model is present
  • Next, click the + Series Variable button, click the dropdown list, and select: FCST_THRESH
  • Click the Select value dropdown list and select: >0.0, >2.54, and >6.35 - to plot statistics for 3 categorical thresholds
• Y2 Axis Tab - these controls specify the statistics and curves for the Y2 axis of the plot; it is not used in this demonstration
   
• Fixed Values - field/value pairs that are held constant for all data on the plot
  • Click the +Fixed Value button - to add a fixed value selection option
  • Click the dropdown list and select VX_MASK
  • Click the Select value dropdown list and select: FULL - to plot data for FULL model domain (the only one present in this dataset)
• Independent Variable - the x-axis field and values over which the data are displayed on the plot
  • Click the dropdown list and select FCST_LEAD
  • Click the Select value dropdown list and click the Check all option - to select all items
• Statistics - 
  The Summary radio button tells MET to plot the mean or median of the statistics pulled directly from the database.
  The Aggregation statistics radio button tells METviewer to aggregate contingency table counts or partial sums and compute aggregate summary statistics and confidence intervals.
  Use the default Summary radio button selection.
   
• Series Formatting - settings to control colors, line types, plotting symbols, and confidence intervals
   
  • There should be settings for 6 lines: 3 lines for ACC followed by 3 lines for CSI. In the Line Color column:
  • Set the 1st and 4th lines to the same RED color.
  • Set the 2nd and 5th lines to the same GREEN color.
  • Set the 3rd and 6th lines to the same BLUE color.
  • In the Series Line Type column:
  • Set the 4th, 5th, and 6th to Dashed.
• Plot Formatting - settings that control the appearance of the plot, along with several miscellaneous features
   
  • On the Titles & Labels tab, set Plot Title to ACC / CSI vs. Lead Time
  • On the Titles & Labels tab, set X label to Forecast Lead
  • On the Titles & Labels tab, set Y1 label to ACC / CSI
  • On the Common tab, select Display Number of Stats
• Click the Generate Plot button at the top of the page - to submit your plot request to the METviewer plotting engine.

METviewer: Time Series of Categorical Statistics Plot Output

If you successfully followed the instructions on the previous page, you should see a plot appear in the in the plot tab of the METviewer window. Your plot should look like this. And here is the corresponding plot xml.

Studying the plot shows that the Critical Success Index (CSI) decreases for higher thresholds, as if often the case. This suggests that the forecast models have more skill at lower precipitation thresholds. This contrasts with the story told by the accuracy statistic, which increases as the threshold increases. This can be explained by the simplicity of the accuracy statistic, and the effect that always predicting no event can have for extreme events.

The following section discusses the output that METviewer generates for each plot. Click through the tabs above the METviewer plot to examine each:

• XML Tab: When the Generate Plot button is clicked, the web application serializes the information in the web page controls into an XML document and sends it across the internet to METviewer. This XML document can be used to generate a plot directly from METviewer using the command line interface, also called the batch engine. The batch engine can create many plots from a single XML document, using only small changes. Thus, a good way to use the METviewer web application is to "design" plots and then take the XML for the plot and extend it to create many different plots.
• Log Tab: The METviewer plot engine generates status output as it runs, and this information can be useful for finding problems.
• R script Tab: METviewer uses the R statistics package to generate plots by constructing an R script and running it in R. The R script that is constructed can be saved and modified if the users would like to make changes to the plot that are not supported in METviewer.
• R data Tab: Along with the R script generated by METviewer for the plot, the plot data is stored in a file that is read when the plot is generated in R. This file can be useful for problem solving, since it shows all of the data used in the plot and can be loaded directly into R for analysis.
• SQL Tab: All of the MET verification statistics output is stored in a database, which METviewer searches when it generates a plot. The SQL generated by METviewer to gather the data for a plot can be viewed, in case the user wants to explore or debug using the SQL directly.

METviewer: Object Based Attribute Area Box Plot

For the next plot, you'll try out the XML upload feature. Each plot created by METviewer corresponds to an XML file that defines the plot. In this example, you'll load the XML from a previous plot and regenerate it.

• Right click on this XML file, select the Save Link As option, and save the file to your machine.
   
• In the METviewer window, click on the Load XML button in the top-right corner.
   
• In the Upload plot XML file dialog box, click the Browse button, and navigate the location of the XML file.
   
• Select the XML file, click the Open button, and then the OK button.
   
• METviewer will now populate all of the selections with those specified in that XML file.
   
• Lastly, click the Generate Plot button at the top of the page to create the image. Your plot should look like this.
   

This plot shows the area of MODE objects defined for 1-hourly accumulated precipitation thresholded at 0.1 inch. The data has been filtered down to the 00Z initialization and the areas are plotted by lead time. Each lead time has 3 boxplots: core1 in red, core2 in blue, and the observations in gray. The numbers in black across the top show the number of objects plotted for each lead time.

Note that we have only plotted the observations once since they should be the same for both models. Look in the Series Formatting section at the bottom of the METviewer webapp in the Hide column to see that we're hiding the 4th series titled core2_hwt APCP_01 AREA_OSA. Also note that in the Legend Text column, we've specified our own strings rather than using the METviewer defaults.

The XML Upload feature of METviewer is very powerful and has saved users a lot of time by uploading previous work rather than starting from scratch each time.

Feel free to experiment with METviewer and make additional plots.

End of Practical Session 5

Congratulations! You have completed Session 5!