1. Overview

EnKF History and Background

The ensemble Kalman filter (EnKF) is a Monte-Carlo algorithm for data assimilation that uses an ensemble of short-term forecasts to estimate the background-error covariance in the Kalman filter. Each ensemble member is cycled through the data assimilation system and updated by EnKF. The EnKF code was developed by the National Oceanic and Atmospheric Administration (NOAA) Earth System Research Lab (ESRL) in collaboration with the research community. It contains two separate algorithms for calculating an analysis increment, a serial Ensemble Square Root Filter (EnSRF) algorithm described by and a Local Ensemble Kalman Filter (LETKF) algorithm described by . The parallelization scheme used by the EnSRF algorithm is based on that used in the Data Assimilation Research Testbed (DART) toolkit developed at the National Center for Atmospheric Research (NCAR) and described by . The LETKF code was contributed by Yoichiro Ota of the Japanese Meteorological Agency (JMA) while he was a visitor at the National Centers for Environmental Prediction (NCEP).

The EnKF code became an operational data assimilation system at NCEP in May 2012, providing ensemble update to the Global Forecast System (GFS) hybrid Ensemble-Variational (EnVar) data assimilation system. It can work with both global forecast models (e.g., GFS) and regional forecast models (e.g., the Hurricane Weather Research and Forecasting (WRF) (HWRF) model, the North American Mesoscale (NAM) system, the Advanced Research WRF (ARW) system).

EnKF Becomes Community Code

The Developmental Testbed Center (DTC), in collaboration with major development groups, began transforming the EnKF operational system into a community system in 2014, following the same protocol as the GSI community effort ( http://www.dtcenter.org/com-GSI/users/). Consequently, the EnKF code and its user support are managed by the DTC, along with the GSI community system.

The DTC complements the development groups in providing EnKF documentation, porting EnKF to multiple platforms, and testing EnKF in an independent and objective environment, while still maintaining functionally equivalent to operational centers. Working with code developers, the DTC is maintaining a unified community GSI/EnKF repository to facilitate community users to develop EnKF. Based on the repository, the DTC releases EnKF code annually with GSI. The first community version of the EnKF system was released on July 31, 2015. This users guide describes the third release of EnKF (v1.3) in October 2018. The DTC provides user support through the EnKF Helpdesk (enkf-help@ucar.edu), tutorials, and workshops. More information about the EnKF community services can be found at the DTC EnKF webpage (http://www.dtcenter.org/EnKF/users/).

EnKF Code Management and Review Committee

The EnKF code development and maintenance are administrated by the Data Assimilation Review Committee (DARC). DARC was originally formed as the GSI Review Committee in 2010. The committee was reformed in 2014 to include members representing the EnKF development and applications. Such a combination enhanced collaboration of development groups in both variational and ensemble data assimilation communities. Currently, DARC contains members from NCEP’s Environmental Modeling Center (EMC), the National Aeronautics and Space Administration (NASA) Goddard Global Modeling and Assimilation Office (GMAO), NOAA/ESRL, the National Center for Atmospheric Research (NCAR) Mesoscale & Microscale Meteorology Laboratory (MMM), the National Environmental Satellite, Data, and Information Service (NESDIS), the Unite States Air Force (USAF), the University of Maryland, Joint Cetner for Satellite Data Assimilation (JCSDA), and the DTC.

DARC primarily steers distributed GSI/EnKF development and community code management and support. The responsibilities of the committee are divided into two major aspects: coordination and code review. The purpose and guiding principles of the review committee are described in the Chapter 1 of the GSI Users Guide.

Community Code Contributions

EnKF is a community data assimilation system, open to contributions from scientists and software engineers from both the operational and research communities. DARC oversees the code transition from prospective contributors. This committee reviews proposals for code commits to the GSI/EnKF repository and monitors that coding standards and tests are being fulfilled. Once the committee reaches approval, the contributed code will be committed to the GSI/EnKF code repository and available for operational implementation and public release.

To facilitate this process, the DTC is providing code transition assistance to the general research community. Prospective contributors should contact the DTC EnKF helpdesk (enkf-help@ucar.edu) as early as possible. It is the contributor’s responsibility to ensure that a proposed code change is correct, meeting the EnKF coding standards, and well-documented. The DTC will help the contributors run the regression tests and merge the code with the top of the repository. Prospective contributors can also apply to the DTC visitor program for their EnKF research and code transition. The visitor program is open to applications year-round. Please check the visitor program webpage (www.dtcenter.org/visitors/) for the latest announcement of opportunity and application procedures.

About This EnKF Release

This users guide was composed for the EnKF community release version(v) 1.3, which is compatible with the GSI community release v3.7. Please note the major focuses of the DTC are currently on testing and evaluation of EnKF for regional numerical weather prediction (NWP) applications though the instructions and cases for EnKF global applications are available with this release.

Running this EnKF system requires running GSI as a prior for its observation operators. Therefore, the GSI Users Guide is referred throughout this documentation. This GSI user’s guide can be obtained at the GSI user’s webpage (http://www.dtcenter.org/com-GSI/users/docs/index.php).

What Is New in This Release Version

The following lists some of the new functions and changes included in the v1.3 release of the EnKF versus v1.2:

  • Added the capability to perform model-space vertical localization using modulated ensembles
  • Added option for GSI to write out diag files in NetCDF format as well as binary format
  • Added option for EnKF to read NetCDF format diag files
  • Added option of running GSI in the observer mode once for the ensemble mean and saving the observation operator jacobian in diag files. The jacobian is then used in EnKF for computing linearized observation operator for ensemble perturbations
  • Added option for EnKF to save ensemble spread in diag files
  • bug fixes

Observations Used by This Version

EnKF is using the GSI system as the observation operator to generate observation innovations. Therefore, the observation types assimilated by EnKF are the same as GSI. This version of EnKF has been tested to work with the community GSI release v3.7. It can assimilate, but is not limited to, the following types of observations:

Conventional observations (including satellite retrievals):

  • Radiosondes
  • Pilot ballon (PIBAL) winds
  • Synthetic tropical cyclone winds
  • Wind profilers: USA, Jan Meteorological Agency (JMA)
  • Conventional aircraft reports
  • Aircraft to Satellite Data Relay (ASDAR) aircraft reports
  • Meteorological Data Collection and Reporting System (MDCRS) aircraft reports
  • Dropsondes
  • Moderate Resolution Imaging Spectroradiometer (MODIS) IR and water vapor winds
  • Geostationary Meteorological Satellite (GMS), JMA, and Meteosat cloud drift IR and visible winds
  • European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) and GOES water vapor cloud top winds
  • GEOS hourly IR and cloud top wind
  • Surface land observations
  • Surface ship and buoy observation
  • Special Sensor Microwave Imager (SSMI) wind speeds
  • Quick Scatterometer (QuikSCAT), the Advanced Scatterometer (ASCAT) and Oceansat-2 Scatterometer (OSCAT) wind speed and direction
  • RapidScat observations
  • SSM/I and Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) precipitation estimates
  • Velocity-Azimuth Display (VAD) Next Generation Weather Radar ((NEXRAD) winds
  • Global Positioning System (GPS) precipitable water estimates
  • Sea surface temperature (SST)
  • Doppler wind Lidar
  • Aviation routine weather report (METAR) cloud coverage
  • Flight level and Stepped Frequency Microwave Radiometer (SFMR) High Density Observation (HDOB) from reconnaissance aircraft
  • Tall tower wind

Satellite radiance/brightness temperature observations (instrument/satellite ID):

  • SBUV: NOAA-17, NOAA-18, NOAA-19
  • High Resolution Infrared Radiation Sounder (HIRS): Meteorological Operational-A (MetOp-A), MetOp-B, NOAA-17, NOAA-19
  • GOES imager: GOES-11, GOES-12
  • Atmospheric IR Sounder (AIRS): aqua
  • AMSU-A: MetOp-A, MetOp-B, NOAA-15, NOAA-18, NOAA-19, aqua
  • AMSU-B: MetOp-B, NOAA-17
  • Microwave Humidity Sounder (MHS): MetOp-A, MetOp-B, NOAA-18, NOAA-19
  • SSMI: DMSP F14, F15, F19
  • SSMI/S: DMSP F16
  • Advanced Microwave Scanning Radiometer for Earth Observing System (AMSR-E): aqua
  • GOES Sounder (SNDR): GOES-11, GOES-12, GOES-13
  • Infrared Atmospheric Sounding Interferometer (IASI): MetOp-A, MetOp-B
  • Global Ozone Monitoring Experiment (GOME): MetOp-A, MetOp-B
  • Ozone Monitoring Instrument (OMI): aura
  • Spinning Enhanced Visible and Infrared Imager (SEVIRI): Meteosat-8, Meteosat-9, Meteosat-10
  • Advanced Technology Microwave Sounder (ATMS): Suomi NPP
  • Cross-track Infrared Sounder (CrIS): Suomi NPP
  • GCOM-W1 AMSR2
  • GPM GMI
  • Megha-Tropiques SAPHIR
  • Himawari AHI

Others:

  • GPS Radio occultation (RO) refractivity and bending angle profiles
  • Solar Backscatter Ultraviolet (SBUV) ozone profiles, Microwave Limb Sounder (MLS) (including NRT) ozone, and Ozone Monitoring Instrument (OMI) total ozone
  • Doppler radar radial velocities
  • Radar reflectivity Mosaic
  • Tail Doppler Radar (TDR) radial velocity and super-observation
  • Tropical Cyclone Vitals Database (TCVital)
  • Particulate matter (PM) of 10-um diameter, 2.5-um diameter or less
  • MODIS AOD (when using GSI-chem package)

Please note some of these above mentioned data are not yet fully tested and/or implemented for operations. Therefore, the current GSI code might not have the optimal setup for these data.