What we do
The DTC conducts rigorous testing and evaluation (T&E) for cutting-edge models, technologies, and techniques to inform Earth System Model (ESM) development. We develop assessment solutions for ESM applications, from high-resolution, limited-area to global, coupled forecasting systems. Testing can address innovations touching on initialization, physical parameterizations, ensemble design, Artificial Intelligence (AI)/machine learning (ML)-based prediction systems, or modeling techniques that leverage AI/ML. Tests can focus on general prediction or specific phenomena, such as tropical cyclones. Not only does DTC facilitate operational implementations but it also provides a meaningful resource for assessing the impact of research from the community.
Core to our T&E activities is the enhanced Model Evaluation Tools (METplus) and the Common Community Physics Package (CCPP) with its companion Single-Column Model, both developed and maintained by the DTC through collaborative efforts with the broader community. We support community physics development through enhanced model physics interoperability, and provide a unified and open-source model evaluation software package that is actively used by the larger research community. In addition to supporting DTC T&E activities, these tools are used for operational NWP and evaluation both within the United States and internationally.
How we do it
Our scientific evaluations span a hierarchy of approaches from case studies to extended retrospective periods. Case studies provide detailed and in-depth diagnostic information that helps characterize processes the model configuration is struggling to accurately represent, whereas robust samples, which cover a broad range of weather regimes, provide essential information on the overall performance. The design of each evaluation is created in consultation with developers, relevant area experts and verification experts. Our goal is to apply the most advanced tools to our T&E activities that both document ESM performance and provide information to enhance the development process. The development and maintenance of automated end-to-end workflows plays an important role in the DTC’s ability to provide timely feedback to the development process.
Close collaboration with the broader community is essential for both our end-to-end T&E and software development. By partnering with academic researchers, we test exploratory ideas for improving forecast accuracy. By collaborating with government agencies and operational centers, we provide evaluations of various model components that are being considered for operations, and develop tools that become integral components of operational forecasting systems. Meanwhile, our collaborators and visitors from the broader community contribute valuable feedback to our tools and model assessment methods, significantly enhancing the quality of our work.
