Results > CMIP mean state and variability
CMIP mean state and variability
PCMDI Simulation Summaries
The PCMDI Metrics Package (PMP) is a capability that is used to produce a diverse suite of “quick-look” objective summaries of Earth System Model (ESM) agreement with observations. The PMP is routinely applied to multiple generations of CMIP, including the most recent results from CMIP6 as they become available. These results are regularly updated as additional simulations become available, new analysis are included, and as presentation improvements and corrections are made.
Quick-look summary statistics with interactive navigation to underlying diagnostics
- Mean Climate
- Benchmarking Simulated Precipitation
- El Niño–Southern Oscillation (ENSO)
- Extratropical Modes of Variability
- Madden-Julian Oscillation (MJO)
- Monsoon Characteristics (example)
- Update history
Mean Climate (results)
- Using well-established statistics, we provide large-scale seasonal and mean state climatology comparisons between CMIP simulations and observationally-based data. These include traditional measures (e.g. bias, pattern correlation and root-mean-square error) for global, hemispheric, tropical, extra-tropical, and other selected domains using satellite data and atmospheric reanalysis as references. These statistics are routinely computed as part of model evaluation. We use summary diagrams developed by PCMDI scientists (Taylor 2001; Gleckler et al. 2008) to objectively compare the consistency between the observed and simulated climate.
Benchmarking Simulated Precipitation (results)
- These results were inspired by the outcomes of a July 2019 DOE workshop. Several teams were established at this workshop with one group tasked to incorporate an initial set of benchmarks into a common analysis framework and applying it to CMIP6 and earlier generations of climate models (Pendergrass, et al., 2019). The results presented here illustrate the progress of this benchmarking effort. In parallel, a second group continues to develop exploratory metrics. Ultimately, this effort aims to provide a guide to modelers as they strive to improve simulated precipitation.
El Niño-Southern Oscillation (results)
- El Niño-Southern Oscillation (ENSO) is the dominant mode of interannual variability in the tropical Pacific and has far reaching impacts on global climate. It is there therefore key to ensure its correct simulation in state-of-the-art climate models. Community-wide synthesis of metrics to evaluate the performance, teleconnections and processes of ENSO in coupled GCMs is proposed by the ENSO working group of the International CLIVAR Pacific panel. The corresponding objective comparisons of simulations against observations shown here result from a collaboration between scientists at Institut Pierre Simon Laplace (IPSL) and PCMDI. This effort strives to improve and expand upon the ENSO model performance tests proposed by Bellenger et al. (2014) for CMIP5. The metrics are demonstrated through application to the CMIP archive following works of Planton et al. (2020, BAMS, under review).
Extratropical Modes of Variability (results)
- Based on the work of Lee et al. (2019), we present skill metrics for the Northern Annular Model (NAM), the North Atlantic Oscillation (NAO), the Southern Annular Mode (SAM), the Pacific North American pattern (PNA), the North Pacific Oscillation (NPO), the Pacific Decadal Oscillation (PDO), and the North Pacific Gyre Oscillation (NPGO). For NAM, NAO, SAM, PNA, and NPO the results are based on sea-level pressure, while the results for PDO and NPGO are based on sea surface temperature. Our approach distinguishes itself from other studies that analyze modes of variability in that we use the Common Basis Function approach (CBF), in which model anomalies are projected onto the observed modes of variability. Using the Historical simulations, the skill of the spatial patterns is given by the Root-Mean-Squared-Error (RMSE), and the Amplitude gives the standard deviation of the Principal Component time series. The skill metrics are calculated with respect to a primary and secondary sets of observations denoted by the triangles in each cell of the Portrait Plots.
Madden-Julian Oscillation (results)
- Based on the work of Ahn et al. (2017), we present skill metrics that indicate how well models simulate eastward propagation of the MJO. We apply frequency-wavenumber decomposition to precipitation from observations (GPCP-based; 1997-2010) and the CMIP5 and CMIP6 Historical simulations (1985-2004).
Monsoon Characteristics (example) (results)
- Based on the work of Sperber and Annamalai (2014), we present skill metrics that indicate how well models simulate the onset, decay, and duration of monsoon based on the analysis of climatological pentads of precipitation. Using Historical simulations, the results are based on area-averaged data for All-India Rainfall (AIR), Sahel, Gulf of Guinea (GoG), North American Monsoon (NAM), South American Monsoon (SAM), and Northern Australia (AUS).
Questions or comments about the PCMDI Simulation Summaries should be sent to the PMP team.
The efforts of the authors are supported by the Regional and Global Climate Modeling Program of the United States Department of Energy’s Office of Science. This work is funded by the Climate and Environmental Sciences Division of the DOE Office of Science and is performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. LLNL-WEB-812310