Model Evaluation Packages
This table describes the metrics and diagnostics available in CMEC packages
| Package | Contents | Source | Documentation |
|---|---|---|---|
| Analyzing Scales of Precipitation (ASoP) | Coherence: Temporal intermittency metrics, spatial intermittency metrics, plots of correlations of precipitation in space and time over regional blocks.
Spectral: Histogram overlap metric, file (for each dataset) containing 100-bin distribution of counts over the time period for each gridpoint, plots of 1d histograms of precipitation, maps of actual and fractional contribution to precipitation rate by intensity range |
GitHub repository | User Guide |
| Cyclone Metrics Package (CyMeP) | Statistics: mean, bias, Pearson correlation, Spearman correlation, hit rate, false alarm ratio
Statistics measure the following storm characteristics: tropical cyclone days, storm genesis, storm intensity (minimum sea level pressure, maximum 10-meter wind speed, accumulated cyclone energy, pressure accumulated cyclone energy), latitude of lifetime-maximum intensity Figures: Scorecards, cyclone trajectories, spatial density maps, Taylor diagrams, interannual cycle line plots, seasonal cycle line plots |
GitHub repository | CyMeP README |
| Drought Metrics | Metrics: Mean precipitation, mean SPI6, mean SPI36, seasonality of precipitation, long term monthly mean, fractional area coverage of drought, proportion of dry months, count of dry months, drought intensity, probability of drought initiation, probability of drought termination, total score
Figures: Taylor diagram, heatmap of total scores |
GitHub repository | Drought Metrics README |
| International Land Model Benchmarking (ILAMB) | Metrics: A set of scores for spatial and temporal statistics of a model compared with observations for different quantities. Scores include overall score, bias score, RMSE score, and others.
Metrics categories include Ecosystem and Carbon Cycle, Hydrology Cycle, Radiation and Energy Cycle, Forcings, and Relationships (Gross Primary Productivity, Leaf area index, and Evapotranspiration). Figures: contour maps, time series plots, Taylor diagrams |
GitHub repository | Documentation website |
| International Ocean Model Benchmarking (IOMB) | Metrics: A set of scores that account for spatial and temporal statistics of a model compared with observations for different quantities. Scores include overall score, bias score, RMSE score, and others.
Metrics categories include Ecosystems, Nutrients, Carbon, Physical Drivers, and Relationships. Figures: contour maps, time series plots, Taylor diagrams |
GitHub repository (shared with ILAMB) | Documentation website (shared with ILAMB) |
| NOAA Model Diagnostics Task Force (MDTF) | Process Oriented Diagnostics (PODs):
ENSO_MSE ENSO_RWS Extratropical Variance Madden-Julian Oscillation Propogation and Amplitude Madden-Julian Oscillation Spectra and Phasing Madden-Julian Oscillation Teleconnections Coupling between Soil Moisture and Evapotranspiration Wavenumber-Frequency Spectra Convective Transition Diagnostics Mixed Layer Depth Ocean Sufrace Latent Heat Flux Diagnostic Diurnal Cycle of Precipitation Surface Temperature Extremes and Distribution Shape Tropical Pacific Sea Level Diagnostic |
GitHub repository | Documentation website Project Website |
| PCMDI Metrics Package (PMP) | Mean Climate: Statistics that characterize atmospheric variables and compare models with observations, including bias, mean, correlation, root mean square error, and standard deviation.
Monsoon (Wang): Correlation, rmsn, threat score for defined regions (North America, South America, North Africa, South Africa, Asia, Australia, and "all" monsoon domains). Monsoon (Sperber): Decay index, duration, onset index, slope for defined regions (All-India, Northern Australia, Sahel, Gulf of Guinea, North America, South America). Diurnal Cycle: Daily mean precipitation, standard deviation of mean diurnal cycle, standard deviation of hourly means, standard deviation of daily means, Fourier harmonics (amplitude and phase) Modes of Variability: Statistics for the Northern Annular Mode, Northern Atlantic Oscillation, Sourthern Annular Mode, Pacific North American Pattern, Pacific Decadal Oscillation. Madden-Julian Oscillation metrics: East power, West power, East-west power ratio El Niño–Southern Oscillation (ENSO) performance: Cold tongue bias, trade wind bias, dry equator bias, double intertropical convergence zone bias, seasonalities of biases, ENSO amplitude, ENSO life cycle, ENSO asymmetry, ENSO diversity El Niño–Southern Oscillation teleconnections: ENSO amplitude, ENSO pattern, ENSO seasonality, DJF precipitation teleconnection, DJF surface temperature teleconnection, JJA precipitation teleconnection, JJA surface temperature teleconnection El Niño–Southern Oscillation processes: Cold tongue bias, trade winds bias, ENSO pattern, ENSO amplitude, ENSO seasonality, ENSO asymmetry, ocean-atmosphere Bjerknes feedback (atmosphere branch, ocean-atmosphere branch, and ocean branch), balance of Bjerknes feedback and heat fluxes on sea surface temperature anomalies Precipitation variability: Forced annual, forced semi-annual, unforced interannual, unforced seasonal-annual, unforced sub-seasonal, unforced synoptic |
GitHub repository | Documentation website |
| xWMT | Surface Water Mass Transformation | GitHub repository | README |