Adding basic sanity checks (#4120)

Co-authored-by: Axel Lauer <axel.lauer@dlr.de>
Co-authored-by: Manuel Schlund <manuel.schlund@dlr.de>
Co-authored-by: kirsy05 <88324002+kirsy05@users.noreply.github.com>
Co-authored-by: Kirsten Schulze <b309296@levante2.lvt.dkrz.de>
Co-authored-by: giuliacast <166797075+giuliacast@users.noreply.github.com>
Co-authored-by: Giulia Castellani <b383308@levante4.lvt.dkrz.de>
Co-authored-by: Manuel Schlund <32543114+schlunma@users.noreply.github.com>
Co-authored-by: Bettina Gier <gier@uni-bremen.de>
Co-authored-by: Julien Lenhardt <45034763+jlenh@users.noreply.github.com>

Author: 9 people

.zenodo.json

@@ -66,6 +66,11 @@
       "affiliation": "University of Bremen, Germany",
       "name": "Adeniyi, Kemisola"
     },
+    {
+      "affiliation": "University of Bremen, Germany",
+      "name": "Castellani, Giulia",
+      "orcid": "0000-0001-6151-015X"
+    },
     {
       "affiliation": "ISAC-CNR, Italy",
       "name": "Arnone, Enrico",
@@ -366,6 +371,10 @@
       "affiliation": "DLR, Germany",
       "name": "Sarauer, Ellen"
     },
+    {
+      "affiliation": "University of Bremen, Germany",
+      "name": "Schulze, Kirsten"
+    },
     {
       "affiliation": "University of Reading, UK",
       "name": "Roberts, Charles",

CITATION.cff

@@ -174,6 +174,10 @@ authors:
     affiliation: "University of Bremen, Germany"
     family-names: Gier
     given-names: Bettina
+  -
+    affiliation: "University of Bremen, Germany"
+    family-names: Castellani
+    given-names: Giulia
   -
     affiliation: "Met Office, UK"
     family-names: Gillett
@@ -358,6 +362,10 @@ authors:
     family-names: Weigel
     given-names: Katja
     orcid: "https://orcid.org/0000-0001-6133-7801"
+  -
+    affiliation: "University of Bremen, Germany"
+    family-names: Schulze
+    given-names: Kirsten
   -
     affiliation: "DLR, Germany"
     family-names: Sarauer

doc/sphinx/source/recipes/figures/sanity_checks/timeseries_ambiguous_variable_group_MPI-ESM1-2-LR_Amon_historical_r1i1p1f1.png

@@ -25,6 +25,7 @@ large variety of input data.
    recipe_model_evaluation
    recipe_monitor
    recipe_portrait
+   recipe_sanity_checks
    recipe_seaborn
    recipe_ref
 

doc/sphinx/source/recipes/index.rst

@@ -0,0 +1,81 @@
+.. _recipe_santiy_checks:
+
+Sanity checks
+=============
+
+Overview
+--------
+
+The sanity check recipe (``recipe_sanity_checks.yml``) is intended to perform
+some basic sanity checks for new simulations during model development. A subset of variables
+is selected to check whether the model is performing in reasonable way. The recipe plots global
+monthly means as well as maximum and minimum across all individual grid cells for each time step.
+This can be used to check for example for negative mass concentrations occuring at any grid cell
+at any time (minimum is less than zero) or whether individual grid cells exceed physically reasonable
+values (e.g. total cloud fraction greater than 100%).
+The global monthly means can be compared to the minimum and maximum values found in reference
+datasets (observations, reanalyses) across all months and all reference datasets. The "reasonable"
+limits are shown as red lines. A good model simulations would be expected to not exceed these
+limits at any time. The minimum and maximum global averages across all months and all reference
+datasets is calculated with ``recipe_create_ranges_obs.yml``.
+
+Available recipes and diagnostics
+---------------------------------
+
+Recipes are stored in `recipes/sanity_checks`
+
+* recipe_sanity_checks.yml
+* recipe_create_ranges_obs.yml
+
+.. note::
+
+  The time frequency used in ``recipe_create_ranges_obs.yml`` (monthly, daily, etc. values) to
+  calculate "reasonable" minimum and maximum values must match the time frequency used in
+  the sanity check recipe (``recipe_sanity_checks.yml``).
+
+Diagnostics are stored in `diag_scripts/monitor/`
+
+* :ref:`multi_datasets.py
+  <api.esmvaltool.diag_scripts.monitor.multi_datasets>`:
+  Monitoring diagnostic to show multiple datasets in one plot (incl. biases).
+
+Variables:
+----------
+
+* ps (air mass)
+* qep (moisture flux)
+* prw (water vapor mass)
+* asr
+* clivi
+* clt
+* hfls
+* hfss
+* lwcre
+* lwp
+* netcre
+* pr
+* prc
+* prw
+* rlds
+* rlut
+* rtnt
+* rsds
+* rsut
+* swcre
+* tas
+* tauu
+* tauv
+
+Example plots
+-------------
+
+.. _fig_sanity_check_clt:
+.. figure:: /recipes/figures/sanity_checks/timeseries_ambiguous_variable_group_MPI-ESM1-2-LR_Amon_historical_r1i1p1f1.png
+   :align:   center
+   :width:   14cm
+
+   Time series of monthly global average (solid line) and minimum / maximum
+   (dashed lines) total cloud cover from MPI-ESM1-2-LR. The red horizontal lines
+   show the monthly minimum and maximum global average across multiple reference
+   dataset (here: ESACCI-CLOUD, CLARA-AVHRR, PATMOS-x, MODIS, ERA5) calculated
+   with recipe_create_ranges_obs.yml. Plot created with recipe_sanity_checks.yml.

doc/sphinx/source/recipes/recipe_sanity_checks.rst

@@ -30,6 +30,12 @@ authors:
     email: diego.cammarano@dlr.de
     github: diegokam
     orcid:
+  castellani_giulia:
+    name: Castellani, Giulia
+    institute: University of Bremen
+    email: giulia.castellani@uni-bremen.de
+    github: giuliacast
+    orcid: https://orcid.org/0000-0001-6151-015X
   debeire_kevin:
     name: Debeire, Kevin
     institute: DLR, Germany
@@ -527,6 +533,11 @@ authors:
     institute: DLR, Germany
     orcid:
     github: ellensarauer
+  schulze_kirsten:
+    name: Schulze, Kirsten
+    institute: Uni Bremen, Germany
+    orcid:
+    github: kirsy05
   serva_federico:
     name: Serva, Federico
     institute: CNR, Italy