Merge branch 'main' into climate_patterns_only

.github/workflows/test.yml

@@ -6,7 +6,6 @@ on:
   push:
     branches:
       - main
-      - fix_recipe_filler_bkwds_incompatibility
   schedule:
     - cron: '0 0 * * *'
 

CITATION.cff

@@ -175,6 +175,11 @@ authors:
     family-names: Hagemann
     given-names: Stefan
     orcid: "https://orcid.org/0000-0001-5444-2945"
+  -
+    affiliation: "University of Canterbury, New Zealand"
+    family-names: Hardacre
+    given-names: Catherine
+    orcid: "https://orcid.org/0000-0001-9093-4656"
   -
     affiliation: "ISAC-CNR, Italy"
     name-particle: von

doc/sphinx/source/recipes/index.rst

@@ -63,6 +63,7 @@ Atmosphere
    recipe_thermodyn_diagtool
    recipe_validation
    recipe_radiation_budget
+   recipe_aod_aeronet_assess
 
 Climate metrics
 ^^^^^^^^^^^^^^^

doc/sphinx/source/recipes/recipe_aod_aeronet_assess.rst

@@ -0,0 +1,161 @@
+.. _recipe_aod_aeronet_assess:
+
+AOD AeroNET Assess
+==================
+
+Overview
+--------
+
+This diagnostic evaluates model aerosol optical depth (AOD) against ground
+based observations from the AeroNET measurement network. Monthly mean AOD
+data is downloaded from the AeroNET website and formatted (CMORized) using the
+AERONET downloader and formatter within ESMValTool.
+
+Multiannual seasonal means are calculated from the model output and compared
+with a multiannual seasonal mean climatology generated from AeroNET
+observational data. At each AeroNET station the data are screened for validity
+according to the following default criteria:
+
+  * 1. Monthly means must be generated from at least one AOD observation in that
+    month.
+
+  * 2. Seasonal means for DJF, MAM, JJA and SON must be calculated from three 
+    monthly means, i.e. a monthly mean from December January and Feburary.
+
+  * 3. For a given year to be valid, there must be a seasonal mean for each climate
+    season i.e. DJF, MAM, JJA and SON.
+
+  * 4. For a multiannual seasonal means there must be at least five seasonaal means
+    over the time range of interest.
+
+NOTE: The code is designed to be flexible and the default criteria can be 
+changed according to the user's requirements (see the user settings below).  
+
+The evaluation is visualised by plotting model output as 2D filled contours and
+overlaying AeroNET observations at model grid cells co-located with the AeroNET
+measurement stations. Statistical data (root mean square error) is generated
+using AeroNET observations at model grid cells co-located with the AeroNET
+measurement stations.
+
+Available recipes and diagnostics
+---------------------------------
+
+Recipes are stored in esmvaltool/recipes/
+
+    * recipe_aod_aeronet_assess.yml
+
+Diagnostics are stored in esmvaltool/diag_scripts/aerosols/
+
+    * aod_aeronet_assess.py: Plot the AOD evaluation.
+    * aero_utils.py: Utility functions commonly used by aerosol assessment routines.
+
+
+User settings in recipe
+-----------------------
+
+#. Script aod_aeronet_assess.py
+
+   *Required settings for script*
+
+   * wavel: The wavelength of interest for the evaluation, currently set up for 440nm
+   * min_days_per_mon: The minimum number of days used to calculate the AOD monthly mean
+   * min_mon_per_seas: The minimum number of seasons used to calculate each
+     seasonal mean. This must be between 1 and 3.
+   * min_seas_per_year: The minimum number of seasonal means in each year. This
+     must be between 1 and 4.
+   * min_seas_per_clim: The minimum number of seasonal means used to calculate
+     the multiannual seasonal mean. This must be btween 1 and the number of years
+     of available AeroNET data.
+
+   *Optional settings for script*
+
+   * None
+
+   *Required settings for variables*
+
+   * None
+
+   *Optional settings for variables*
+
+   * None
+
+   *Required settings for preprocessor*
+
+   * None
+
+   *Optional settings for preprocessor*
+
+   * None
+
+   *Color tables*
+
+   * brewer_Spectral_11
+
+
+Variables
+---------
+
+* od440aer (atmos, monthly mean, longitude latitude time)
+
+
+Observations and reformat scripts
+---------------------------------
+
+* Note: (1) obs4MIPs data can be used directly without any preprocessing; (2) 
+  see headers of reformat scripts for non-obs4MIPs data for download
+  instructions.
+
+* The AeroNET data is downloaded from the AeroNET website using the downloader:
+
+  .. code-block:: yaml
+
+        $ esmvaltool data download AERONET.
+
+* The AeroNET data is formatteed (CMORized) using the formatter:
+
+  .. code-block:: yaml
+
+        $ esmvaltool data format AERONET.
+
+
+
+References
+----------
+* Holben B.N., T.F.Eck, I.Slutsker, D.Tanre, J.P.Buis, A.Setzer, E.Vermote, J.A.Reagan, Y.Kaufman, T.Nakajima, F.Lavenu, I.Jankowiak, and A.Smirnov, 1998: AERONET - A federated instrument network and data archive for aerosol characterization, Rem. Sens. Environ., 66, 1-16.
+
+* Holben, B.N., D.Tanre, A.Smirnov, T.F.Eck, I.Slutsker, N.Abuhassan, W.W.Newcomb, J.Schafer, B.Chatenet, F.Lavenue, Y.J.Kaufman, J.Vande Castle, A.Setzer, B.Markham, D.Clark, R.Frouin, R.Halthore, A.Karnieli, N.T.O'Neill, C.Pietras, R.T.Pinker, K.Voss, and G.Zibordi, 2001: An emerging ground-based aerosol climatology: Aerosol Optical Depth from AERONET, J. Geophys. Res., 106, 12 067-12 097.
+
+* Mulcahy, J. P., Johnson, C., Jones, C. G., Povey, A. C., Scott, C. E., Sellar, A., Turnock, S. T., Woodhouse, M. T., Abraham, N. L., Andrews, M. B., Bellouin, N., Browse, J., Carslaw, K. S., Dalvi, M., Folberth, G. A., Glover, M., Grosvenor, D. P., Hardacre, C., Hill, R., Johnson, B., Jones, A., Kipling, Z., Mann, G., Mollard, J., O’Connor, F. M., Palmiéri, J., Reddington, C., Rumbold, S. T., Richardson, M., Schitgens, N. A. J., Stier, P., Stringer, M., Tang, Y., Walton, J., Woodward, S., and Yool. A.: Description and evaluation of aerosol in UKESM1 and HadGEM3-GC3.1 CMIP6 historical simulations, Geosci. Model Dev., 13, 6383–6423, 2020
+
+Example plots
+-------------
+
+.. _fig_aod_aeronet_assess_1:
+.. figure::  /recipes/figures/aod_aeronet_assess/UKESM1-0-LL_CMIP_AERmon_historical_od440aer_gn_1994_2014_DJF.png
+   :align:   center
+
+   Evaluation of AOD at 440 nm from UKESM1 historical ensemble member r1i1p1f2 against the AeroNET climatology from ground-based observations for Dec-Jan-Feb. The multiannual seasonal mean is calculated for the model data for the period 1994-2014. The model output is overlaid with the observational climatology.
+
+.. _fig_aod_aeronet_assess_2:
+.. figure::  /recipes/figures/aod_aeronet_assess/UKESM1-0-LL_CMIP_AERmon_historical_od440aer_gn_1994_2014_MAM.png
+   :align:   center
+
+   Evaluation of AOD at 440 nm from UKESM1 historical ensemble member r1i1p1f2 against the AeroNET climatology from ground-based observations for Mar_Apr_May. The multiannual seasonal mean is calculated for the model data for the period 1994-2014. The model output is overlaid with the observational climatology.
+
+.. _fig_aod_aeronet_assess_3:
+.. figure::  /recipes/figures/aod_aeronet_assess/UKESM1-0-LL_CMIP_AERmon_historical_od440aer_gn_1994_2014_JJA.png
+   :align:   center
+
+   Evaluation of AOD at 440 nm from UKESM1 historical ensemble member r1i1p1f2 against the AeroNET climatology from ground-based observations for Jun-Jul-Aug. The multiannual seasonal mean is calculated for the model data for the period 1994-2014. The model output is overlaid with the observational climatology.
+
+.. _fig_aod_aeronet_assess_4:
+.. figure::  /recipes/figures/aod_aeronet_assess/UKESM1-0-LL_CMIP_AERmon_historical_od440aer_gn_1994_2014_SON.png
+   :align:   center
+
+   Evaluation of AOD at 440 nm from UKESM1 historical ensemble member r1i1p1f2 against the AeroNET climatology from ground-based observations for Sep-Oct-Nov. The multiannual seasonal mean is calculated for the model data for the period 1994-2014. The model output is overlaid with the observational climatology.
+
+.. _fig_aod_aeronet_assess_5:
+.. figure::  /recipes/figures/aod_aeronet_assess/UKESM1-0-LL_CMIP_AERmon_historical_od440aer_gn_1994_2014_scatter.png
+   :align:   center
+
+   Evaluation of AOD at 440 nm from UKESM1 historical ensemble member r1i1p1f2 against the AeroNET climatology from ground-based observations for Dec-Jan-Feb, Mar_Apr_May, Jun-Jul-Aug and Sep-Oct-Nov. The multiannual seasonal mean is calculated for the model data for the period 1994-2014.

esmvaltool/config-references.yml

@@ -252,6 +252,10 @@ authors:
     name: Hansson, Ulf
     institute: SMHI, Sweden
     orcid:
+  hardacre_catherine:
+    name: Hardacre, Catherine
+    institute: University of Canterbury, New Zealand
+    orcid: https://orcid.org/0000-0001-9093-4656
   hassler_birgit:
     name: Hassler, Birgit
     institute: DLR, Germany

esmvaltool/diag_scripts/aerosols/aero_utils.py

@@ -0,0 +1,193 @@
+"""Part of the ESMValTool Aerosol diagnostics.
+
+This module contains utility functions commonly used by aerosol
+assessment routines.
+"""
+
+import iris
+import numpy as np
+
+
+class AeroAnsError(Exception):
+
+    """Exception class for errors raised when model data is checked in the
+    extract_pt module.
+    """
+
+
+def add_bounds(cube):
+    """Add bounds to a cube's latitude and longitude coordinates.
+
+    Parameters
+    ----------
+    cube : Iris cube
+        Iris cube with latitude and longitude coordinates.
+
+    Returns
+    -------
+    cube : Iris cube.
+        Iris cube with bounds added to the latitude and longitude coordinates.
+    """
+
+    if not cube.coord('latitude').has_bounds():
+        cube.coord('latitude').guess_bounds()
+    if not cube.coord('longitude').has_bounds():
+        cube.coord('longitude').guess_bounds()
+
+    return cube
+
+
+def extract_pt(icube, pt_lat, pt_lon, height=None, level=None, nearest=False):
+    """Extracts given location(s) (3-D) from a cube.
+
+    Method
+    ------
+    Uses Iris module Analysis.Interpolate to extract values,
+    initially based on horizontal coordinates, and then based on
+    height, if specified.
+
+    If height ('altitude') is requested, checks if cube heights
+    include orography, i.e. HybridHeights have been derived.
+
+    Parameters
+    ----------
+    icube : Iris cube
+    pt_lat, pt_lon : Float or list/array of floats. Latitude and longitude
+                     coordinates of desired points.
+    args:
+        height  : Float or list/array of floats. Altitude (above geoid) of
+                  point. Initialized to None.
+        level   : Integer . Model level or pseudo level or tile number.
+                  Initialized to None, meaning that all available levels in
+                  the cube are used.
+        nearest : Boolean. Specify whether to use 'nearest neighbour', instead
+                  of 'linear' method while extracting data. Default is False.
+
+    Returns
+    -------
+    data_out : List
+        List of single point values, corresponding to each point specified.
+
+    Raises
+    ------
+    AeroAnsError : If the number of latitude and longitude points are
+        mismatched. OR if both and level and height are passed as args.
+        OR if the cube contains a time coordinate. OR if a pseudo level
+        coordinate is requested, but not present in the cube. OR if the numbers
+        of latitude/longitude and height points are mismatched. OR if height
+        is requested but the cube does not contain an altitude coordinate.
+    """
+
+    # Check that input data is a (single) cube
+    if not isinstance(icube, iris.cube.Cube):
+        raise AeroAnsError('Extract_pt:First argument must be a single cube')
+
+    # Check if the cube contains a time dimension, which is
+    # currently unsupported.
+    if icube.coords()[0].name() == 'time':
+        raise AeroAnsError(
+            'Extract_pt:Cannot handle time dimension at present')
+
+    # Check that equal number of lat/lon pairs are passed in point coordinates.
+    # Convert arguments to lists for easier processing if necessary.
+    pt_lat1 = []
+    pt_lon1 = []
+
+    if not isinstance(pt_lat, list):
+        pt_lat1.append(pt_lat)
+        pt_lon1.append(pt_lon)
+
+    else:
+        for n_lat in np.arange(len(pt_lat)):
+            pt_lat1.append(pt_lat[n_lat])
+            pt_lon1.append(pt_lon[n_lat])
+
+    if len(pt_lat1) != len(pt_lon1):
+        raise AeroAnsError('Extract_pt:Mismatch in number of lat/long values')
+
+    # Check that both level and height haven't been requested.
+    if level is not None and height is not None:
+        raise AeroAnsError('Extract_pt: Both Level and Height requested')
+
+    # Check that the cube has a level coordinate if level has been requested.
+    if level is not None and not icube.coord(
+            'model_level_number') and not icube.coord('pseudo_level'):
+        raise AeroAnsError('Extract_pt:Level requested, but not found in cube')
+
+    # Check that the number of height points is equal to the number of
+    # lat/lon pairs. Convert the argument to a list for easier
+    # processing if necessary.
+    if height is not None:
+        pt_hgt = []
+
+#        if isinstance(height, list):
+#            pt_hgt.extend(height)
+#        else:
+#            pt_hgt.append(height)
+        pt_hgt.extend(height) if \
+            isinstance(height, list) else \
+            pt_hgt.append(height)
+
+        if len(pt_lat1) != len(pt_hgt):
+            raise AeroAnsError(
+                'Extract_pt:Mismatch in number of points for lat/long/height')
+
+        # Check that heights have been merged with orography.
+        if not icube.coords('altitude'):
+            raise AeroAnsError(
+                'Extract_pt:Height requested but input data does not contain \
+                "Altitude" coordinate')
+
+        # Store the min and max altitudes from cube data so that user
+        # cannot request points located below/ above that.
+        # Will extract =min/max if beyond limits.
+        hgt_min = icube.coord('altitude').points.min()
+        hgt_max = icube.coord('altitude').points.max()
+
+    # ---------- Finished checks -- begin processing -------------------------
+
+    # If level specified, extract slice first
+    if level is not None:
+
+        try:
+            icube = icube.extract(
+                iris.Constraint(model_level_number=level))
+
+        except Exception:
+            print('Model level number not available. Use pseudo level')
+
+        else:
+            icube = icube.extract(
+                iris.Constraint(pseudo_level=level))
+
+    # Extract values for specified points lat/lon
+    # NOTE: Does not seem to handle multiple points if 3-D
+    data_out = []
+
+    # Set lat/lon coordinates for model grid cell interpolation
+    for n_lat1 in np.arange(len(pt_lat1)):
+        latlon_coords = [('latitude', pt_lat1[n_lat1]),
+                         ('longitude', pt_lon1[n_lat1])]
+
+        if nearest:
+            tcube = icube.interpolate(latlon_coords, iris.analysis.Nearest())
+        else:
+            tcube = icube.interpolate(latlon_coords, iris.analysis.Linear())
+
+        # If height specified, interpolate to requested height
+        if height is not None:
+
+            # Set vertical coordinates for model grid cell interpolation
+            point = max(pt_hgt[n_lat1], hgt_min)
+            point = min(pt_hgt[n_lat1], hgt_max)
+            hgt_coords = [('altitude', point)]
+
+            if nearest:
+                tcube = tcube.interpolate(hgt_coords, iris.analysis.Nearest())
+            else:
+                tcube = tcube.interpolate(hgt_coords, iris.analysis.Linear())
+
+        # Append processed data point
+        data_out.append(tcube.data)
+
+    return data_out