Skip to content

Commit

Permalink
Read derived variables from other MIP tables (#2256)
Browse files Browse the repository at this point in the history
  • Loading branch information
bouweandela authored Jan 18, 2024
1 parent a9e2f51 commit 9debbbc
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 12 deletions.
3 changes: 2 additions & 1 deletion doc/quickstart/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -683,7 +683,8 @@ related to CMOR table settings available:
extended with variables from the :ref:`custom_cmor_tables` (by default loaded
from the ``esmvalcore/cmor/tables/custom`` directory) and it is possible to
use variables with a ``mip`` which is different from the MIP table in which
they are defined.
they are defined. Note that this option is always enabled for
:ref:`derived <Variable derivation>` variables.
* ``cmor_path``: path to the CMOR table.
Relative paths are with respect to `esmvalcore/cmor/tables`_.
Defaults to the value provided in ``cmor_type`` written in lower case.
Expand Down
16 changes: 9 additions & 7 deletions doc/recipe/preprocessor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -201,13 +201,15 @@ the ozone 3D field. In this case, a derivation function is provided to
vertically integrate the ozone and obtain total column ozone for direct
comparison with the observations.

To contribute a new derived variable, it is also necessary to define a name for
it and to provide the corresponding CMOR table. This is to guarantee the proper
metadata definition is attached to the derived data. Such custom CMOR tables
are collected as part of the `ESMValCore package
<https://github.com/ESMValGroup/ESMValCore>`_. By default, the variable
derivation will be applied only if the variable is not already available in the
input data, but the derivation can be forced by setting the appropriate flag.
The tool will also look in other ``mip`` tables for the same ``project`` to find
the definition of derived variables. To contribute a completely new derived
variable, it is necessary to define a name for it and to provide the
corresponding CMOR table. This is to guarantee the proper metadata definition
is attached to the derived data. Such custom CMOR tables are collected as part
of the `ESMValCore package <https://github.com/ESMValGroup/ESMValCore/tree/main/esmvalcore/cmor/tables/custom>`_.
By default, the variable derivation will be applied only if the variable is not
already available in the input data, but the derivation can be forced by
setting the ``force_derivation`` flag.

.. code-block:: yaml
Expand Down
8 changes: 4 additions & 4 deletions esmvalcore/cmor/table.py
Original file line number Diff line number Diff line change
Expand Up @@ -296,8 +296,8 @@ def get_variable(
pass

# If that didn't work, look in all tables (i.e., other MIPs) if
# cmor_strict=False
var_info = self._look_in_all_tables(alt_names_list)
# cmor_strict=False or derived=True
var_info = self._look_in_all_tables(derived, alt_names_list)

# If that didn' work either, look in default table if cmor_strict=False
# or derived=True
Expand All @@ -324,10 +324,10 @@ def _look_in_default(self, derived, alt_names_list, table_name):
break
return var_info

def _look_in_all_tables(self, alt_names_list):
def _look_in_all_tables(self, derived, alt_names_list):
"""Look for variable in all tables."""
var_info = None
if not self.strict:
if (not self.strict or derived):
for alt_names in alt_names_list:
var_info = self._look_all_tables(alt_names)
if var_info:
Expand Down
5 changes: 5 additions & 0 deletions tests/integration/cmor/test_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,11 @@ def test_get_variable_from_alt_names(self):
var = self.variables_info.get_variable('SImon', 'sic')
self.assertEqual(var.short_name, 'siconc')

def test_get_variable_derived(self):
"""Test that derived variable are looked up from other MIP tables."""
var = self.variables_info.get_variable('3hr', 'sfcWind', derived=True)
self.assertEqual(var.short_name, 'sfcWind')

def test_get_variable_from_custom(self):
"""Get a variable from default."""
self.variables_info.strict = False
Expand Down

0 comments on commit 9debbbc

Please sign in to comment.