Skip to content

Commit

Permalink
Merge branch 'main' into update_nsidc_g02202
Browse files Browse the repository at this point in the history
  • Loading branch information
@rbeucher
rbeucher authored Jan 30, 2025
2 parents 11cf6f2 + 983de78 commit 9344628
Show file tree
Hide file tree
Showing 9 changed files with 276 additions and 308 deletions.
177 changes: 101 additions & 76 deletions doc/sphinx/source/utils/RTW/add_a_recipe.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,116 +3,141 @@ How to add a recipe to the |RTW|

.. include:: common.txt

.. note::
Before you follow these steps to add your recipe, you must be able to
successfully run the recipe with the latest version of ESMValTool on the
compute server you use at your site, as detailed by the ``platform`` option
in the ``[[COMPUTE]]`` section in the site-specific ``.cylc`` file in the
``esmvaltool/utils/recipe_test_workflow/site/`` directory.
Overview
--------

#. Open a `new ESMValTool issue`_ on GitHub, assign yourself to the issue, and
add the ``Recipe Test Workflow (RTW)`` label to the issue, see
`ESMValTool issue #3663`_ for an example.
To add a recipe to the |RTW| you will:

#. Create a branch.
* Run the recipe at your site
* Note the actual duration and memory usage
* Edit your site's recipe file
* Create the recipe's KGOs
* Request a review

#. Obtain the duration and memory usage of the recipe from the messages printed
to screen, or at the end of the ``run/main_log.txt`` file in the recipe
output directory after running your recipe on the compute cluster you use at
your site; these messages will look something like::
The recipe will then run at your site whenever the |RTW| is run.

YYYY-MM-DD HH:MM:SS:sss UTC [12345] INFO Time for running the recipe was: 0:02:13.334742
YYYY-MM-DD HH:MM:SS:sss UTC [12345] INFO Maximum memory used (estimate): 2.4 GB
[...]
YYYY-MM-DD HH:MM:SS:sss UTC [12345] INFO Run was successful
Preparation
-----------

#. Add the recipe to the ``[task parameters]`` section in the
``esmvaltool/utils/recipe_test_workflow/flow.cylc`` file.
#. Open a `new ESMValTool issue`_ on GitHub. Assign yourself to the issue and
add the ``Recipe Test Workflow (RTW)`` label. `ESMValTool issue #3663`_
provides an example.

.. hint::
If the recipe takes less than 10 minutes to run then it should be added
to the ``fast`` option. Recipes that take longer than ten minutes should
be added to the ``medium`` option.
#. Create a branch.

#. Run the recipe:

* with the latest version of ESMValTool
* on the compute server you use at your site

.. hint::
The line added should follow the format of ``recipe_new_recipe, \``,
unless the line is the last one in the list, in which case the line added
should follow the format of ``recipe_new_recipe``.

#. If the duration of the recipe is larger than the value specified by the
``execution time limit`` option in the ``[[COMPUTE]]`` section in the
aforementioned site-specific ``.cylc`` file, and / or the memory usage of
the recipe is larger than the value specified by the ``--mem`` option in the
``[[[directives]]]`` section in the ``[[COMPUTE]]`` section, add a section
(in alphabetical order) to this file as shown below (round the duration to
the nearest second)::

[[process<fast=recipe_albedolandcover>]]
# Actual: 0m31s, 2.5 GB on 2024-04-08.
execution time limit = PT2M
[[[directives]]]
--mem = 3G
Your compute server is defined in the
``esmvaltool/utils/recipe_test_workflow/site/<site>.cylc`` file as
follows::

[[COMPUTE]]
platform = <your compute server>

#. Obtain the actual duration and memory usage of the recipe. This can be found
either in the message printed to screen, or at the end of the
``run/main_log.txt`` file in the recipe output directory. The relevant lines will
look something like::

YYYY-MM-DD HH:MM:SS:sss UTC [12345] INFO Time for running the recipe was: 0:02:13.334742
YYYY-MM-DD HH:MM:SS:sss UTC [12345] INFO Maximum memory used (estimate): 2.4 GB
[...]
YYYY-MM-DD HH:MM:SS:sss UTC [12345] INFO Run was successful

Adding the recipe
-----------------

#. Add the recipe in alphabetical order to either ``FAST_RECIPES`` or
``MEDIUM_RECIPES`` in the ``<site>-recipes.jinja`` file. It should look
something like::

{
'recipe_path': 'recipe_a_fast_recipe',
'actual': '2m13s, 2.4 GB on YYYY-MM-DD',
'max_time': 'PT3M',
'max_memory': '3G',
}

.. important::
Add the recipe to ``FAST_RECIPES`` if it takes *less* than 10 mins to
run at your site. Add the recipe to ``MEDIUM_RECIPES`` if it takes *more*
than 10 mins.

.. hint::
The ``fast`` key in the example task definition above
(``[[process<fast=recipe_albedolandcover>]]``) should match name of the
option the recipe was added to in the ``[task parameters]`` section in
the ``esmvaltool/utils/recipe_test_workflow/flow.cylc`` file
The :ref:`site/site-recipes.jinja <site_recipes_file>`.
file provides more information.

.. hint::
Set the ``execution time limit`` to 10-20% more than the actual duration.
For actual durations of up to ``1m45s``, set the ``execution time limit``
to ``PT2M`` (2 minutes).
Set the ``max_time`` to 10-20% more than the actual duration. For actual
durations of up to ``1m45s``, set ``max_time`` to ``PT2M`` (2 minutes).

.. hint::
Try not to regularly waste more than 500 MiB in memory usage. Typically,
rounding the actual memory usage up to the nearest integer is acceptable.

#. Stop any running ``recipe_test_workflow`` workflows::

cylc stop recipe_test_workflow/*
Create the |KGOs|
-----------------

#. Run the |RTW|, as detailed in the :ref:`quick_start_guide`; it is expected
that the ``compare`` task will fail.

#. Update the Known Good Outputs (|KGOs|):
.. important::
The ``compare`` task fails because the |KGOs| for the recipe do not yet
exist. This run of the |RTW| will generate the outputs that will be
used as |KGOs|.

* Recursively copy the recipe output directory (i.e.
``recipe_<recipe>_<date>_<time>/``) from the
``${HOME}/cylc-run/recipe_test_workflow/runN/share/cycle/<cycle>/``
directory to your site-specific |KGO| directory, as detailed by the
``KGO_ROOT_PATH`` option in the site-specific ``.conf`` file in the
``esmvaltool/utils/recipe_test_workflow/opt/`` directory::
#. Recursively copy the recipe output directory
``recipe_<recipe>_<date>_<time>/`` to your site-specific |KGO| directory::

cp -r ${HOME}/cylc-run/recipe_test_workflow/runN/share/cycle/<cycle>/recipe_<recipe>_<date>_<time> <KGO_ROOT_PATH>
cp -r ${HOME}/cylc-run/recipe_test_workflow/runN/share/cycle/<cycle>/recipe_<recipe>_<date>_<time> <KGO_ROOT_PATH>

.. note::
Cylc is typically configured such that
``${HOME}/cylc-run/recipe_test_workflow/runN/share`` is a symbolic link
to a share directory located on a scratch disk; the recipe output
directory is not stored in ``${HOME}``.
.. hint::
``<cycle>`` will look something like: ``20250101T0900Z``. The recipe output
directory will look something like: ``recipe_python_20250101_090000``

.. hint::
Find your site specific ``KGO_ROOT_PATH`` in the ``rose-suite-<site>.conf`` file
in the ``esmvaltool/utils/recipe_test_workflow/opt/`` directory.

.. note::
Cylc is typically configured such that
``${HOME}/cylc-run/recipe_test_workflow/runN/share`` is a symbolic link
to a share directory located on a scratch disk; the recipe output
directory is not stored in ``${HOME}``.

* Enable write permissions for all users on the recipe output directory in
your site-specific |KGO| directory::
#. Enable write permissions for all users on the recipe output directory in
your site-specific |KGO| directory::

chmod -R a+w <KGO_ROOT_PATH>/recipe_<recipe>_<date>_<time>
chmod -R a+w <KGO_ROOT_PATH>/recipe_<recipe>_<date>_<time>

#. Stop any running ``recipe_test_workflow`` workflows::

cylc stop recipe_test_workflow/*
cylc stop recipe_test_workflow/*

#. Run the |RTW| again, as detailed in the :ref:`quick_start_guide`; the
``compare`` task should now succeed.

#. Add the recipe to the documentation:
.. important::
Unless you want the |RTW| to continue running every night, repeat the
previous step to stop the workflow.

* Add a URL for the recipe in ``doc/sphinx/source/utils/RTW/common.txt``
under the ``.. Links`` section in alphabetical order (follow the format
``.. _<name>: <URL>``).
Request a review
----------------

* Add the recipe to the list of :ref:`currently_tested_recipes` in
alphabetical order (follow the format ``* `<name>`_``).
#. Commit and push your changes.

#. Commit and push your changes, create a PR, assign yourself to the PR, and
add the ``Recipe Test Workflow (RTW)`` label to the PR, see
#. Create a PR. Assign yourself to the PR, and add the
``Recipe Test Workflow (RTW)`` label to the PR. Refer to
`ESMValTool PR #3664`_ for an example.

.. note::
Reviewers will automatically be assigned to your PR.

Congratulations!
----------------

The recipe will now run at your site whenever the RTW is run.
14 changes: 5 additions & 9 deletions doc/sphinx/source/utils/RTW/common.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,16 @@

.. Links

.. _Autoassess landsurface soilmoisture: https://docs.esmvaltool.org/en/latest/recipes/recipe_autoassess_landsurface_soilmoisture.html#available-recipes-and-diagnostics
.. _Consecutive dry days: https://docs.esmvaltool.org/en/latest/recipes/recipe_consecdrydays.html
.. _CMIP Documentation: https://www.wcrp-climate.org/wgcm-cmip
.. _Cylc Documentation: https://cylc.github.io/cylc-doc/stable/html/index.html
.. _Cylc ISO8601 Durations: https://cylc.github.io/cylc-doc/stable/html/tutorial/scheduling/datetime-cycling.html#iso8601-durations
.. _Cylc Jinja2: https://cylc.github.io/cylc-doc/stable/html/user-guide/writing-workflows/jinja2.html
.. _Cylc Review: https://cylc.github.io/cylc-doc/7.9.3/html/tutorial.html#viewing-suite-logs-in-a-web-browser-cylc-review
.. _Ensclus: https://docs.esmvaltool.org/en/latest/recipes/recipe_ensclus.html
.. _ESMValTool Documentation: https://esmvaltool.org/
.. _ESMValTool issue #3663: https://github.com/ESMValGroup/ESMValTool/issues/3663
.. _ESMValTool PR #3664: https://github.com/ESMValGroup/ESMValTool/pull/3664
.. _Examples python: https://docs.esmvaltool.org/en/latest/recipes/recipe_examples.html
.. _Heatwaves coldwaves: https://docs.esmvaltool.org/en/latest/recipes/recipe_heatwaves_coldwaves.html
.. _Landcover - Albedo: https://docs.esmvaltool.org/en/latest/recipes/recipe_albedolandcover.html#landcover-albedo
.. _Jinja2: https://jinja.palletsprojects.com/en/stable/
.. _new ESMValTool issue: https://github.com/ESMValGroup/ESMValTool/issues/new
.. _Ocean Atlantic Meridional Overturning Circulation (AMOC): https://docs.esmvaltool.org/en/latest/recipes/recipe_oceans.html#recipe-ocean-amoc-yml
.. _Ocean multimap: https://docs.esmvaltool.org/en/latest/recipes/recipe_oceans.html#recipe-ocean-multimap-yml
.. _Radiation budget: https://docs.esmvaltool.org/en/latest/recipes/recipe_radiation_budget.html
.. _Rose Documentation: https://metomi.github.io/rose/doc/html/index.html
.. _metoffice-recipes.cylc: https://github.com/ESMValGroup/ESMValTool/blob/main/esmvaltool/utils/recipe_test_workflow/site/metoffice-recipes.cylc
.. _Slurm sbatch --mem: https://slurm.schedmd.com/sbatch.html#OPT_mem
19 changes: 9 additions & 10 deletions doc/sphinx/source/utils/RTW/tested_recipes.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,13 @@ Currently tested recipes

.. include:: common.txt

The following currently tested recipes are available in the |RTW|:
The recipes tested with the |RTW| at each site are defined in the following
files:

* `Autoassess landsurface soilmoisture`_
* `Consecutive dry days`_
* `Ensclus`_
* `Examples python`_
* `Heatwaves coldwaves`_
* `Landcover - Albedo`_
* `Ocean Atlantic Meridional Overturning Circulation (AMOC)`_
* `Ocean multimap`_
* `Radiation budget`_
* `metoffice-recipes.cylc`_
* dkrz-recipes.cylc **(COMING SOON!)**
* jasmin-recipes.cylc **(COMING SOON!)**

.. hint::
The :ref:`site/site-recipes.jinja <site_recipes_file>`.
file provides more information.
11 changes: 11 additions & 0 deletions doc/sphinx/source/utils/RTW/user_guide/workflow.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,17 @@ Portability
The |RTW| is portable; site-specific information can be found in the ``site``
and ``opt`` directories within the |RTW|. The files required are:

.. _site_recipes_file:

``site/<site>-recipes.jinja``
Contains all the recipes run at the ``SITE``

.. hint::
* The file uses the `Jinja2`_ templating language, which has a similar syntax
to Python
* Jinja2 gives |Cylc| many powerful features. Refer to `Cylc Jinja2`_ for
more information

``site/<site>.cylc``
Contains task definitions specific to the ``SITE``, for example, ``COMPUTE``

Expand Down
Loading

0 comments on commit 9344628

Please sign in to comment.