-
Notifications
You must be signed in to change notification settings - Fork 129
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge remote-tracking branch 'public/main' into rapid_evaluation_fram…
…ework_tcre
- Loading branch information
Showing
2,113 changed files
with
255,438 additions
and
145,013 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
^\.circleci/ | ||
^environment\.yml$ | ||
^esmvaltool/install/ | ||
^pyproject.toml$ | ||
^setup\.py$ | ||
^setup\.cfg$ | ||
^MANIFEST.in$ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
|
||
**/__pycache__ | ||
.* | ||
doc | ||
tests | ||
ESMValTool.egg-info | ||
|
||
!.git | ||
!.zenodo.json | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
esmvaltool/cmorizers @ESMValGroup/obs-maintainers | ||
.github/workflows @valeriupredoi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
<!-- | ||
Thank you for contributing to our project! | ||
Please do not delete this text completely, but read the text below and keep | ||
items that seem relevant. If in doubt, just keep everything and add your | ||
own text at the top, a reviewer will update the checklist for you. | ||
While the checklist is intended to be filled in by the technical and scientific | ||
reviewers, it is the responsibility of the author of the pull request to make | ||
sure all items on it are properly implemented. | ||
--> | ||
|
||
## Description | ||
|
||
<!-- | ||
Please describe your changes here, especially focusing on why this pull request makes | ||
ESMValTool better and what problem it solves. | ||
Before you start, please read our contribution guidelines: https://docs.esmvaltool.org/en/latest/community/ | ||
Please fill in the GitHub issue that is closed by this pull request, e.g. Closes #1903 | ||
--> | ||
- Closes #issue_number | ||
- Link to documentation: | ||
|
||
* * * | ||
|
||
## Before you get started | ||
|
||
<!-- | ||
Please discuss your idea with the development team before getting started, | ||
to avoid disappointment or unnecessary work later. The way to do this is | ||
to open a new issue on GitHub. | ||
--> | ||
|
||
- [ ] [☝ Create an issue](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#contributing-code-and-documentation) to discuss what you are going to do | ||
|
||
## Checklist | ||
|
||
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the [🛠 Technical][1] or [🧪 Scientific][2] review. | ||
|
||
<!-- The next two lines turn the 🛠 and 🧪 below into hyperlinks --> | ||
[1]: https://docs.esmvaltool.org/en/latest/community/review.html#technical-review | ||
[2]: https://docs.esmvaltool.org/en/latest/community/review.html#scientific-review | ||
|
||
- [ ] [🛠][1] This pull request has a [descriptive title](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#pull-request-title) | ||
- [ ] [🛠][1] Code is written according to the [code quality guidelines](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#code-quality) | ||
- [ ] [🛠][1] [Documentation](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#documentation) is available | ||
- [ ] [🛠][1] [Tests](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#tests) run successfully | ||
- [ ] [🛠][1] The [list of authors](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#list-of-authors) is up to date | ||
- [ ] [🛠][1] Any changed dependencies have been [added or removed correctly](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#dependencies) | ||
- [ ] [🛠][1] All [checks below this pull request](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#pull-request-checks) were successful | ||
|
||
### [New or updated recipe/diagnostic](https://docs.esmvaltool.org/en/latest/community/diagnostic.html) | ||
|
||
- [ ] [🧪][2] [Recipe runs successfully](https://docs.esmvaltool.org/en/latest/community/diagnostic.html#testing-recipes) | ||
- [ ] [🧪][2] [Recipe is well documented](https://docs.esmvaltool.org/en/latest/community/diagnostic.html#recipe-and-diagnostic-documentation) | ||
- [ ] [🧪][2] [Figure(s) and data](https://docs.esmvaltool.org/en/latest/community/diagnostic.html#diagnostic-output) look as expected from literature | ||
- [ ] [🛠][1] [Provenance information](https://docs.esmvaltool.org/en/latest/community/diagnostic.html#recording-provenance) has been added | ||
|
||
### [New or updated data reformatting script](https://docs.esmvaltool.org/en/latest/develop/dataset.html) | ||
|
||
- [ ] [🛠][1] [Documentation](https://docs.esmvaltool.org/en/latest/community/dataset.html#dataset-documentation) is available | ||
- [ ] [🛠][1] The dataset has been [added to the CMOR check recipe](https://docs.esmvaltool.org/en/latest/community/dataset.html#testing) | ||
- [ ] [🛠][1] The dataset has been added to the shared [data pools](https://docs.esmvaltool.org/en/latest/community/dataset.html#cmorized-data) of DKRZ and Jasmin by the @ESMValGroup/OBS-maintainers team | ||
- [ ] [🧪][2] Numbers and units of the data look [physically meaningful](https://docs.esmvaltool.org/en/latest/community/dataset.html#scientific-sanity-check) | ||
|
||
*** | ||
|
||
To help with the number of pull requests: | ||
|
||
- 🙏 We kindly ask you to [review](https://docs.esmvaltool.org/en/latest/community/review.html#review-of-pull-requests) two other [open pull requests](https://github.com/ESMValGroup/ESMValTool/pulls) in this repository | ||
|
||
<!-- | ||
If you need help with any of the items on the checklists above, please do not hesitate to ask by commenting in the issue or pull request. | ||
--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# workflow that performs a validity check of CITATION.cff file | ||
# authors: Abel S. Siqueira, Faruk Diblen, Jurriaan Spaaks GH: @abelsiqueira, @fdiblen, @jspaaks | ||
|
||
name: CFF File Validator | ||
|
||
on: | ||
push: | ||
paths: | ||
- CITATION.cff | ||
schedule: | ||
- cron: '0 0 1 * *' | ||
|
||
jobs: | ||
validate: | ||
name: "validate" | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Check out a copy of the repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Check whether the citation metadata from CITATION.cff is valid | ||
uses: citation-file-format/[email protected] | ||
with: | ||
args: "--validate" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
name: Conda lock file creation | ||
|
||
on: | ||
# Trigger on push on main or other branch for testing | ||
# NOTE that push: main will create the file very often | ||
# and hence lots of automated PRs | ||
# push: | ||
# branches: | ||
# - main | ||
schedule: | ||
- cron: '0 4 */10 * *' | ||
|
||
# Required shell entrypoint to have properly configured bash shell | ||
defaults: | ||
run: | ||
shell: bash -l {0} | ||
|
||
jobs: | ||
create-lock-file: | ||
name: Create conda lock file for latest Python | ||
runs-on: 'ubuntu-latest' | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
- uses: conda-incubator/setup-miniconda@v3 | ||
with: | ||
auto-update-conda: true | ||
activate-environment: esmvaltool-fromlock | ||
python-version: "3.12" | ||
miniforge-version: "latest" | ||
use-mamba: true | ||
- name: Show conda config | ||
run: | | ||
conda update -n base -c conda-forge conda | ||
conda --version | ||
# setup-miniconda@v3 installs an old conda and mamba | ||
# forcing a modern mamba updates both mamba and conda | ||
# unpin mamba after conda-lock=3 release | ||
# see github.com/ESMValGroup/ESMValTool/issues/3782 | ||
conda install -c conda-forge "mamba>=1.4.8,<2" | ||
conda config --show-sources | ||
conda config --show | ||
conda --version | ||
mamba --version | ||
- name: Python info | ||
run: | | ||
which python | ||
python --version | ||
- name: Install conda-lock | ||
run: mamba install -y -c conda-forge conda-lock | ||
- name: Check version of conda-lock | ||
run: conda-lock --version | ||
- name: Create conda lock file for linux-64 | ||
run: conda-lock lock --platform linux-64 -f environment.yml --mamba --kind explicit | ||
- name: Show conda version again | ||
run: conda --version | ||
- name: Show Python exec and version again | ||
run: | | ||
which python | ||
python -V | ||
- name: Create conda-lock environment | ||
run: conda create --name esmvaltool-fromlock --file conda-linux-64.lock | ||
- name: Install pip | ||
run: mamba install -y pip | ||
- name: Check Python and pip versions post pip-install | ||
run: | | ||
which python | ||
pip --version | ||
- name: Install ESMValTool | ||
run: pip install -e .[develop] | ||
- name: Check ESMValTool version | ||
run: esmvaltool --help | ||
- name: Check ESMValTool help | ||
run: esmvaltool version | ||
- name: Run flake8 | ||
run: flake8 | ||
- name: Run tests | ||
run: pytest -n 2 -m "not installation" | ||
# Automated PR | ||
# see https://github.com/marketplace/actions/create-pull-request | ||
- name: Create Automated PR if conda lock file has changed | ||
uses: peter-evans/create-pull-request@v3 | ||
with: | ||
token: ${{ secrets.GITHUB_TOKEN }} | ||
commit-message: Updating Linux condalock file | ||
author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com> | ||
signoff: false | ||
branch: condalock-update | ||
delete-branch: true | ||
title: '[Condalock] Update Linux condalock file' | ||
body: | | ||
Update condalock file | ||
Automatic Pull Request. | ||
labels: | | ||
testing | ||
condalock | ||
automatedPR | ||
assignees: valeriupredoi | ||
reviewers: valeriupredoi | ||
draft: false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
--- | ||
name: Install from Conda | ||
|
||
# runs on a push on main and at the end of every day | ||
on: | ||
push: | ||
branches: | ||
- main | ||
schedule: | ||
- cron: '0 4 * * *' | ||
|
||
# Required shell entrypoint to have properly configured bash shell | ||
defaults: | ||
run: | ||
shell: bash -l {0} | ||
|
||
jobs: | ||
linux: | ||
runs-on: "ubuntu-latest" | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ["3.10", "3.11", "3.12"] | ||
name: Linux Python ${{ matrix.python-version }} | ||
steps: | ||
- uses: conda-incubator/setup-miniconda@v3 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
miniforge-version: "latest" | ||
use-mamba: true | ||
- run: mkdir -p conda_install_linux_artifacts_python_${{ matrix.python-version }} | ||
- name: Record versions | ||
run: | | ||
mamba --version 2>&1 | tee conda_install_linux_artifacts_python_${{ matrix.python-version }}/conda_version.txt | ||
which conda 2>&1 | tee conda_install_linux_artifacts_python_${{ matrix.python-version }}/conda_path.txt | ||
which mamba 2>&1 | tee -a conda_install_linux_artifacts_python_${{ matrix.python-version }}/conda_path.txt | ||
python -V 2>&1 | tee conda_install_linux_artifacts_python_${{ matrix.python-version }}/python_version.txt | ||
- name: Install ESMValTool | ||
run: mamba install esmvaltool 2>&1 | tee conda_install_linux_artifacts_python_${{ matrix.python-version }}/install.txt | ||
- name: Verify installation | ||
run: | | ||
esmvaltool --help | ||
esmvaltool version 2>&1 | tee conda_install_linux_artifacts_python_${{ matrix.python-version }}/version.txt | ||
- name: Upload artifacts | ||
if: ${{ always() }} # upload artifacts even if fail | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: Conda_Install_Linux_python_${{ matrix.python-version }} | ||
path: conda_install_linux_artifacts_python_${{ matrix.python-version }} | ||
|
||
# uncomment from here when we have a testing environment on an OSX machine | ||
# and we know that this should work | ||
# | ||
# osx: | ||
# runs-on: "macos-latest" | ||
# strategy: | ||
# matrix: | ||
# python-version: ["3.10", "3.11"] | ||
# fail-fast: false | ||
# name: OSX Python ${{ matrix.python-version }} | ||
# steps: | ||
# - uses: actions/checkout@v2 | ||
# - uses: conda-incubator/setup-miniconda@v3 | ||
# with: | ||
# python-version: ${{ matrix.python-version }} | ||
# miniconda-version: "latest" | ||
# channels: conda-forge | ||
# - run: mkdir -p conda_install_osx_artifacts_python_${{ matrix.python-version }} | ||
# - run: conda --version 2>&1 | tee conda_install_osx_artifacts_python_${{ matrix.python-version }}/conda_version.txt | ||
# - run: which conda 2>&1 | tee conda_install_osx_artifacts_python_${{ matrix.python-version }}/conda_path.txt | ||
# - run: python -V 2>&1 | tee conda_install_osx_artifacts_python_${{ matrix.python-version }}/python_version.txt | ||
# # ncurses needs to be from conda-forge and not main channel | ||
# # for now it's turned off since we're not testing R/Julia installs | ||
# # - run: conda uninstall -y ncurses | ||
# # - run: conda list ncurses | ||
# # - run: conda install -y conda-forge::ncurses | ||
# # - run: conda list ncurses | ||
# - run: conda install esmvaltool --no-update-deps 2>&1 | tee conda_install_osx_artifacts_python_${{ matrix.python-version }}/install.txt | ||
# - run: conda install esmvaltool-python esmvaltool-ncl 2>&1 | tee conda_install_osx_artifacts_python_${{ matrix.python-version }}/install.txt | ||
# - run: esmvaltool --help | ||
# - run: esmvaltool version 2>&1 | tee conda_install_osx_artifacts_python_${{ matrix.python-version }}/version.txt | ||
# - name: Upload artifacts | ||
# if: ${{ always() }} # upload artifacts even if fail | ||
# uses: actions/upload-artifact@v4 | ||
# with: | ||
# name: Conda_Install_OSX_python_${{ matrix.python-version }} | ||
# path: conda_install_osx_artifacts_python_${{ matrix.python-version }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Install esmvaltool from a conda lock file | ||
# To build a conda lock file install conda-lock first then | ||
# run conda-lock lock --platform linux-64 -f environment.yml --mamba | ||
# (mamba activated for speed). Change platform for osx-64 or win-64. | ||
# Env creation then happens as per normal use with | ||
# conda create --name esmvaltool-fromlock --file conda-linux-64.lock | ||
# note that pip and conda are NOT installed. | ||
|
||
name: Conda-lock Install | ||
on: | ||
push: | ||
branches: | ||
- main | ||
# - condalock-update | ||
# run the test only if the PR is to main | ||
# turn it on if required | ||
#pull_request: | ||
# branches: | ||
# - main | ||
schedule: | ||
- cron: '0 0 * * *' | ||
|
||
# Required shell entrypoint to have properly configured bash shell | ||
defaults: | ||
run: | ||
shell: bash -l {0} | ||
|
||
jobs: | ||
linux: | ||
runs-on: "ubuntu-latest" | ||
strategy: | ||
matrix: | ||
python-version: ["3.10", "3.11", "3.12"] | ||
fail-fast: false | ||
name: Linux Python ${{ matrix.python-version }} | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
- uses: conda-incubator/setup-miniconda@v3 | ||
with: | ||
activate-environment: esmvaltool-fromlock | ||
python-version: ${{ matrix.python-version }} | ||
miniconda-version: "latest" | ||
channels: conda-forge | ||
- run: mkdir -p source_install_linux_artifacts_python_${{ matrix.python-version }} | ||
- run: conda --version 2>&1 | tee source_install_linux_artifacts_python_${{ matrix.python-version }}/conda_version.txt | ||
- run: which python | ||
- run: python -V 2>&1 | tee source_install_linux_artifacts_python_${{ matrix.python-version }}/python_version.txt | ||
- run: conda create --name esmvaltool-fromlock --file conda-linux-64.lock | ||
- run: which python | ||
- run: pip --version | ||
- run: pip install -e .[develop] | ||
- run: esmvaltool --help | ||
- run: esmvaltool version 2>&1 | tee source_install_linux_artifacts_python_${{ matrix.python-version }}/version.txt | ||
- run: flake8 | ||
- run: pytest -n 2 -m "not installation" | ||
- name: Upload artifacts | ||
if: ${{ always() }} # upload artifacts even if fail | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: Source_Install_Linux_python_${{ matrix.python-version }} | ||
path: source_install_linux_artifacts_python_${{ matrix.python-version }} |
Oops, something went wrong.