Skip to content

WIP: Test CI speedups #321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+23 −4
Open

WIP: Test CI speedups #321

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5597ec7
CI: use ARRAY_API_TESTS_XFAIL_MARK on CI
ev-br May 11, 2025
c5e7613
WIP: checkout array_api_tests from a fork/branch
ev-br May 11, 2025
46e9711
CI: run 500 examples on NumPy and PyTorch; 50 on Dask
ev-br May 11, 2025
232e703
CI: use 4 workers; bump the # of examples to 1000 (np/torch), 200 (dask)
ev-br May 12, 2025
ff0e22a
MAINT: update numpy-1.22 xfails
ev-br May 12, 2025
12f5ff2
MAINT: remove --ci pytest switch
ev-br May 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/array-api-tests-dask.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,7 @@ jobs:
# workflow is barely more than a smoke test, and one should expect extreme
# flakiness. Before changes to dask-xfails.txt or dask-skips.txt, please run
# the full test suite with at least 200 examples.
pytest-extra-args: --max-examples=5
pytest-extra-args: --max-examples=200 -n 4
python-versions: '[''3.10'', ''3.13'']'
extra-env-vars: |
ARRAY_API_TESTS_XFAIL_MARK=skip
3 changes: 3 additions & 0 deletions .github/workflows/array-api-tests-numpy-1-22.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ jobs:
package-version: '== 1.22.*'
xfails-file-extra: '-1-22'
python-versions: '[''3.10'']'
pytest-extra-args: -n 4
extra-env-vars: |
ARRAY_API_TESTS_XFAIL_MARK=skip
3 changes: 3 additions & 0 deletions .github/workflows/array-api-tests-numpy-1-26.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ jobs:
package-version: '== 1.26.*'
xfails-file-extra: '-1-26'
python-versions: '[''3.10'', ''3.12'']'
pytest-extra-args: -n 4
extra-env-vars: |
ARRAY_API_TESTS_XFAIL_MARK=skip
3 changes: 3 additions & 0 deletions .github/workflows/array-api-tests-numpy-dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ jobs:
extra-requires: '--pre --extra-index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple'
xfails-file-extra: '-dev'
python-versions: '[''3.11'', ''3.13'']'
pytest-extra-args: -n 4
extra-env-vars: |
ARRAY_API_TESTS_XFAIL_MARK=skip
3 changes: 3 additions & 0 deletions .github/workflows/array-api-tests-numpy-latest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ jobs:
with:
package-name: numpy
python-versions: '[''3.10'', ''3.13'']'
pytest-extra-args: -n 4
extra-env-vars: |
ARRAY_API_TESTS_XFAIL_MARK=skip
2 changes: 2 additions & 0 deletions .github/workflows/array-api-tests-torch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,6 @@ jobs:
extra-requires: '--index-url https://download.pytorch.org/whl/cpu'
extra-env-vars: |
ARRAY_API_TESTS_SKIP_DTYPES=uint16,uint32,uint64
ARRAY_API_TESTS_XFAIL_MARK=skip
python-versions: '[''3.10'', ''3.13'']'
pytest-extra-args: -n 4
6 changes: 4 additions & 2 deletions .github/workflows/array-api-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ on:
description: Multiline string of environment variables to set for the test run.

env:
PYTEST_ARGS: "--max-examples 200 -v -rxXfE --ci ${{ inputs.pytest-extra-args }} --hypothesis-disable-deadline --durations 10"
PYTEST_ARGS: "--max-examples 1000 -v -rxXfE ${{ inputs.pytest-extra-args }} --hypothesis-disable-deadline --durations 20"

jobs:
tests:
Expand All @@ -56,7 +56,8 @@ jobs:
- name: Checkout array-api-tests
uses: actions/checkout@v4
with:
repository: data-apis/array-api-tests
repository: ev-br/array-api-tests
ref: skip_xfails
submodules: 'true'
path: array-api-tests

Expand All @@ -76,6 +77,7 @@ jobs:
python -m pip install --upgrade pip
python -m pip install '${{ inputs.package-name }} ${{ inputs.package-version }}' ${{ inputs.extra-requires }}
python -m pip install -r ${GITHUB_WORKSPACE}/array-api-tests/requirements.txt
python -m pip install pytest-xdist
- name: Dump pip environment
run: pip freeze
Expand Down
3 changes: 2 additions & 1 deletion numpy-1-22-xfails.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,8 @@ array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_sc
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[maximum]
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[copysign]
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[pow]
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[pow]
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[minimum]
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[atan2]

array_api_tests/test_operators_and_elementwise_functions.py::test_where_with_scalars

Expand Down
Loading