Skip to content

Use uv for other workflows #3424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/workflows/code-style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,15 @@ jobs:
steps:
- if: github.event_name == 'push'
uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@v6
with:
python-version: "3.11"
activate-environment: true
enable-cache: true
- name: Check lock
id: check-lock
run: |
uv lock --check
- run: |
bash ./tests/run_code_style.sh install
bash ./tests/run_code_style.sh fmt
Expand Down
42 changes: 27 additions & 15 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,17 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@v6
with:
python-version: "3.10"
activate-environment: true
enable-cache: true

- run: sudo npm install katex -g
- uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ hashFiles('requirements-dev.txt') }}-${{ hashFiles('docs/requirements.txt') }}
# - uses: actions/cache@v4
# with:
# path: ~/.cache/pip
# key: pip-${{ hashFiles('requirements-dev.txt') }}-${{ hashFiles('docs/requirements.txt') }}

- name: Install docs deps
run: bash .github/workflows/install_docs_deps.sh
Expand All @@ -51,14 +53,16 @@ jobs:
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@v6
with:
python-version: "3.10"
activate-environment: true
enable-cache: true

- uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ hashFiles('requirements-dev.txt') }}-${{ hashFiles('docs/requirements.txt') }}
# - uses: actions/cache@v4
# with:
# path: ~/.cache/pip
# key: pip-${{ hashFiles('requirements-dev.txt') }}-${{ hashFiles('docs/requirements.txt') }}

- name: Install docs deps
run: bash .github/workflows/install_docs_deps.sh
Expand All @@ -72,15 +76,23 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5

- uses: astral-sh/setup-uv@v6
with:
python-version: "3.10"
activate-environment: true
enable-cache: true

- name: Check lock
id: check-lock
run: |
uv lock --check

- run: sudo npm install katex -g
- uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ hashFiles('requirements-dev.txt') }}-${{ hashFiles('docs/requirements.txt') }}
# - uses: actions/cache@v4
# with:
# path: ~/.cache/pip
# key: pip-${{ hashFiles('requirements-dev.txt') }}-${{ hashFiles('docs/requirements.txt') }}

- name: Install docs deps
run: bash .github/workflows/install_docs_deps.sh
Expand Down
33 changes: 20 additions & 13 deletions .github/workflows/hvd-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,29 +41,36 @@ jobs:

- name: Get pip cache dir
id: pip-cache
shell: bash -l {0}
run: |
python3 -m pip install -U pip
uv pip install -U pip
echo "pip_cache=$(python3 -m pip cache dir)" >> $GITHUB_OUTPUT
shell: bash -l {0}

- uses: actions/cache@v4
with:
path: |
~/conda_pkgs_dir
${{ steps.pip-cache.outputs.pip_cache }}
key: ${{ steps.get-date.outputs.date }}-horovod-${{ hashFiles('requirements-dev.txt') }}
# - uses: actions/cache@v4
# with:
# path: |
# ~/conda_pkgs_dir
# ${{ steps.pip-cache.outputs.pip_cache }}
# key: ${{ steps.get-date.outputs.date }}-horovod-${{ hashFiles('requirements-dev.txt') }}

- uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@v6
with:
python-version: ${{ matrix.python-version }}
activate-environment: true
enable-cache: true

- name: Check lock
id: check-lock
run: |
uv lock --check

- name: Install dependencies
shell: bash -l {0}
run: |

#install other dependencies
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements-dev.txt
# install other dependencies
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
uv pip install -r requirements-dev.txt

# Install Horovod from source and apply a patch to build with recent pytorch
# We can't use pip install <whatever> as build-env can't find pytorch and
Expand All @@ -78,7 +85,7 @@ jobs:
python -c "import horovod.torch as hvd; hvd.mpi_ops.Sum"

# Install ignite
pip install .
uv pip install .

# Download MNIST: https://github.com/pytorch/ignite/issues/1737
# to "/tmp" for cpu tests
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/install_docs_deps.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# remove pkg-resources as it causes failure when installing https://github.com/pytorch-ignite/sphinxcontrib-versioning
pip uninstall -y pkg-resources setuptools && pip install --upgrade setuptools pip wheel
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu -U
pip install -r requirements-dev.txt
pip install -r docs/requirements.txt
pip install git+https://github.com/pytorch-ignite/sphinxcontrib-versioning.git
uv pip uninstall pkg-resources setuptools && pip install --upgrade setuptools pip wheel
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu -U
uv pip install -r requirements-dev.txt
uv pip install -r docs/requirements.txt
uv pip install git+https://github.com/pytorch-ignite/sphinxcontrib-versioning.git
38 changes: 20 additions & 18 deletions .github/workflows/tpu-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,49 +36,51 @@ jobs:

steps:
- uses: actions/checkout@v4

- name: Set up Python 3.10
uses: actions/setup-python@v5
uses: astral-sh/setup-uv@v6
with:
python-version: "3.10"
architecture: "x64"
activate-environment: true
enable-cache: true

- name: Check lock
id: check-lock
run: |
uv lock --check

- name: Get year & week number
id: get-date
run: echo "date=$(/bin/date "+%Y-%U")" >> $GITHUB_OUTPUT
shell: bash -l {0}
run: echo "date=$(/bin/date "+%Y-%U")" >> $GITHUB_OUTPUT

- name: Get pip cache dir
id: pip-cache
shell: bash -l {0}
run: |
pip3 install -U "pip<24"
uv pip install -U "pip<24"
echo "pip_cache=$(pip cache dir)" >> $GITHUB_OUTPUT
shell: bash -l {0}

- uses: actions/cache@v4
with:
path: |
${{ steps.pip-cache.outputs.pip_cache }}
key: ${{ steps.get-date.outputs.date }}-pytorch-${{ runner.os }}-${{ matrix.xla-version }}-${{ hashFiles('requirements-dev.txt') }}
restore-keys: |
${{ steps.get-date.outputs.date }}-pytorch-${{ runner.os }}-${{ matrix.xla-version }}-

- name: Install Torch XLA and others
run: |

## Install mkl (alternative approach to https://github.com/pytorch/xla/blob/b0ba29f98a695671972d4a4cc07441014dba2892/.kokoro/common.sh#L31-L32)
sudo apt-get update && sudo apt-get install -y libopenblas-dev libomp5
pip install mkl==2021.4.0
uv pip install mkl==2021.4.0

## Install torch & xla and torchvision
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu
pip install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-nightly-cp310-cp310-linux_x86_64.whl
uv pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu
## TODO: download file and rename as valid file with version
curl -o torch_xla-2.5.0+gitf0bbaeb-cp310-cp310-linux_x86_64.whl https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-nightly-cp310-cp310-linux_x86_64.whl
uv pip install torch_xla-2.5.0+gitf0bbaeb-cp310-cp310-linux_x86_64.whl
# uv pip install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-nightly-cp310-cp310-linux_x86_64.whl

# Check installation
python -c "import torch"

## Install test deps and Ignite
pip install -r requirements-dev.txt
pip install .
uv pip install -r requirements-dev.txt
uv pip install .

# Download MNIST: https://github.com/pytorch/ignite/issues/1737
# to "/tmp" for tpu tests
Expand Down
18 changes: 10 additions & 8 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,11 @@ jobs:

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@v6
with:
python-version: ${{ matrix.python-version }}
activate-environment: true
enable-cache: true

- name: Get year & week number
id: get-date
Expand All @@ -82,21 +84,21 @@ jobs:
restore-keys: |
${{ steps.get-date.outputs.date }}-pytorch-${{ runner.os }}-${{ matrix.python-version }}-${{ matrix.pytorch-channel }}-

- run: pip install pip wheel setuptools -Uqq
- run: uv pip install pip wheel setuptools -Uqq

- name: Install PyTorch
if: ${{ matrix.pytorch-channel == 'pytorch' }}
run: pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
run: uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

- name: Install PyTorch (nightly)
if: ${{ matrix.pytorch-channel == 'pytorch-nightly' }}
run: pip install torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu --pre
run: uv pip install torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu --pre

- name: Install dependencies
run: |
pip install -r requirements-dev.txt
pip install .
pip list
uv pip install -r requirements-dev.txt
uv pip install .
uv pip list

- name: Check code formatting
run: |
Expand Down Expand Up @@ -156,7 +158,7 @@ jobs:
# 3.1) mnist_with_tensorboard.py with tbX
python examples/mnist/mnist_with_tensorboard.py --epochs=1
# 3.2) mnist_with_tensorboard.py with native torch tb
pip uninstall -y tensorboardX
uv pip uninstall tensorboardX
python examples/mnist/mnist_with_tensorboard.py --epochs=1

- name: Run MNIST Example With Crash
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
sphinx<6
-e git+https://github.com/pytorch/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
git+https://github.com/pytorch/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
sphinxcontrib-katex
sphinx-copybutton==0.4.0
docutils<0.18
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ license-files = ["LICENSE"]
classifiers = [
"Programming Language :: Python :: 3",
]
requires-python = ">=3.9,<3.13"
dependencies = [
"torch>=1.3,<3",
"packaging"
Expand Down
10 changes: 5 additions & 5 deletions tests/run_code_style.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
set -xeu

if [ $1 = "lint" ]; then
flake8 ignite tests examples --config setup.cfg
ufmt diff .
uv run flake8 ignite tests examples --config setup.cfg
uv run ufmt diff .
elif [ $1 = "fmt" ]; then
ufmt format .
uv run ufmt format .
elif [ $1 = "mypy" ]; then
mypy --config-file mypy.ini
uv run mypy --config-file mypy.ini
elif [ $1 = "install" ]; then
pip install --upgrade flake8 "black==24.10.0" "usort==1.0.8.post1" "ufmt==2.7.3" "mypy"
uv pip install --upgrade flake8 "black==24.10.0" "usort==1.0.8.post1" "ufmt==2.7.3" "mypy"
fi
Loading