Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 6 additions & 46 deletions .github/workflows/durabletask-azurefunctions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,11 @@ jobs:
with:
python-version: 3.13
- name: Install dependencies
working-directory: azure-functions-durable
run: |
python -m pip install --upgrade pip
pip install setuptools wheel tox
pip install flake8
- name: Run flake8 Linter
working-directory: azure-functions-durable
run: flake8 .
- name: Run flake8 Linter
working-directory: tests/azure-functions-durable
run: flake8 .
pip install nox
- name: Run lint checks
run: nox -s lint

run-tests:
strategy:
Expand All @@ -51,35 +45,13 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Install durabletask dependencies
- name: Install Nox
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
pip install -r requirements.txt

- name: Install durabletask locally
run: |
pip install . --no-deps --force-reinstall

- name: Install azure-functions-durable locally
working-directory: azure-functions-durable
run: |
pip install . --no-deps --force-reinstall

# ``requirements.txt`` (shared with the core 3.10-3.14 CI) leaves
# azure-functions unpinned so it can resolve on Python < 3.13. The
# provider needs azure-functions>=2.3.0b2 (which requires 3.13+ and
# provides ``register_converter``); the local installs above use
# ``--no-deps``, so install that floor explicitly here. This job only runs
# on 3.13/3.14, where 2.3.0b2 is available.
- name: Install azure-functions (>=2.3.0b2)
run: |
pip install "azure-functions>=2.3.0b2"
pip install nox

- name: Run unit tests
working-directory: tests/azure-functions-durable
run: |
pytest -m "not dts and not azurite and not functions_e2e" --verbose
run: nox -s functions_unit-${{ matrix.python-version }}

e2e-tests:
needs: run-tests
Expand Down Expand Up @@ -115,17 +87,6 @@ jobs:
npm install -g azurite@latest
npm install -g azure-functions-core-tools@4 --unsafe-perm true

- name: Start Azurite
shell: bash
# The DurableTask.AzureStorage provider in recent extension bundles uses
# a newer Azure Storage REST API version than the CI Azurite build
# recognizes, which otherwise fails task-hub creation with "The API
# version ... is not supported by Azurite". ``--skipApiVersionCheck``
# bypasses that guard so the durable provider can talk to Azurite.
run: |
azurite --silent --skipApiVersionCheck --location /tmp/azurite --blobPort 10000 --queuePort 10001 --tablePort 10002 &
sleep 2

- name: Install nox
run: |
python -m pip install --upgrade pip
Expand All @@ -151,4 +112,3 @@ jobs:
path: tests/azure-functions-durable/e2e/apps/*/_func_host.log
if-no-files-found: warn
retention-days: 7

56 changes: 10 additions & 46 deletions .github/workflows/durabletask-azuremanaged.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,68 +23,32 @@ jobs:
with:
python-version: 3.14
- name: Install dependencies
working-directory: durabletask-azuremanaged
run: |
python -m pip install --upgrade pip
pip install setuptools wheel tox
pip install flake8
- name: Run flake8 Linter
working-directory: durabletask-azuremanaged
run: flake8 .
- name: Run flake8 Linter
working-directory: tests/durabletask-azuremanaged
run: flake8 .
pip install nox
- name: Run lint checks
run: nox -s lint

run-docker-tests:
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
env:
EMULATOR_VERSION: "latest"
needs: lint
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Pull Docker image
run: docker pull mcr.microsoft.com/dts/dts-emulator:$EMULATOR_VERSION
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Run Docker container
run: |
docker run --name dtsemulator -d -p 8080:8080 mcr.microsoft.com/dts/dts-emulator:$EMULATOR_VERSION

- name: Wait for container to be ready
run: sleep 10 # Adjust if your service needs more time to start

- name: Set environment variables
run: |
echo "TASKHUB=default" >> $GITHUB_ENV
echo "ENDPOINT=http://localhost:8080" >> $GITHUB_ENV

- name: Install durabletask dependencies
- name: Install Nox
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
pip install -r requirements.txt

- name: Install durabletask-azuremanaged dependencies
working-directory: examples
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Install durabletask-azuremanaged locally
working-directory: durabletask-azuremanaged
run: |
pip install . --no-deps --force-reinstall

- name: Install durabletask locally
run: |
pip install . --no-deps --force-reinstall
pip install nox

- name: Run the tests
working-directory: tests/durabletask-azuremanaged
run: |
pytest -m "dts" --verbose
run: nox -s azuremanaged_tests-${{ matrix.python-version }}
31 changes: 6 additions & 25 deletions .github/workflows/durabletask.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,9 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel tox
pip install flake8
- name: Run flake8 Linter
working-directory: durabletask
run: flake8 .
- name: "Run flake8 linter: tests"
working-directory: tests/durabletask
run: flake8 .
- name: "Run flake8 linter: examples"
working-directory: examples
run: flake8 .
pip install nox
- name: Run lint checks
run: nox -s lint

run-tests:
strategy:
Expand Down Expand Up @@ -67,21 +59,10 @@ jobs:
- name: Install Azurite
run: npm install -g azurite

- name: Start Azurite
shell: bash
run: |
azurite-blob --silent --blobPort 10000 &
sleep 2

- name: Install durabletask dependencies and the library itself
- name: Install Nox
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
pip install -r requirements.txt
pip install ".[azure-blob-payloads]"
pip install aiohttp
pip install nox

- name: Pytest unit tests
working-directory: tests/durabletask
run: |
pytest -m "not dts" --verbose
run: nox -s core_tests-${{ matrix.python-version }}
24 changes: 4 additions & 20 deletions .github/workflows/typecheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,23 +30,10 @@ jobs:
- name: Install packages and dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
# Install third-party dependencies declared by the examples so they
# type-check cleanly. Each example's requirements.txt is the single
# source of truth for its dependencies.
for req in examples/requirements.txt examples/*/requirements.txt; do
pip install -r "$req"
done
# Install the packages under test from local source last (non-editable
# so the durabletask / durabletask.azuremanaged namespace packages are
# physically merged in site-packages, which pyright can resolve). This
# overrides any PyPI copy pulled in by the example requirements above.
pip install --force-reinstall --no-deps . ./durabletask-azuremanaged
pip install ".[azure-blob-payloads,opentelemetry]"
pip install pyright
pip install nox

- name: Run pyright (strict, Python 3.10)
run: pyright
run: nox -s typecheck_core

pyright-azurefunctions:
runs-on: ubuntu-latest
Expand All @@ -62,10 +49,7 @@ jobs:
- name: Install packages and dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -e ".[azure-blob-payloads,opentelemetry]"
pip install -e ./azure-functions-durable
pip install pyright
pip install nox

- name: Run pyright (strict, Python 3.13)
run: pyright -p azure-functions-durable/pyrightconfig.json
run: nox -s typecheck_functions
75 changes: 64 additions & 11 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,72 @@
# Development

The following is more information about how to develop this project. Note that development commands
require that `make` is installed on your local machine. If you're using Windows, you can install
`make` using [Chocolatey](https://chocolatey.org/) or use WSL.
The following is more information about how to develop this project.

## Validation

Use Nox to run the representative local correctness checks:

```sh
python -m pip install -r dev-requirements.txt
nox -s ci
```

The `ci` session runs linting, strict type checks, core SDK and Azure Managed
tests on Python 3.10, Azure Functions unit tests on Python 3.13, and Azure
Functions end-to-end tests on Python 3.13. It intentionally runs one
representative version rather than the complete CI matrix, keeping routine
local validation fast. To use another supported version (3.10--3.14) for the
core SDK and Azure Managed tests, pass it after `--`:

```sh
nox -s ci -- 3.14
```

Run the versioned test sessions directly when a change needs complete matrix
coverage, for example `nox -s core_tests` or `nox -s azuremanaged_tests`.
Nox fails rather than silently skipping a session when its required Python
interpreter is unavailable.

Nox starts Azurite automatically for the core and Azure Functions tests. The
Azure Managed tests start a disposable DTS emulator Docker container. Start
Docker before running that session. Azure Functions end-to-end tests also
require the Azure Functions Core Tools (`func`) on `PATH`. Azure Managed Nox
runs use a disposable DTS emulator with an automatically assigned port, and
Functions E2E uses a unique Durable Functions hub. Both can run alongside other
local validation sessions.

CodeQL remains a GitHub-hosted security scan; it requires the CodeQL CLI and
its query packs, so it is not included in the local `ci` session.

For iterative validation, use `-R` to reuse the session virtual environment.
The local packages are editable in test sessions, so source changes are picked
up without reinstalling. Pass file or directory paths after `--` to lint and
type-check sessions. Pytest-based sessions also accept test node IDs and pytest
selectors such as `-k`. Nox forwards these arguments unchanged to the
underlying tool, and multiple paths or selectors are supported.

For a focused change, run just the relevant session:

```sh
nox -R -s lint -- durabletask/client.py tests/durabletask/test_client.py
nox -R -s typecheck_core -- durabletask/client.py durabletask/extensions/history_export
nox -R -s typecheck_functions -- azure-functions-durable/azure
nox -R -s core_tests-3.10 -- tests/durabletask/test_client.py::test_get_grpc_channel_insecure
nox -R -s azuremanaged_tests-3.10 -- tests/durabletask-azuremanaged/test_dts_orchestration_e2e.py
nox -R -s functions_unit-3.13 -- tests/azure-functions-durable/test_client_compat.py
nox -R -s functions_e2e -- -k "dtask_client"
```

> [!NOTE]
> The Azure Managed session still starts a fresh isolated DTS emulator for every
> run, and Functions E2E starts a fresh Functions host. `-R` skips environment
> provisioning, not these required runtime services.

## Generating protobufs

These commands require `make`. If you're using Windows, you can install `make`
using [Chocolatey](https://chocolatey.org/) or use WSL.

```sh
pip3 install -r dev-requirements.txt
make gen-proto
Expand All @@ -14,11 +75,3 @@ make gen-proto
This will download the `orchestrator_service.proto` from the `microsoft/durabletask-protobuf` repo
and compile it using `grpcio-tools`. The version of the source proto file that was downloaded can be
found in the file `durabletask/internal/PROTO_SOURCE_COMMIT_HASH`.

### Running tests

Tests can be run using the following command from the project root.

```sh
make test
```
Loading
Loading