Skip to content

Commit 1709591

Browse files
author
Bernd Verst
committed
Merge remote-tracking branch 'origin/main' into berndverst-entity-method-signature-cache
2 parents 7567860 + 0f316cc commit 1709591

112 files changed

Lines changed: 10979 additions & 62 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.flake8

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,7 @@ ignore = E501,C901,W503
33
exclude =
44
.git
55
*_pb2*
6-
__pycache__
6+
__pycache__
7+
.venv
8+
.nox
9+
.python_packages
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
name: Durable Task Scheduler SDK (azure-functions-durable)
2+
3+
on:
4+
push:
5+
branches:
6+
- "main"
7+
tags:
8+
- "azurefunctions-v*" # Only run for tags starting with "azurefunctions-v"
9+
pull_request:
10+
branches:
11+
- "main"
12+
13+
permissions:
14+
contents: read
15+
16+
jobs:
17+
lint:
18+
runs-on: ubuntu-latest
19+
steps:
20+
- uses: actions/checkout@v4
21+
- name: Set up Python 3.13
22+
uses: actions/setup-python@v5
23+
with:
24+
python-version: 3.13
25+
- name: Install dependencies
26+
working-directory: azure-functions-durable
27+
run: |
28+
python -m pip install --upgrade pip
29+
pip install setuptools wheel tox
30+
pip install flake8
31+
- name: Run flake8 Linter
32+
working-directory: azure-functions-durable
33+
run: flake8 .
34+
- name: Run flake8 Linter
35+
working-directory: tests/azure-functions-durable
36+
run: flake8 .
37+
38+
run-tests:
39+
strategy:
40+
fail-fast: false
41+
matrix:
42+
python-version: ["3.13", "3.14"]
43+
needs: lint
44+
runs-on: ubuntu-latest
45+
steps:
46+
- name: Checkout repository
47+
uses: actions/checkout@v4
48+
49+
- name: Set up Python ${{ matrix.python-version }}
50+
uses: actions/setup-python@v5
51+
with:
52+
python-version: ${{ matrix.python-version }}
53+
54+
- name: Install durabletask dependencies
55+
run: |
56+
python -m pip install --upgrade pip
57+
pip install flake8 pytest
58+
pip install -r requirements.txt
59+
60+
- name: Install durabletask locally
61+
run: |
62+
pip install . --no-deps --force-reinstall
63+
64+
- name: Install azure-functions-durable locally
65+
working-directory: azure-functions-durable
66+
run: |
67+
pip install . --no-deps --force-reinstall
68+
69+
# ``requirements.txt`` (shared with the core 3.10-3.14 CI) leaves
70+
# azure-functions unpinned so it can resolve on Python < 3.13. The
71+
# provider needs azure-functions>=2.3.0b2 (which requires 3.13+ and
72+
# provides ``register_converter``); the local installs above use
73+
# ``--no-deps``, so install that floor explicitly here. This job only runs
74+
# on 3.13/3.14, where 2.3.0b2 is available.
75+
- name: Install azure-functions (>=2.3.0b2)
76+
run: |
77+
pip install "azure-functions>=2.3.0b2"
78+
79+
- name: Run unit tests
80+
working-directory: tests/azure-functions-durable
81+
run: |
82+
pytest -m "not dts and not azurite and not functions_e2e" --verbose
83+
84+
e2e-tests:
85+
needs: run-tests
86+
runs-on: ubuntu-latest
87+
# The end-to-end suite launches a real Azure Functions host, which requires
88+
# ``azure-functions>=2.3.0b2``. That build is published to PyPI and pulled
89+
# in as a declared dependency of ``azure-functions-durable``.
90+
# Run the suite in both payload-typing modes: the default (loose) codec and
91+
# strict typing (``AZURE_FUNCTIONS_DURABLE_STRICT_TYPING=1``), which requires
92+
# every decode call site to supply the destination type. ``0`` is not a
93+
# truthy value for the flag, so it selects loose mode.
94+
strategy:
95+
fail-fast: false
96+
matrix:
97+
strict-typing: ["0", "1"]
98+
name: e2e-tests (strict-typing=${{ matrix.strict-typing }})
99+
steps:
100+
- name: Checkout repository
101+
uses: actions/checkout@v4
102+
103+
- name: Set up Python 3.13
104+
uses: actions/setup-python@v5
105+
with:
106+
python-version: "3.13"
107+
108+
- name: Set up Node.js (needed for Azurite and Functions Core Tools)
109+
uses: actions/setup-node@v4
110+
with:
111+
node-version: "20.x"
112+
113+
- name: Install Azurite and Azure Functions Core Tools
114+
run: |
115+
npm install -g azurite@latest
116+
npm install -g azure-functions-core-tools@4 --unsafe-perm true
117+
118+
- name: Start Azurite
119+
shell: bash
120+
# The DurableTask.AzureStorage provider in recent extension bundles uses
121+
# a newer Azure Storage REST API version than the CI Azurite build
122+
# recognizes, which otherwise fails task-hub creation with "The API
123+
# version ... is not supported by Azurite". ``--skipApiVersionCheck``
124+
# bypasses that guard so the durable provider can talk to Azurite.
125+
run: |
126+
azurite --silent --skipApiVersionCheck --location /tmp/azurite --blobPort 10000 --queuePort 10001 --tablePort 10002 &
127+
sleep 2
128+
129+
- name: Install nox
130+
run: |
131+
python -m pip install --upgrade pip
132+
pip install nox
133+
134+
- name: Run end-to-end tests
135+
# nox provisions each sample app's in-app virtual environment (a link to
136+
# a single editable install) so the Functions worker loads our
137+
# grpc/protobuf rather than its bundled copies.
138+
env:
139+
AZURE_FUNCTIONS_DURABLE_STRICT_TYPING: ${{ matrix.strict-typing }}
140+
run: nox -s functions_e2e
141+
142+
# The harness captures each sample app's ``func start`` stdout/stderr to
143+
# ``<app>/_func_host.log``. On failure that log holds the real host/worker
144+
# error (durable extension load, storage wiring, worker crash) that the
145+
# pytest summary does not surface, so upload it for diagnosis.
146+
- name: Upload Functions host logs
147+
if: failure()
148+
uses: actions/upload-artifact@v4
149+
with:
150+
name: func-host-logs-strict-${{ matrix.strict-typing }}
151+
path: tests/azure-functions-durable/e2e/apps/*/_func_host.log
152+
if-no-files-found: warn
153+
retention-days: 7
154+

.github/workflows/typecheck.yml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ on:
77
tags:
88
- "v*"
99
- "azuremanaged-v*"
10+
- "azurefunctions-v*"
1011
pull_request:
1112
branches:
1213
- "main"
@@ -46,3 +47,25 @@ jobs:
4647
4748
- name: Run pyright (strict, Python 3.10)
4849
run: pyright
50+
51+
pyright-azurefunctions:
52+
runs-on: ubuntu-latest
53+
steps:
54+
- name: Checkout repository
55+
uses: actions/checkout@v4
56+
57+
- name: Set up Python 3.13 (lowest supported by azure-functions-durable)
58+
uses: actions/setup-python@v5
59+
with:
60+
python-version: "3.13"
61+
62+
- name: Install packages and dependencies
63+
run: |
64+
python -m pip install --upgrade pip
65+
pip install -r requirements.txt
66+
pip install -e ".[azure-blob-payloads,opentelemetry]"
67+
pip install -e ./azure-functions-durable
68+
pip install pyright
69+
70+
- name: Run pyright (strict, Python 3.13)
71+
run: pyright -p azure-functions-durable/pyrightconfig.json

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,12 @@ ENV/
111111
env.bak/
112112
venv.bak/
113113

114+
# Azure Functions E2E test host logs (written by the test harness)
115+
_func_host.log
116+
117+
# Azure Functions E2E history-export output (written by the sample app writer)
118+
_export_output/
119+
114120
# Spyder project settings
115121
.spyderproject
116122
.spyproject

.vscode/settings.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,6 @@
3131
"coverage.cobertura.xml"
3232
],
3333
"makefile.configureOnOpen": false,
34-
"debugpy.debugJustMyCode": false
34+
"debugpy.debugJustMyCode": false,
35+
"python-envs.defaultEnvManager": "ms-python.python:system"
3536
}
Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## 2.0.0b1
9+
10+
First preview (beta) release of `azure-functions-durable` 2.x — a ground-up
11+
rewrite of the Azure Durable Functions Python SDK built on top of the
12+
[`durabletask`](https://pypi.org/project/durabletask/) SDK. This is a preview
13+
release; APIs may change before the stable 2.0.0.
14+
15+
### Why the rewrite
16+
17+
The 1.x SDK implemented the Durable Functions out-of-process programming model
18+
directly, with its own orchestration action/state model and a JSON protocol
19+
tailored to the classic Durable Functions host extension. 2.x instead builds on
20+
the `durabletask` Python SDK — the same gRPC-based runtime that powers the
21+
Durable Task Scheduler (DTS) and the modern Durable Functions host. Building on
22+
durabletask means:
23+
24+
- a single orchestration/entity execution core, serialization pipeline, and
25+
retry/versioning implementation shared with the broader durabletask
26+
ecosystem, instead of a Functions-specific reimplementation;
27+
- Functions users can adopt durabletask-native APIs and patterns directly,
28+
while existing v1 code keeps working through the compatibility layer; and
29+
- less protocol drift between the Python worker and the durable backend.
30+
31+
### Underlying packages
32+
33+
- [`durabletask`](https://pypi.org/project/durabletask/) — orchestration and
34+
entity client/worker, executed over gRPC.
35+
- [`azure-functions`](https://pypi.org/project/azure-functions/) — the
36+
decorator/binding programming model (`DFApp` / `Blueprint`).
37+
- [`azure-identity`](https://pypi.org/project/azure-identity/) — Managed
38+
Identity token acquisition for durable HTTP calls.
39+
40+
### Breaking changes (from `azure-functions-durable` 1.x)
41+
42+
- **Python 3.13+ is now required** (1.x supported 3.10+). On Functions Python
43+
workers older than 3.13, worker dependencies are not isolated from your app's
44+
dependencies, which causes a `grpc` version conflict with the durable runtime
45+
at load time. Python 3.13 enables worker dependency isolation, avoiding the
46+
collision.
47+
- **The classic (v1) programming model has been dropped.** Only the
48+
decorator-based application model (`DFApp` / `Blueprint`, the Python v2
49+
programming model) is supported; the `function.json`-based model is not.
50+
- **The OpenAI Agents integration has been removed.** The
51+
`azure.durable_functions.openai_agents` package (durable OpenAI Agents SDK
52+
orchestration) is not part of 2.x.
53+
- **Runtime target.** 2.x speaks the durabletask gRPC protocol used by the
54+
Durable Task Scheduler and the modern Durable Functions host, rather than the
55+
classic Durable Functions extension protocol.
56+
- **Primary client and context APIs use durabletask names.** The main surface
57+
is now durabletask's (e.g. `schedule_new_orchestration`,
58+
`get_orchestration_state`, `wait_for_orchestration_completion`). The v1
59+
`DurableOrchestrationClient` method names remain available as deprecated
60+
aliases that emit `DeprecationWarning` (see [Deprecated](#deprecated)).
61+
62+
### Added
63+
64+
New capabilities beyond the v1 surface, most inherited from `durabletask`:
65+
66+
- **durabletask-native authoring.** Two-argument orchestrator, entity, and
67+
activity functions (`def orchestrator(ctx, input)`, `def entity(ctx, input)`,
68+
`def activity(ctx, input)`) and class-based entities (`DurableEntity`) are
69+
first-class, alongside the supported v1-style single-argument functions. For
70+
activities, `activity_trigger` adapts a two-argument `(ctx, input)` function
71+
to the host's single-input calling convention automatically (the context is a
72+
placeholder object; accessing context attributes raises `NotImplementedError`).
73+
- **`DurableFunctionsClient.rewind_orchestration(...)`** rewinds a failed
74+
orchestration to its last known good state (inherited from durabletask).
75+
- **`DFApp.configure_scheduled_tasks()`** opts an app in to durabletask
76+
scheduled (recurring) tasks by registering the schedule entity and operation
77+
orchestrator. Schedules are then managed from a client via
78+
`durabletask.scheduled.ScheduledTaskClient`. Scheduled tasks are not
79+
registered unless this method is called.
80+
- **`DFApp.configure_history_export(writer=...)`** opts an app in to durabletask
81+
history export by registering the export-job entity, driving orchestrator, and
82+
activities. Supply the `HistoryWriter` here; the activities resolve their
83+
durabletask client per invocation from a `durable_client_input` binding, so
84+
the export activities run correctly across a scaled-out, multi-worker
85+
deployment (each invocation resolves its own client). This is a correctness
86+
property, not a large-export throughput guarantee. Export jobs are
87+
driven from a client via
88+
`durabletask.extensions.history_export.ExportHistoryClient`. Continuous export
89+
(`ExportMode.CONTINUOUS`) is not supported on Azure Functions: the
90+
Functions-registered export entity rejects it at job creation (the job ends
91+
`FAILED` with an explanatory reason). Continuous tailing needs the host's
92+
`ListInstanceIds` gRPC call, which the Durable Functions host extension does
93+
not implement; the instance-enumeration activity uses a Functions-specific
94+
implementation based on `QueryInstances` for the same reason. This is an
95+
experimental beta feature intended for bounded, low-volume batch-export
96+
windows: the `QueryInstances`-based enumeration re-scans and re-sorts the
97+
terminal-instance population for each batch, so it is not yet suited to
98+
production-scale history export. Efficient large exports depend on a
99+
host-side completed-time paging API that the host extension does not yet
100+
provide.
101+
- **`DurableOrchestrationContext.call_http(...)`** makes durable HTTP calls from
102+
orchestrators, restoring the v1 API. The request is executed by a built-in
103+
activity and, when the endpoint responds with `202 Accepted` and a `Location`
104+
header, is automatically polled to completion (honoring `Retry-After`).
105+
`ManagedIdentityTokenSource` can be supplied to attach a Managed Identity
106+
bearer token to the request. `DurableHttpRequest` and `DurableHttpResponse`
107+
are exported from `azure.durable_functions`.
108+
- **`orchestration_trigger(..., input_type=...)`** decodes a v1-style
109+
`context.get_input()` to the declared type; a call-site `expected_type` on
110+
`get_input` takes precedence.
111+
112+
### Compatibility with v1
113+
114+
To ease migration, 2.x ships a compatibility layer over the durabletask
115+
surface:
116+
117+
- **v1-style single-argument functions** (`def orchestrator(context)`,
118+
`def entity(context)`) are supported. The worker detects the function shape
119+
and, for single-argument functions, delivers a functional
120+
`DurableOrchestrationContext` / `DurableEntityContext` that wraps the
121+
durabletask context and exposes the v1 API — for orchestrations:
122+
`get_input`, `call_activity`/`call_activity_with_retry`,
123+
`call_sub_orchestrator`/`call_sub_orchestrator_with_retry`, `create_timer`,
124+
`wait_for_external_event`, `continue_as_new`, `set_custom_status`,
125+
`task_all`/`task_any`, `call_entity`/`signal_entity`, `new_uuid`/`new_guid`,
126+
`custom_status`, `will_continue_as_new`, `parent_instance_id`, and
127+
`function_context`; and for entities: `entity_name`, `entity_key`,
128+
`operation_name`, `get_input`, `get_state` (with `initializer`), `set_state`,
129+
`set_result`, and `destruct_on_exit`. The operation result is taken from
130+
`set_result(...)`, falling back to the function's return value.
131+
- **v1 return-type wrappers** `DurableOrchestrationStatus`,
132+
`PurgeHistoryResult`, and `EntityStateResponse` are returned by the deprecated
133+
client methods and exported from `azure.durable_functions`.
134+
- **`HttpManagementPayload`** subclasses `dict`, so it is directly
135+
JSON-serializable via `json.dumps(payload)` and supports mapping-style access,
136+
matching v1 usage.
137+
- **`create_http_management_payload`** accepts either the durabletask
138+
`(request, instance_id)` or the v1 `(instance_id)` signature.
139+
140+
### Deprecated
141+
142+
These v1 names are retained as shims that delegate to their durabletask
143+
equivalents and emit `DeprecationWarning`; prefer the durabletask names in new
144+
code:
145+
146+
- `DurableOrchestrationClient` (alias for `DurableFunctionsClient`) and its
147+
method names: `start_new`, `get_status`, `get_status_all`, `get_status_by`,
148+
`raise_event`, `terminate`, `purge_instance_history`,
149+
`purge_instance_history_by`, `suspend`, `resume`, `restart`,
150+
`read_entity_state`, `get_client_response_links`, and
151+
`wait_for_completion_or_create_check_status_response`.
152+
- `rewind(...)` — delegates to `rewind_orchestration(...)`.
153+
- `signal_entity(..., operation_input=...)``operation_input` is an alias for
154+
`input`; `task_hub_name` / `connection_name` are accepted and ignored.
155+
- `RetryOptions` — maps the v1 millisecond-based constructor onto durabletask
156+
`RetryPolicy` (which uses `timedelta`). `RetryPolicy` is also exported from
157+
`azure.durable_functions`.
158+
- Compatibility aliases exported from `azure.durable_functions`:
159+
`DurableOrchestrationContext`, `DurableEntityContext`, `EntityId`,
160+
`ManagedIdentityTokenSource`, `TokenSource`, `Entity`, and
161+
`OrchestrationRuntimeStatus`.
162+
163+
### Known limitations
164+
165+
- Orchestration history is not exposed on the context;
166+
`DurableOrchestrationContext.histories` raises `NotImplementedError`. Use the
167+
client's `get_orchestration_history(...)` instead.
168+
- The client status methods accept the v1 `show_history` /
169+
`show_history_output` flags for signature compatibility but ignore them, so
170+
the returned status has no `historyEvents`. Use
171+
`get_orchestration_history(...)` to retrieve history.
172+
- Distributed tracing is not yet wired up. The Durable Functions host delivers
173+
the parent trace context and emits the orchestration/activity spans itself,
174+
so orchestrator user-code spans in the Python worker are not yet correlated
175+
to it, and durabletask's own span emission is intentionally left disabled to
176+
avoid duplicating the host's spans.

0 commit comments

Comments
 (0)