Reuse Azure Functions durable workers - #217
Conversation
Cache host-driven workers per function handle and make direct-execute registration thread-safe and idempotent without changing core durabletask registration semantics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: b29a8f40-77ba-48d9-a3f7-2748491f3791
There was a problem hiding this comment.
Pull request overview
This PR updates the azure-functions-durable package to reuse a single host-driven DurableFunctionsWorker instance across repeated orchestration/entity invocations, avoiding per-invocation worker initialization and making registration safe under repeated and concurrent execution.
Changes:
- Reused a shared
DurableFunctionsWorkerinstance per orchestrator/entity handle (instead of constructing a new worker per invocation). - Made direct-execute orchestrator/entity registration idempotent and added concurrency-focused tests.
- Documented the behavioral change in the
azure-functions-durablechangelog under## Unreleased.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/azure-functions-durable/test_worker_compat.py | Adds concurrency/reinvocation coverage for direct-execute orchestration and entity paths. |
| tests/azure-functions-durable/test_orchestrator_compat.py | Updates orchestrator adapter tests to validate worker reuse across multiple handle invocations. |
| tests/azure-functions-durable/test_decorator_compat.py | Adds decorator-level test ensuring entity trigger reuses a single worker across invocations. |
| azure-functions-durable/CHANGELOG.md | Records the user-visible change under ## Unreleased → CHANGED. |
| azure-functions-durable/azure/durable_functions/worker.py | Introduces lock-guarded “register once” logic for orchestrators/entities in the host-driven worker. |
| azure-functions-durable/azure/durable_functions/orchestrator.py | Reworks Orchestrator to hold/reuse a worker instance and have create() capture a shared worker per handle. |
Keep repeated invocation idempotent only for the original callable and preserve duplicate-registration errors when a shared worker receives a different function with the same name. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: b29a8f40-77ba-48d9-a3f7-2748491f3791
|
We considered the retained-memory tradeoff of keeping one A local measurement with 200 orchestration/entity handles found:
This is a directional measurement from the Windows development environment rather than a production Functions SKU benchmark, but it indicates the overhead is modest even for an unusually large app. The per-worker Consolidating to one worker per app, blueprint, or process remains possible and would remove most of this overhead. It is more complex because it introduces shared registry lifecycle and re-index/reload semantics, cross-blueprint ownership, callable/name conflict handling, and broader concurrency considerations. Given the measured impact, this PR keeps the safer per-handle isolation model and does not make that broader lifecycle change. |
Summary
DurableFunctionsWorkerper orchestration or entity function handledurabletaskduplicate-registration behaviorTesting
python -m pytest tests/azure-functions-durable -m "not dts and not azurite and not functions_e2e" -qpython -m flake8 azure-functions-durablepython -m flake8 tests/azure-functions-durablepython -m pyright -p azure-functions-durable/pyrightconfig.json azure-functions-durable/azure/durable_functions/worker.py azure-functions-durable/azure/durable_functions/orchestrator.py azure-functions-durable/azure/durable_functions/decorators/durable_app.pyFixes #178