Skip to content

Reuse Azure Functions durable workers - #217

Merged
andystaples merged 2 commits into
mainfrom
andystaples-resolve-issue-178
Jul 29, 2026
Merged

Reuse Azure Functions durable workers#217
andystaples merged 2 commits into
mainfrom
andystaples-resolve-issue-178

Conversation

@andystaples

Copy link
Copy Markdown
Contributor

Summary

  • reuse one host-driven DurableFunctionsWorker per orchestration or entity function handle
  • make Functions-only direct-execute registration idempotent and thread-safe, including mixed-case entity names
  • preserve existing core durabletask duplicate-registration behavior
  • cover repeated and concurrent orchestration/entity invocations

Testing

  • python -m pytest tests/azure-functions-durable -m "not dts and not azurite and not functions_e2e" -q
  • python -m flake8 azure-functions-durable
  • python -m flake8 tests/azure-functions-durable
  • python -m pyright -p azure-functions-durable/pyrightconfig.json azure-functions-durable/azure/durable_functions/worker.py azure-functions-durable/azure/durable_functions/orchestrator.py azure-functions-durable/azure/durable_functions/decorators/durable_app.py

Fixes #178

Cache host-driven workers per function handle and make direct-execute registration thread-safe and idempotent without changing core durabletask registration semantics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b29a8f40-77ba-48d9-a3f7-2748491f3791
Copilot AI review requested due to automatic review settings July 28, 2026 19:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the azure-functions-durable package to reuse a single host-driven DurableFunctionsWorker instance across repeated orchestration/entity invocations, avoiding per-invocation worker initialization and making registration safe under repeated and concurrent execution.

Changes:

  • Reused a shared DurableFunctionsWorker instance per orchestrator/entity handle (instead of constructing a new worker per invocation).
  • Made direct-execute orchestrator/entity registration idempotent and added concurrency-focused tests.
  • Documented the behavioral change in the azure-functions-durable changelog under ## Unreleased.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/azure-functions-durable/test_worker_compat.py Adds concurrency/reinvocation coverage for direct-execute orchestration and entity paths.
tests/azure-functions-durable/test_orchestrator_compat.py Updates orchestrator adapter tests to validate worker reuse across multiple handle invocations.
tests/azure-functions-durable/test_decorator_compat.py Adds decorator-level test ensuring entity trigger reuses a single worker across invocations.
azure-functions-durable/CHANGELOG.md Records the user-visible change under ## UnreleasedCHANGED.
azure-functions-durable/azure/durable_functions/worker.py Introduces lock-guarded “register once” logic for orchestrators/entities in the host-driven worker.
azure-functions-durable/azure/durable_functions/orchestrator.py Reworks Orchestrator to hold/reuse a worker instance and have create() capture a shared worker per handle.

Comment thread azure-functions-durable/azure/durable_functions/worker.py Outdated
Keep repeated invocation idempotent only for the original callable and preserve duplicate-registration errors when a shared worker receives a different function with the same name.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b29a8f40-77ba-48d9-a3f7-2748491f3791
Copilot AI review requested due to automatic review settings July 28, 2026 19:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@andystaples

Copy link
Copy Markdown
Contributor Author

We considered the retained-memory tradeoff of keeping one DurableFunctionsWorker per orchestration/entity handle, particularly for apps with a large durable-function surface.

A local measurement with 200 orchestration/entity handles found:

  • one worker holding 200 registrations: about 45 KB of traced Python heap;
  • 200 workers holding one registration each: about 1.37 MB of traced Python heap;
  • observed process RSS attributable to the 200-worker case: roughly 2.5–2.7 MB.

This is a directional measurement from the Windows development environment rather than a production Functions SKU benchmark, but it indicates the overhead is modest even for an unusually large app. The per-worker ThreadPoolExecutor does not start threads because the host-driven path never submits work to it; the retained cost is the executor and associated registries, locks/events, caches, and options.

Consolidating to one worker per app, blueprint, or process remains possible and would remove most of this overhead. It is more complex because it introduces shared registry lifecycle and re-index/reload semantics, cross-blueprint ownership, callable/name conflict handling, and broader concurrency considerations. Given the measured impact, this PR keeps the safer per-handle isolation model and does not make that broader lifecycle change.

@andystaples
andystaples merged commit 9bf3f14 into main Jul 29, 2026
25 checks passed
@andystaples
andystaples deleted the andystaples-resolve-issue-178 branch July 29, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

azure-functions-durable: avoid constructing a new DurableFunctionsWorker per invocation

3 participants