Skip to content

Commit 674e282

Browse files
committed
feat(testing): per-execution worker serialization
Replace the per-ARN lock with a per-execution worker that runs every operation for one execution one at a time on its own serial lane. - Worker model: SerialTaskLane, ExecutionWorker, ExecutionRegistry, ExecutionTask, CallableTask. - One checkpoint write transaction in checkpoint/core.py shared by the web and in process paths. - Checkpoint lifecycle actions returned as effects applied after the write. The reentrant notifier is gone. - The invocation gate lives on the worker status. - The blocking invoke runs off the lane so handler callbacks can run. - Store integration test on memory, filesystem, and sqlite. Caught and fixed a non-atomic write in FileSystemStore. - Runner accepts an optional store for store injection testing. - docs/architecture.md: new, describes the worker model. Closes #435
1 parent b839392 commit 674e282

35 files changed

Lines changed: 2771 additions & 1045 deletions

packages/aws-durable-execution-sdk-python-testing/README.md

Lines changed: 4 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -112,66 +112,10 @@ def test_my_durable_functions():
112112
assert three_result.result == '"5 6"'
113113
```
114114
## Architecture
115-
![Durable Functions Python Test Framework Architecture](assets/dar-python-test-framework-architecture.svg)
116-
117-
## Event Flow
118-
![Event Flow Sequence Diagram](assets/dar-python-test-framework-event-flow.svg)
119-
120-
1. **DurableTestRunner** starts execution via **Executor**
121-
2. **Executor** creates **Execution** and schedules initial invocation
122-
3. During execution, checkpoints are processed by **CheckpointProcessor**
123-
4. **Individual Processors** transform operation updates and may trigger events
124-
5. **ExecutionNotifier** broadcasts events to **Executor** (observer)
125-
6. **Executor** updates **Execution** state based on events
126-
7. **Execution** completion triggers final event notifications
127-
8. **DurableTestRunner** run() blocks until it receives completion event, and then returns `DurableFunctionTestResult`.
128-
129-
## Major Components
130-
131-
### Core Execution Flow
132-
- **DurableTestRunner** - Main entry point that orchestrates test execution
133-
- **Executor** - Manages execution lifecycle. Mutates Execution.
134-
- **Execution** - Represents the state and operations of a single durable execution
135-
136-
### Service Client Integration
137-
- **InMemoryServiceClient** - Replaces AWS Lambda service client for local testing. Injected into SDK via `DurableExecutionInvocationInputWithClient`
138-
139-
### Checkpoint Processing Pipeline
140-
- **CheckpointProcessor** - Orchestrates operation transformations and validation
141-
- **Individual Validators** - Validate operation updates and state transitions
142-
- **Individual Processors** - Transform operation updates into operations (step, wait, callback, context, execution)
143-
144-
### Execution status changes (Observer Pattern)
145-
- **ExecutionNotifier** - Notifies observers of execution events
146-
- **ExecutionObserver** - Interface for receiving execution lifecycle events
147-
- **Executor** implements `ExecutionObserver` to handle completion events
148-
149-
## Component Relationships
150-
151-
### 1. DurableTestRunner → Executor → Execution
152-
- **DurableTestRunner** serves as the main API entry point and sets up all components
153-
- **Executor** manages the execution lifecycle, handling invocations and state transitions
154-
- **Execution** maintains the state of operations and completion status
155-
156-
### 2. Service Client Injection
157-
- **DurableTestRunner** creates **InMemoryServiceClient** with **CheckpointProcessor**
158-
- **InProcessInvoker** injects the service client into SDK via `DurableExecutionInvocationInputWithClient`
159-
- When durable functions call checkpoint operations, they're intercepted by **InMemoryServiceClient**
160-
- **InMemoryServiceClient** delegates to **CheckpointProcessor** for local processing
161-
162-
### 3. CheckpointProcessor → Individual Validators → Individual Processors
163-
- **CheckpointProcessor** orchestrates the checkpoint processing pipeline
164-
- **Individual Validators** (CheckpointValidator, TransitionsValidator, and operation-specific validators) ensure operation updates are valid
165-
- **Individual Processors** (StepProcessor, WaitProcessor, etc.) transform `OperationUpdate` into `Operation`
166-
167-
### 4. Observer Pattern Flow
168-
The observer pattern enables loose coupling between checkpoint processing and execution management:
169-
170-
1. **CheckpointProcessor** processes operation updates
171-
2. **Individual Processors** detect state changes (completion, failures, timer scheduling)
172-
3. **ExecutionNotifier** broadcasts events to registered observers
173-
4. **Executor** (as ExecutionObserver) receives notifications and updates **Execution** state
174-
5. **Execution** complete_* methods finalize the execution state
115+
116+
See [docs/architecture.md](docs/architecture.md) for framework
117+
internals. It covers the components, the worker model, the checkpoint
118+
flow, pagination, and a map of the code.
175119

176120

177121
## Documentation

packages/aws-durable-execution-sdk-python-testing/assets/dar-python-test-framework-architecture.svg

Lines changed: 0 additions & 1 deletion
This file was deleted.

packages/aws-durable-execution-sdk-python-testing/assets/dar-python-test-framework-event-flow.svg

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)