Skip to content

Commit 5ff66f1

Browse files
sfmskywalkersfmskywalker
committed
Adds activity execution call stack support
Implements a call stack mechanism to track the execution chain, enabling visibility into the invocation hierarchy. Introduces new fields to scheduling models and runtime contexts to store call stack information. Includes EF Core migrations for various database providers to support new columns in `ActivityExecutionRecords`. Provides an API to query and reconstruct the call stack for a given activity execution.
1 parent 6e79781 commit 5ff66f1

58 files changed

Lines changed: 4118 additions & 69 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Elsa.sln‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "solution", "solution", "{7D
1919
icon.png = icon.png
2020
NuGet.Config = NuGet.Config
2121
README.md = README.md
22+
plan-activityExecutionCallStack.prompt.md = plan-activityExecutionCallStack.prompt.md
2223
EndProjectSection
2324
EndProject
2425
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "doc", "doc", "{0354F050-3992-4DD4-B0EE-5FBA04AC72B6}"

‎plan-activityExecutionCallStack.prompt.md‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
# Plan: Activity Execution Call Stack Implementation (Hybrid: explicit + ambient)
2+
3+
This plan implements a comprehensive call stack mechanism to track the execution chain from root workflow through all parent activities to a specific activity execution, enabling visibility into the complete invocation hierarchy when viewing activity execution records.
4+
5+
Core design
6+
- Explicit predecessor: Activities that know the causal predecessor (e.g., a completed child that schedules the next) set `SchedulingActivityExecutionId` directly via scheduling options.
7+
- Ambient fallback: During completion callbacks, bookmark resumes, and child-workflow starts, the workflow sets an ambient "current scheduling source" on the `WorkflowExecutionContext`. If a schedule call omits `SchedulingActivityExecutionId`, the scheduler fills it from the ambient. This minimizes code churn while preserving correctness.
8+
- Structural vs temporal: Keep `Owner`/`ParentActivityExecutionContext` for structural containment; use `SchedulingActivityExecutionId`/`SchedulingWorkflowInstanceId` for the temporal execution chain.
9+
10+
## Steps
11+
12+
### 1. Add call stack fields to scheduling models throughout the chain
13+
14+
Add `SchedulingActivityExecutionId` (nullable `string`) to:
15+
- [`ScheduleWorkOptions`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Options/ScheduleWorkOptions.cs)
16+
- [`ScheduledActivityOptions`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Models/ScheduledActivityOptions.cs)
17+
- [`ActivityWorkItem`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Models/ActivityWorkItem.cs)
18+
- [`ActivityInvocationOptions`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Options/ActivityInvocationOptions.cs)
19+
20+
Include clear XML documentation explaining this tracks the temporal/execution predecessor (distinct from structural `Owner`/`ParentActivityExecutionContext`).
21+
22+
Update all constructors and property mappings in:
23+
- [`WorkflowExecutionContextSchedulerStrategy.Schedule`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Services/WorkflowExecutionContextSchedulerStrategy.cs)
24+
- [`DefaultActivitySchedulerMiddleware.ExecuteWorkItemAsync`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Middleware/Workflows/DefaultActivitySchedulerMiddleware.cs)
25+
26+
Thread this value through the scheduling chain.
27+
28+
### 2. Store call stack fields in runtime and persisted contexts
29+
30+
Add the following fields to [`ActivityExecutionContext`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Contexts/ActivityExecutionContext.cs):
31+
- `SchedulingActivityExecutionId` (nullable `string`)
32+
- `SchedulingActivityId` (nullable `string` - denormalized for convenience)
33+
- `SchedulingWorkflowInstanceId` (nullable `string` - for cross-workflow tracking)
34+
35+
Include XML comments distinguishing these from `ParentActivityExecutionContext`:
36+
- **`ParentActivityExecutionContext`**: The structural container activity (e.g., Flowchart contains all its children). Represents the hierarchical parent in the workflow structure.
37+
- **`SchedulingActivityExecutionId`**: The temporal/execution predecessor that directly triggered execution of this activity. Tracks the execution sequence, not the structural hierarchy.
38+
- **`SchedulingWorkflowInstanceId`**: The workflow instance ID of the activity that invoked this activity's workflow. Set when crossing workflow boundaries (e.g., via `ExecuteWorkflow` or `DispatchWorkflow`).
39+
40+
Update [`WorkflowExecutionContext.CreateActivityExecutionContextAsync`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Contexts/WorkflowExecutionContext.cs) to accept and store these fields from `ActivityInvocationOptions`.
41+
42+
Add corresponding fields to [`ActivityExecutionRecord`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Runtime/Entities/ActivityExecutionRecord.cs):
43+
- `SchedulingActivityExecutionId` (nullable `string`)
44+
- `SchedulingActivityId` (nullable `string`)
45+
- `SchedulingWorkflowInstanceId` (nullable `string`)
46+
- `CallStackDepth` (nullable `int`)
47+
48+
Update [`DefaultActivityExecutionMapper.MapAsync`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Runtime/Services/DefaultActivityExecutionMapper.cs) to populate these fields from the execution context. Calculate `CallStackDepth` by traversing the `SchedulingActivityExecutionId` chain until reaching null.
49+
50+
### 3. Add ambient scheduling source to the workflow context
51+
52+
Add an ambient scheduling source to [`WorkflowExecutionContext`]:
53+
- Fields (transient): `CurrentSchedulingActivityExecutionId`, `CurrentSchedulingWorkflowInstanceId`.
54+
- API: `IDisposable BeginSchedulingScope(string? activityExecutionId, string? workflowInstanceId)` that pushes values and restores previous values on dispose.
55+
56+
Set ambient scope in these places:
57+
- Around owner completion callbacks (where next activities are scheduled).
58+
- Around bookmark-resume handlers (background completions resuming the workflow).
59+
- At child-workflow start (root activity creation).
60+
61+
Modify [`WorkflowExecutionContextSchedulerStrategy.Schedule`] to set `SchedulingActivityExecutionId` and `SchedulingWorkflowInstanceId` from `ScheduleWorkOptions` if provided; otherwise fall back to the ambient `WorkflowExecutionContext` values.
62+
63+
### 4. Update composite activities to capture scheduling activity context
64+
65+
In [`Flowchart.Counters.cs`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Activities/Flowchart/Activities/Flowchart.Counters.cs):
66+
- Update `ScheduleOutboundActivityAsync` to populate `ScheduleWorkOptions.SchedulingActivityExecutionId = completedActivityContext.Id` when a completed activity schedules its outbound activities.
67+
- Update `MaybeScheduleBackwardConnectionActivityAsync` to include `SchedulingActivityExecutionId` in the `ScheduleWorkOptions`.
68+
- Update `MaybeScheduleWaitAllActivityAsync`, `MaybeScheduleWaitAllActiveActivityAsync`, `MaybeScheduleWaitAnyActivityAsync` to pass `SchedulingActivityExecutionId` when scheduling.
69+
70+
In [`Flowchart.Tokens.cs`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Core/Activities/Flowchart/Activities/Flowchart.Tokens.cs):
71+
- Update `OnChildCompletedTokenBasedLogicAsync` to pass `SchedulingActivityExecutionId` in `ScheduleWorkOptions` when scheduling subsequent activities.
72+
73+
Apply the same pattern to other composite activities:
74+
- `Sequence`
75+
- `ForEach`
76+
- `Parallel`
77+
- `While`
78+
- `Do`
79+
- Any other activities that schedule child activities based on completion
80+
81+
When scheduling a child activity that was directly triggered by another activity's completion, set `SchedulingActivityExecutionId` to the completing activity's execution context ID. When omitted, the ambient scope ensures a sensible fallback.
82+
83+
### 5. Handle cross-workflow call stack linkage (span by default)
84+
85+
For [`ExecuteWorkflow`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Runtime/Activities/ExecuteWorkflow.cs) and [`DispatchWorkflow`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Runtime/Activities/DispatchWorkflow.cs):
86+
87+
- Capture the calling activity's `ExecutionId` and current `WorkflowInstanceId`.
88+
- Pass them through `RunWorkflowOptions` and `DispatchWorkflowRequest` to the child workflow.
89+
- When the child workflow starts, set the first activity's:
90+
- `SchedulingActivityExecutionId` to the parent's invocation activity's execution ID.
91+
- `SchedulingWorkflowInstanceId` to the parent workflow instance ID.
92+
- Also set the ambient scope (`BeginSchedulingScope`) for the duration of child start so subsequent schedules inherit these values by default.
93+
94+
Cross-workflow chains should be considered part of the call stack by default (span by default).
95+
96+
### 6. Add call stack depth and create database migration
97+
98+
Add `CallStackDepth` (nullable `int`) field to [`ActivityExecutionRecord`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Runtime/Entities/ActivityExecutionRecord.cs).
99+
100+
In [`DefaultActivityExecutionMapper.MapAsync`](file:///Users/sipke/Projects/Elsa/elsa-core/main/src/modules/Elsa.Workflows.Runtime/Services/DefaultActivityExecutionMapper.cs):
101+
- Calculate `CallStackDepth` by traversing the source `ActivityExecutionContext.SchedulingActivityExecutionId` chain until reaching null.
102+
- Use root depth = 0 (documented convention).
103+
- Store this value in the `ActivityExecutionRecord`.
104+
105+
Create EF Core migration adding indexed columns to `ActivityExecutionRecord` table:
106+
- `SchedulingActivityExecutionId` (indexed, nullable)
107+
- `SchedulingActivityId` (indexed, nullable)
108+
- `SchedulingWorkflowInstanceId` (indexed, nullable)
109+
- `CallStackDepth` (indexed, nullable)
110+
111+
The `CallStackDepth` index enables efficient filtering by execution depth without reconstructing the full chain (e.g., "show me all activities at depth > 5").
112+
113+
### 7. Implement call stack query and reconstruction APIs
114+
115+
Implement `IActivityExecutionStore.GetExecutionChainAsync(string activityExecutionId, bool includeCrossWorkflowChain = true, int? skip = null, int? take = null)` that:
116+
- Recursively queries `SchedulingActivityExecutionId` until reaching root (null).
117+
- Follows `SchedulingWorkflowInstanceId` across workflow boundaries by default (span by default). Optionally allow disabling cross-workflow span via parameter.
118+
- Supports pagination via `skip` and `take` parameters to handle deep call stacks efficiently.
119+
- Returns a paginated result containing:
120+
- `Items`: List of execution records (ordered from root to current activity, or subset if paginated)
121+
- `TotalCount`: Total number of items in the full chain
122+
- `Skip`: The skip value used
123+
- `Take`: The take value used
124+
- When pagination is not specified (`skip` and `take` are null), returns the full chain.
125+
126+
Add extension methods:
127+
- **`ActivityExecutionContext.GetExecutionChain(int? skip = null, int? take = null)`**: Reconstruct the runtime call stack by traversing `SchedulingActivityExecutionId`, returning a paginated result from root to current activity.
128+
- **`ActivityExecutionRecord.GetExecutionChainAsync(IActivityExecutionStore, bool includeCrossWorkflowChain = true, int? skip = null, int? take = null)`**: Reconstruct persisted call stacks by querying the store with pagination support.
129+
130+
Both methods should return results ordered from root to current activity, with pagination applied after ordering.
131+
132+
Add REST API endpoint:
133+
- **`GET /api/workflow-instances/{workflowInstanceId}/activity-executions/{activityExecutionId}/call-chain`**
134+
- Query parameters:
135+
- `includeCrossWorkflowChain` (bool, default: true): Include parent workflow activities across workflow boundaries
136+
- `skip` (int?, optional): Number of items to skip (for pagination)
137+
- `take` (int?, optional): Number of items to return (for pagination, recommended max: 100)
138+
- Response:
139+
- `items`: Array of activity execution records
140+
- `totalCount`: Total number of items in the full chain
141+
- `skip`: The skip value used
142+
- `take`: The take value used (or null if full chain returned)
143+
- This enables UI to implement paginated/lazy loading for deep call stacks.
144+
145+
## Further Considerations
146+
147+
### 1. Structural vs temporal hierarchy documentation
148+
149+
`ParentActivityExecutionContext` represents the structural container (e.g., Flowchart contains all its children), while `SchedulingActivityExecutionId` tracks the temporal execution predecessor (e.g., Activity B completed and directly triggered Activity C).
150+
151+
These are orthogonal relationships:
152+
- A Flowchart can own many children (structural), but only a predecessor directly triggers the next (temporal).
153+
- When Activity B completes, it schedules the next child, establishing a temporal link via `SchedulingActivityExecutionId`.
154+
155+
All XML comments for these fields should explicitly clarify this distinction to prevent developer confusion and misuse.
156+
157+
### 2. Ambient scope guardrails
158+
159+
- The ambient scope must be short-lived and always disposed via `using`/`finally` to avoid leakage between unrelated scheduling operations.
160+
- The scheduler should prefer explicit `ScheduleWorkOptions.SchedulingActivityExecutionId`/`SchedulingWorkflowInstanceId` and only fall back to ambient when not provided.
161+
- Document that the ambient exists to reduce boilerplate and should not be relied upon when explicit causal context is readily available.
162+
163+
### 3. Cross-workflow boundary reconstruction (default span)
164+
165+
`SchedulingWorkflowInstanceId` enables reconstructing call stacks that span multiple workflow instances:
166+
- Parent Workflow (Instance A) → ExecuteWorkflow activity (in Instance A) → Child Workflow (Instance B) → failing activity (in Instance B).
167+
168+
Since span is the default, cross-instance traversal should occur unless explicitly disabled.
169+
170+
### 4. Call stack depth optimization trade-offs
171+
172+
Storing `CallStackDepth` trades a small amount of storage for simpler, faster queries and analytics:
173+
174+
**Benefits:**
175+
- Efficient filtering by depth ranges (e.g., "depth > 5").
176+
- Early termination in chain reconstruction.
177+
- Index-based analytics queries.
178+
- Lower storage than persisting full chains.
179+
180+
**Drawbacks:**
181+
- `CallStackDepth` becomes stale if parent records are deleted or altered post-hoc. Prefer immutable execution records.
182+
- Document that `CallStackDepth` is an optimization hint for querying, not an authoritative source if retention policies prune ancestors.
183+
184+
### 5. Testing matrix
185+
186+
- Sequential flow: A → B → C (explicit predecessor set, ambient unused).
187+
- Parallel fan-out: A schedules B and C (both record A as predecessor; ambient vs explicit).
188+
- Nested composites: Multiple owners scheduling into the same queue.
189+
- Background resume: Bookmark-based resumes interleaving with other work; ambient set during resume.
190+
- Cross-workflow: Execute/Dispatch child workflow; default spanning chain.
191+
- Deduplication scenarios: `PreventDuplicateScheduling` and re-scheduling.
192+
- Persistence/round-trips: Background/persisted scheduled activities using `ScheduledActivityOptions`.
193+
- Deep call stacks: Test pagination with chains deeper than 100 activities.
194+
- Cross-workflow pagination: Ensure pagination works correctly when spanning workflow boundaries.
195+
196+
### 6. Performance and pagination considerations
197+
198+
**Deep call stack handling:**
199+
- For very deep call stacks (e.g., recursive workflows or long-running sequential processes), retrieving the entire chain in a single query can be expensive.
200+
- Pagination (`skip`/`take`) enables efficient loading in the UI with incremental/lazy loading patterns.
201+
- Recommend default `take` of 50-100 items per page for REST API calls.
202+
- The `CallStackDepth` field enables quick assessment of chain depth before deciding whether to paginate.
203+
204+
**Query optimization strategies:**
205+
- Use indexed lookups on `SchedulingActivityExecutionId` to traverse the chain efficiently.
206+
- Consider caching strategies for frequently accessed chains (e.g., recently failed activities).
207+
- For cross-workflow queries, implement efficient join strategies or batched lookups to minimize round-trips.
208+
- Document that pagination is "forward-only" (skip/take from root toward current) to align with typical debugging workflows (start at root, drill down).
209+
210+
**REST API rate limiting:**
211+
- Consider rate limiting on the call chain endpoint if it becomes a performance bottleneck.
212+
- Monitor query performance and adjust default pagination sizes based on observed data.

0 commit comments

Comments
 (0)