perf: parallelize delegation export fetches across subunits - #2338
perf: parallelize delegation export fetches across subunits#2338TheTechArch wants to merge 1 commit into
Conversation
The export fetched delegation data one giver (parent org + each subunit) at a time, with a sequential await inside the per-giver loop for each of the four data types (roles, access packages, single rights, instances). For an org with N subunits this chained ~4*N downstream calls end-to-end, so export time grew linearly with subunit count and was slow for large companies. Fan the per-giver calls out concurrently via a bounded FetchPerGiver helper (Task.WhenAll, capped at MaxGiverConcurrency=10 via SemaphoreSlim) so we speed things up without overwhelming the downstream API. Results are returned in giver order, keeping CSV row content and ordering unchanged. Role- and package-name lookups now overlap with the per-giver fetches. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthrough
ChangesBounded Concurrent Per-Giver Export Fetching
Sequence Diagram(s)sequenceDiagram
participant ExportService as DelegationExportService
participant FetchPerGiver
participant Semaphore as SemaphoreSlim
participant API as Downstream APIs
ExportService->>FetchPerGiver: givers[] + fetchFunc (BuildRoleRows / BuildAccessPackageRows / etc.)
loop Fan-out per giver (bounded by MaxGiverConcurrency)
FetchPerGiver->>Semaphore: WaitAsync()
FetchPerGiver->>API: fetchFunc(giver)
API-->>FetchPerGiver: TResult
FetchPerGiver->>Semaphore: Release()
end
FetchPerGiver-->>ExportService: TResult[] (giver-ordered)
ExportService->>ExportService: iterate by index to build export rows
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Improves delegation export performance for organizations with many subunits by fetching per-giver delegation data concurrently (bounded) while preserving deterministic output ordering.
Changes:
- Introduces bounded parallel fan-out for per-giver downstream fetches via a new
FetchPerGiverhelper usingSemaphoreSlim+Task.WhenAll. - Overlaps role/package name lookups with the per-giver permission/delegation fetches to reduce end-to-end latency.
- Refactors row-building loops to consume pre-fetched per-giver results in giver order (stable CSV ordering).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Dictionary<Guid, string> roleNameLookup = roleNameLookupTask.Result; | ||
| List<RolePermission>[] permissionsPerGiver = permissionsTask.Result; |
There was a problem hiding this comment.
Taking the change, but noting the stated reasoning does not hold here. After await Task.WhenAll(a, b) both tasks are already complete, so there is no deadlock risk; and if either had faulted, the await would have thrown before this line is reached, so there is no AggregateException wrapping either. That said, await roleNameLookupTask is free and more idiomatic, so I will switch it.
| Dictionary<Guid, string> packageNames = packageNamesTask.Result; | ||
| Dictionary<Guid, List<PackagePermission>>[] delegationsPerGiver = delegationsTask.Result; |
There was a problem hiding this comment.
Same as the sibling thread — accepting for readability, though the AggregateException/deadlock rationale does not apply after an await Task.WhenAll.
|
|
Triaging the automated review so this is easier to pick up — all seven Copilot comments now have replies, and CodeRabbit reported no actionable comments. Accepted (3):
Declined (4): the What I would actually value a human opinion on, since the bots did not raise it:
For what it is worth on thread-safety: the clients only read from |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (3)
backend/src/Altinn.AccessManagement.UI/Altinn.AccessManagement.UI.Core/Services/DelegationExportService.cs:27
- The comment on
MaxGiverConcurrencyreads like a global safeguard, but the semaphore gate is created insideFetchPerGiver, so the bound applies per export invocation. Clarify the comment to avoid implying a global cap across concurrent exports.
// Upper bound on concurrent downstream calls when fanning out over givers (parent + subunits),
// so exports for orgs with many subunits stay fast without overwhelming the downstream API.
private const int MaxGiverConcurrency = 10;
backend/src/Altinn.AccessManagement.UI/Altinn.AccessManagement.UI.Core/Services/DelegationExportService.cs:177
- After awaiting
Task.WhenAll(...), preferawaiting the already-completed tasks instead of accessing.Result. This keeps the async flow idiomatic and preserves the original exception propagation semantics without sync-over-async accessors.
Dictionary<Guid, string> roleNameLookup = roleNameLookupTask.Result;
List<RolePermission>[] permissionsPerGiver = permissionsTask.Result;
backend/src/Altinn.AccessManagement.UI/Altinn.AccessManagement.UI.Core/Services/DelegationExportService.cs:230
- Same pattern here: after
Task.WhenAll(...), useawaiton the tasks instead of.Resultfor idiomatic async code and consistent exception behavior.
Dictionary<Guid, string> packageNames = packageNamesTask.Result;
Dictionary<Guid, List<PackagePermission>>[] delegationsPerGiver = delegationsTask.Result;



Why
The delegation export (zip of CSV files) is slow for large companies with many subunits.
The export fetched delegation data one giver at a time (parent org + each subunit), with a sequential
awaitinside the per-giver loop for each of the four data types (roles, access packages, single rights, instances). For an org with N subunits this chained roughly4 × Ndownstream calls end-to-end, so export time grew linearly with subunit count.What changed
DelegationExportServicenow fans the per-giver calls out concurrently:FetchPerGiverhelper runs the per-giver downstream calls in parallel (Task.WhenAll), bounded byMaxGiverConcurrency = 10via aSemaphoreSlimgate — speeds things up without overwhelming the downstream API.Per data type, ~N sequential round-trips collapse to ~⌈N/10⌉ batches (e.g. 50 subunits → ~5 batches instead of 50 chained calls).
Notes
MaxGiverConcurrency = 10is a conservative cap and is the knob to tune if the downstream API tolerates more.Testing
dotnet buildsucceeds.--filter FullyQualifiedName~DelegationExport).🤖 Generated with Claude Code
Summary by CodeRabbit