ci: enable parallel integration test execution#3506
Merged
tippmar-nr merged 11 commits intomainfrom Mar 27, 2026
Merged
Conversation
… apps
Pre-publish all .NET Core test apps in the CI build jobs so test
fixtures can copy pre-built output instead of invoking dotnet publish
at runtime. This eliminates file-lock collisions on shared dependency
obj/ directories (e.g., ApplicationLifecycle.dll) when multiple
fixtures publish in parallel.
Changes:
- RemoteService.cs: Add TryCopyPrebuiltPublishOutput() that copies from
pre-published bin/Release/{TFM}/{RID}/publish/ when
NR_DOTNET_TEST_PREBUILT_APPS=1 is set, falling back to dotnet publish
- all_solutions.yml: Add pre-publish step in both build jobs, set env
var in both run jobs, replace hard-coded parallelization disable with
workflow input (default: enabled)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Filter to projects with <OutputType>Exe</OutputType> (skip libraries) - Drop --no-restore (RID-specific publish needs its own restore) - Suppress dotnet error output to prevent ##[error] annotations from failing the step on non-fatal publish failures - Add success/failure counters for visibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ng Exe ASP.NET Core web apps don't have explicit <OutputType>Exe</OutputType> (the Web SDK infers it), so the previous filter skipped them. Now we skip only explicit <OutputType>Library</OutputType> projects, catching web apps and console apps alike. Also: show error lines on failure instead of suppressing all output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…NServiceBus5 - DotnetTool.cs: Add static lock per package name to serialize concurrent dotnet tool install calls that race on the shared NuGet package cache - all_solutions.yml: Disable parallelization for MsSql and NServiceBus5 unbounded tests which use shared external infrastructure with hard-coded object names (e.g., NewRelic.dbo.TeamMembers, MSMQ queues) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Azure Functions Core Tools (func.exe) crashes (exit code -1) when multiple instances launch simultaneously, likely due to shared state in the storage emulator or lock files. Each failure then waits 180s for an agent log that never appears, cascading into ~15min of wasted time. Disable parallelism for this namespace until the contention is resolved. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
dotnet publish --runtime win-x64 creates large intermediate files in
bin/Release/{TFM}/win-x64/ alongside the publish/ subdirectory. Only
the publish/ output is needed at test runtime. Delete the intermediate
files before archiving to significantly reduce artifact size.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
func.exe start --no-build expects framework-dependent build output, not RID-specific publish output (missing local.settings.json, different runtimes/ structure). Azure Function tests already run serially so runtime dotnet publish won't cause contention. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add --no-self-contained to dotnet publish in both the CI pre-publish step and the runtime fallback in RemoteService.PublishWithDotnetExe. Framework-dependent publish is ~5MB vs ~60MB per app since it doesn't bundle the .NET runtime. CI runners have .NET 8 and 10 installed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3506 +/- ##
=======================================
Coverage 81.79% 81.80%
=======================================
Files 508 508
Lines 34220 34220
Branches 4040 4040
=======================================
+ Hits 27990 27993 +3
+ Misses 5265 5261 -4
- Partials 965 966 +1
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
jaffinito
approved these changes
Mar 26, 2026
nr-ahemsath
approved these changes
Mar 26, 2026
Member
nr-ahemsath
left a comment
nr-ahemsath
left a comment
There was a problem hiding this comment.
Approved. The changes to pre-publish .NET Core apps are worth integrating even if we end up disabling parallel execution (which is as easy as changing the default value of the new input).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR enables parallel xUnit test execution (within a single namespace) for the integration and unbounded tests.
The lynchpin to making this work is pre-publishing the .NET Core test apps so that test execution just copies the output instead of running
dotnet publishon every app. (.NET framework test apps are already built by MSBuild and didn't need any updates).Summary
Re-enable xUnit parallelization for integration tests, with a workflow input (default: enabled) that can be used to disable parallelization if needed.
Disable parallelization for certain namespaces with shared external resource contention:
AzureFunction,MsSql,NServiceBus,NServiceBus5- those will be revisited in a future PR to refactor the tests to enable parallelism where possible.Pre-publish .NET Core test apps in CI build jobs so fixtures copy pre-built output instead of running
dotnet publishat runtime, eliminating file-lock collisions on sharedobj/directoriesClean up intermediate runtime id-specific build artifacts after pre-publish to keep artifact size manageable
Exclude Azure Function apps from pre-publish (
func.exerequires framework-dependent build output, not runtime id-specific publish output)Add
DotnetToolinstall lock to prevent concurrent NuGet cache contention (Lambda tests)