Test asset NuGet restore: fragile resolution of Microsoft.Testing.Platform via transitive CodeCoverage prerelease (NU1102)

[Evangelink]

Summary

Test assets using <PackageReference Include="MSTest" Version="$MSTestVersion$" /> (≈20 csprojs in test/IntegrationTests/) can fail to restore with NU1102 when the local-build chain demands a Microsoft.Testing.Platform prerelease that isn't in any of the test-asset's configured feeds.

Concrete repro: see the MSTestJUnitReportMTPRetry asset in test/IntegrationTests/MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs — failed during PR #9029 CI runs and several previous builds.

Failure chain

1. Test-asset csproj: <PackageReference Include="MSTest" Version="$MSTestVersion$" /> → resolves to local-built MSTest 4.3.0-ci.
2. MSTest 4.3.0-ci's packed nuspec hardcodes:

   <dependency id="Microsoft.Testing.Extensions.CodeCoverage" version="18.9.0-preview.26309.4" />

   (Pinned via eng/Versions.props → MicrosoftTestingExtensionsCodeCoverageVersion.)
3. Microsoft.Testing.Extensions.CodeCoverage 18.9.0-preview.26309.4 lives only on the dotnet-tools feed. Its nuspec transitively demands:

   <dependency id="Microsoft.Testing.Platform" version="2.3.0-preview.26308.7" />

4. Test-asset's NuGet.config (TestAsset.GetNuGetConfig) lists 6 feeds — none includes dotnet-tools. The closest matches:
   • local-shipping has MTP 2.3.0-ci — sorts below 2.3.0-preview.26308.7 in SemVer 2.0 (c < p).
   • dotnet-public maxes out at MTP 2.2.3.
5. NuGet emits NU1102 for MTP. The error misleadingly points at the local feed instead of the actual upstream constraint.

Root cause

Two-part design weakness in the test-asset infrastructure:

1. No exact-match pinning: Version="$X$" is interpreted by NuGet as [$X$, ) — open-ended minimum. Drift to higher prereleases from any configured feed is technically allowed.
2. No local-folder priority: packageSourceMapping in the generated NuGet.config uses <package pattern="*"/> for every feed — no priority for local. There is no isolation of testfx-family packages.

Combined with MSTest's packed nuspec hardcoding a CodeCoverage prerelease that only exists on dotnet-tools, this creates a fragile resolution chain that explodes for any test asset using the MSTest meta package.

Proposed fixes (mix and match)

A. Add dotnet-tools to TestAsset's default NuGet.config

Smallest mechanical change; mirrors the repo's top-level NuGet.config. Restores parity with the main build's restore chain.

 <add key="local-tmp-packages" value="..." />
 <add key="dotnet-public" value="..." />
+<add key="dotnet-tools" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" />
 <add key="dotnet10" value="..." />
 <add key="dotnet11" value="..." />

B. Exact-match pinning in test-asset csprojs

Use Version="[$X$]" everywhere. Hardens against version drift on direct PackageReferences.

- <PackageReference Include="MSTest" Version="$MSTestVersion$" />
+ <PackageReference Include="MSTest" Version="[$MSTestVersion$]" />

C. Per-package packageSourceMapping

Force testfx-family packages to come only from local feeds (and dotnet-tools for CodeCoverage previews):

<packageSource key="local-shipping">
    <package pattern="Microsoft.Testing.*" />
    <package pattern="MSTest*" />
</packageSource>
<packageSource key="dotnet-tools">
    <package pattern="Microsoft.Testing.Extensions.CodeCoverage" />
</packageSource>
<packageSource key="dotnet-public">
    <package pattern="*" />
</packageSource>

D. Downgrade CodeCoverage dep in MSTest's packed nuspec to nearest stable

Cleanest at the source — pin to 18.8.0 (stable, on dotnet-public) for the packed metadata while still consuming the preview at build time. Requires coordinating with the VSTest CodeCoverage release train and verifying no behavioral regression.

Recommended path

I'd suggest A + B + C combined:

• A restores feed parity (smallest change, instant relief).
• B prevents direct-ref drift (general hardening).
• C makes future failures fail-fast with a clearer error message.

D is the right long-term solution but is a larger conversation about how MSTest's nuspec dependencies should be versioned (always-stable vs. always-preview-matched).

Related

• PR Fix telemetry tests for new default diagnostic file name shape #9029 (where this surfaced) — telemetry fixes shipped; this asset failure was a separate pre-existing issue. See comment for context.
• Same failure mode visible on main build 741048f7 (internal AzDO build 2996915).

[Evangelink]

Refined understanding (after @Evangelink's follow-up)

The original "exact match + local-folder priority" framing isn't enough on its own. Key constraint: 2.3.0-ci sorts below 2.3.0-preview.X per SemVer 2.0 (c (99) < p (112) lexically). So even if we force local feeds to win, NuGet still cannot satisfy a transitive >= 2.3.0-preview.26308.7 constraint with 2.3.0-ci — it would emit NU1605 (package downgrade) instead of NU1102. Same end state: red build, slightly different error code.

Refined recommendation: explicit preview MTP + dotnet-tools feed

For test assets that pull preview transitives (via MSTest meta → Microsoft.Testing.Extensions.CodeCoverage <preview> → MTP <preview>), the asset must explicitly own the preview MTP version:

<ItemGroup>
  <!-- Explicit preview MTP to satisfy the CodeCoverage transitive demand. -->
  <PackageReference Include="Microsoft.Testing.Platform" Version="$MicrosoftTestingPlatformPreviewVersion$" />
  <PackageReference Include="MSTest" Version="$MSTestVersion$" />
  <PackageReference Include="Microsoft.Testing.Extensions.JUnitReport" Version="$MicrosoftTestingExtensionsJUnitReportVersion$" />
  <PackageReference Include="Microsoft.Testing.Extensions.Retry" Version="$MicrosoftTestingPlatformVersion$" />
</ItemGroup>

Combined with adding dotnet-tools to the test-asset NuGet.config (or scoping the preview MTP/CodeCoverage to dotnet-tools via packageSourceMapping).

What $MicrosoftTestingPlatformPreviewVersion$ should resolve to

The matching preview version is what Microsoft.Testing.Extensions.CodeCoverage <pinned> (eng/Versions.props's MicrosoftTestingExtensionsCodeCoverageVersion) transitively demands. As of main today: CodeCoverage 18.9.0-preview.26309.4 → MTP 2.3.0-preview.26308.7.

Options for sourcing this version into the test asset:

• (a) Add a new property to eng/Versions.props (MicrosoftTestingPlatformPreviewVersion) kept in sync with what CodeCoverage demands. Flow it through AcceptanceTestBase like the other versions.
• (b) Extract it at runtime by reading the CodeCoverage nuspec from the NuGet cache and parsing its MTP dep.
• (c) Compile-time inject it from Microsoft.Testing.Extensions.CodeCoverage nuspec at pack time.

(a) is the most pragmatic and consistent with the existing pattern.

Updated fix list

#	Fix	Notes
A	Add dotnet-tools to TestAsset's default NuGet.config	Required for preview MTP/CodeCoverage to be restorable. Smallest mechanical change.
B	Explicit <PackageReference Include="Microsoft.Testing.Platform" Version="$preview$" /> in affected assets	Recommended — makes the preview-MTP dependency explicit and avoids the NU1605 downgrade trap.
C	Per-package packageSourceMapping (testfx-family → local feeds + dotnet-tools)	Optional hardening — prevents accidental drift to public previews of testfx packages.
D	Exact-match pinning (Version="[X]") on direct refs	General hardening — doesn't fix this specific bug.
E	Downgrade CodeCoverage dep in MSTest's packed nuspec to stable 18.8.0	Cleanest long-term but requires VSTest CodeCoverage release-train coordination.

Recommended path

A + B is the minimum viable fix:

1. Add dotnet-tools to TestAsset.GetNuGetConfig so the preview MTP/CodeCoverage is restorable.
2. Add an explicit Microsoft.Testing.Platform PackageReference (at the matching preview version) to test assets that use the MSTest meta package.

C can be layered on for defense-in-depth.

E is the right long-term fix at the MSTest pack-time level but doesn't need to block A+B.

[Evangelink]

PR for Fix A (add dotnet-tools feed to test-asset NuGet.config) opened: #9032