[EXPERIMENT] enumerate partition-mode split tests from source (skip closure compile)

The enumerate-tests step (~67s baseline, the second-biggest cost in
setup_for_tests after restore) is dominated by compiling the dependency
closures of the 4 SplitTestsOnCI projects, just to reflect their split metadata
off the built assemblies. Aspire.Hosting.Tests is the worst: it references the
hosting "god-edge" closure (~40 integration projects), so its closure compile
is the bulk of the step. This recurs for any PR whose selection includes
Aspire.Hosting.Tests, not only the full-matrix case.

Partition-mode discovery only needs the [Trait("Partition","<n>")] values, which
are compile-time string literals on the test classes. This change reads them
straight from source via eng/scripts/scan-test-partitions-from-source.ps1, so
the partition list is produced with no compile. build-test-matrix.ps1 already
appends an "uncollected:*" entry (--filter-not-trait "Partition=*"), so any
class the scan misses (or one with no trait) still runs.

The scan writes .tests-partitions.json only when it finds partitions; class-mode
projects (Aspire.Cli.EndToEnd.Tests, Aspire.Templates.Tests — no Partition
traits in source) leave the file absent and fall back to the original
Build + --list-tests path. GenerateTestPartitionsForCI is untouched, so the
AzDO/Helix prepare path that calls it directly is unaffected.

Verified locally: the scan emits collection:1..6 + uncollected:* for
Aspire.Hosting.Tests (identical to the compiled enumeration's 7 entries) and
writes nothing for the class-mode projects.

Caveat (experiment): a partition value written as a non-literal (const/computed)
would be missed by the source scan and, because it still carries a Partition
trait, would not fall under uncollected:* — those tests would be dropped. All
current Aspire.Hosting.Tests partitions are string literals, so this does not
occur today; productionizing should add a CI parity check vs the compiled list.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: radical

eng/TestEnumerationRunsheetBuilder/TestEnumerationRunsheetBuilder.targets

@@ -42,11 +42,39 @@
   <Target Name="RunTests"
           Outputs="%(TestToRun.ResultsStdOutPath)"
           Condition="'$(_ShouldSkipProject)' != 'true' and '$(IsGitHubActionsRunner)' == 'true' and '$(RunOnGithubActions)' == 'true'">
-    <!-- Build and generate test partitions for split test projects -->
+    <!-- [EXPERIMENT] Split-test partition discovery without compiling the closure.
+
+         For SplitTestsOnCI projects we first try to read [Trait("Partition","<n>")]
+         literals straight from source (no build). Partition-mode projects such as
+         Aspire.Hosting.Tests reference the whole hosting "god-edge" closure (~40
+         integrations); building that closure just to reflect partition traits off
+         the test DLL is the bulk of the enumerate-tests step. The trait values are
+         compile-time string literals, so a source scan yields the identical set,
+         and build-test-matrix.ps1 always appends an "uncollected:*" entry (which
+         maps to filter-not-trait Partition=*) so any class the scan misses runs.
+
+         The scan writes the .tests-partitions.json only when it finds partitions.
+         If none are found (class-mode projects like Aspire.Cli.EndToEnd.Tests /
+         Aspire.Templates.Tests, which enumerate classes against the built assembly),
+         the file is absent and we fall back to the original Build + class discovery.
+         GenerateTestPartitionsForCI itself is untouched, so the AzDO/Helix prepare
+         path that calls it directly is unaffected. -->
+    <PropertyGroup Condition="'$(SplitTestsOnCI)' == 'true'">
+      <_SourcePartitionScanScript>$(MSBuildThisFileDirectory)../scripts/scan-test-partitions-from-source.ps1</_SourcePartitionScanScript>
+      <_SourcePartitionsJsonPath>$([MSBuild]::NormalizePath('$(RepoRoot)', '$(TestArchiveTestsDir)$(MSBuildProjectName).tests-partitions.json'))</_SourcePartitionsJsonPath>
+    </PropertyGroup>
+
+    <Exec Condition="'$(SplitTestsOnCI)' == 'true'"
+          Command="pwsh -NoProfile -ExecutionPolicy Bypass -File &quot;$(_SourcePartitionScanScript)&quot; -ProjectDirectory &quot;$(MSBuildProjectDirectory)&quot; -TestPartitionsJsonFile &quot;$(_SourcePartitionsJsonPath)&quot;"
+          IgnoreExitCode="false"
+          WorkingDirectory="$(RepoRoot)" />
+
+    <!-- Class-mode fallback (and any partition-mode project where the scan found nothing):
+         build the project + its closure and discover via the original DLL-based path. -->
     <MSBuild Projects="$(MSBuildProjectFullPath)"
              Properties="GenerateCIPartitions=true;BuildOs=$(BuildOs)"
              Targets="Build;GenerateTestPartitionsForCI"
-             Condition="'$(SplitTestsOnCI)' == 'true'" />
+             Condition="'$(SplitTestsOnCI)' == 'true' and !Exists('$(_SourcePartitionsJsonPath)')" />
 
     <!-- Write .tests-metadata.json file -->
     <PropertyGroup>

eng/scripts/scan-test-partitions-from-source.ps1

@@ -0,0 +1,116 @@
+<#
+.SYNOPSIS
+  Discovers test partitions for CI splitting by scanning SOURCE (.cs) files,
+  without building the test project or its dependency closure.
+
+.DESCRIPTION
+  The compiled enumeration path (split-test-projects-for-ci.ps1 +
+  ExtractTestPartitions) reads [Trait("Partition", "<name>")] attributes from
+  the built test assembly. Building a partition-mode project such as
+  Aspire.Hosting.Tests forces a compile of its entire ProjectReference closure
+  (the hosting "god-edge": ~40 integrations), which dominates the
+  enumerate-tests step on CI.
+
+  Partition trait values are plain string literals on the test classes, e.g.:
+
+      [Trait("Partition", "5")]
+      public class WithUrlsTests { ... }
+
+  so they can be read directly from source with no compile. This script scans
+  the project directory for those literals and emits the same
+  .tests-partitions.json shape the compiled path produces:
+
+      { "testPartitions": ["collection:1", ..., "collection:6", "uncollected:*"] }
+
+  build-test-matrix.ps1 turns each "collection:N" into
+  --filter-trait "Partition=N" and "uncollected:*" into
+  --filter-not-trait "Partition=*", so the "uncollected" entry runs every test
+  that lacks a Partition trait. That backstop means a class missed by the scan
+  (or one with no trait) is never dropped — it still runs under "uncollected".
+
+  Class-mode projects (no Partition traits in source) are NOT handled here: the
+  caller falls back to the build + --list-tests path, because class lists have
+  no equivalent always-runs backstop and must match the runtime exactly.
+
+.PARAMETER ProjectDirectory
+  Directory of the test project to scan (recursively) for partition traits.
+
+.PARAMETER TestPartitionsJsonFile
+  Path to write the .tests-partitions.json output file. Only written when at
+  least one partition trait is found (so the caller can detect class mode by
+  the file's absence).
+
+.NOTES
+  PowerShell 7+. Exit code is always 0; "no partitions found" is signalled by
+  not writing the output file, not by failure.
+#>
+
+[CmdletBinding()]
+param(
+  [Parameter(Mandatory = $true)]
+  [string]$ProjectDirectory,
+
+  [Parameter(Mandatory = $true)]
+  [string]$TestPartitionsJsonFile
+)
+
+$ErrorActionPreference = 'Stop'
+Set-StrictMode -Version Latest
+
+if (-not (Test-Path $ProjectDirectory)) {
+  Write-Error "ProjectDirectory not found: $ProjectDirectory"
+}
+
+# Matches [Trait("Partition", "<value>")] allowing arbitrary inner whitespace.
+# Examples matched:
+#   [Trait("Partition", "5")]
+#   [Trait( "Partition" , "BasicTests" )]
+# The value is a string literal (xunit trait values are compile-time constants),
+# so a source scan captures every partition the compiled assembly would expose.
+$partitionRegex = [regex]'\[\s*Trait\s*\(\s*"Partition"\s*,\s*"([^"]+)"\s*\)\s*\]'
+
+$partitions = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::OrdinalIgnoreCase)
+
+# Skip obj/bin so generated/copied sources don't introduce phantom partitions.
+$sourceFiles = Get-ChildItem -Path $ProjectDirectory -Recurse -File -Filter '*.cs' |
+  Where-Object { $_.FullName -notmatch '[\\/](obj|bin)[\\/]' }
+
+foreach ($file in $sourceFiles) {
+  $content = Get-Content -Raw -LiteralPath $file.FullName
+  foreach ($m in $partitionRegex.Matches($content)) {
+    $value = $m.Groups[1].Value.Trim()
+    if (-not [string]::IsNullOrWhiteSpace($value)) {
+      [void]$partitions.Add($value)
+    }
+  }
+}
+
+if ($partitions.Count -eq 0) {
+  Write-Host "No [Trait(`"Partition`", ...)] attributes found in source under '$ProjectDirectory'. Falling back to class-mode (build + --list-tests)."
+  # Remove any stale output so the caller's !Exists() class-mode fallback is not
+  # fooled by a partitions file left over from a previous build.
+  if (Test-Path $TestPartitionsJsonFile) {
+    Remove-Item -LiteralPath $TestPartitionsJsonFile -Force -ErrorAction SilentlyContinue
+  }
+  exit 0
+}
+
+$lines = [System.Collections.Generic.List[string]]::new()
+foreach ($p in ($partitions | Sort-Object)) {
+  $lines.Add("collection:$p")
+}
+# Always include the uncollected backstop so tests without a Partition trait still run.
+$lines.Add("uncollected:*")
+
+$outputDir = Split-Path -Parent $TestPartitionsJsonFile
+if (-not [string]::IsNullOrEmpty($outputDir) -and -not (Test-Path $outputDir)) {
+  New-Item -ItemType Directory -Path $outputDir -Force | Out-Null
+}
+
+$testPartitionsJson = @{}
+$testPartitionsJson | Add-Member -Force -MemberType NoteProperty -Name 'testPartitions' -Value @($lines)
+$testPartitionsJson | ConvertTo-Json -Depth 20 | Set-Content -Path $TestPartitionsJsonFile -Encoding UTF8
+
+Write-Host "Source partition scan found $($partitions.Count) partition(s): $(( $partitions | Sort-Object ) -join ', ')"
+Write-Host "Wrote: $TestPartitionsJsonFile"
+exit 0