Skip to content

Refactor toolchains and runtimes - #3232

Merged
timcassell merged 4 commits into
masterfrom
refactor-toolchain-runtime
Sep 5, 2026
Merged

Refactor toolchains and runtimes#3232
timcassell merged 4 commits into
masterfrom
refactor-toolchain-runtime

Conversation

@timcassell

@timcassell timcassell commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Fixes #2609
Fixes #3036

Core changes:

  1. Runtime and Toolchain are now coupled in InfrastructureMode:
    • WithToolchain(...) sets the job's Runtime from the toolchain's target runtime.
    • WithRuntime(...) clears any explicitly set toolchain; the toolchain is then derived from the runtime when the benchmark is built.
    • Setting both is order-dependent (last assignment wins) — flagged by the new analyzer below.
  2. Runtimes are now future-proofed so users no longer need to rely on BDN updates to benchmark new .NET versions
    • Runtime has a Version property instead of RuntimeMoniker/MsBuildMoniker
    • RuntimeMoniker enum replaced with string constants + a real parser. Runtime.Parse(string)/Runtime.TryParse(...)
  3. One runtime type per VM/JIT/interpreter/AOT
    • WasmRuntime split into MonoWasmRuntime (interpreter/Jiterpreter), MonoWasmAotRuntime (AOT), and CoreWasmRuntime (CoreCLR-on-wasm, experimental).
    • MonoRuntime split out MonoCoreRuntime (.NET 6+ on the Mono VM), plus MonoAotRuntime (legacy Mono AOT).
  4. Toolchain construction is unified to a single pattern*Toolchain.From(runtime, settings), or From(settings) for toolchains with a fixed runtime.
  5. Settings moved out of runtimes into settings records.
  6. New toolchain naming pattern[BuildType][Runtime]Toolchain, which sets us up for future toolchains with different build types on the same runtimes (Source Generators based Toolchain #1770/[Feature Request] Support dotnet run benchmark.cs -c Release #2754)
  7. Same toolchain with different settings are now comparable and properties are added to the summary table.

Behavior changes:

  1. Job de-duplication is now value-based — jobs that previously compared equal by presentation but differ in runtime/toolchain/settings are kept separate.
  2. Setting both Runtime and Toolchain is last-wins (the coupling clears/overwrites accordingly); BDN1800 warns about it.
  3. Runtime.Name no longer includes the versionName is the family (e.g. .NET Core); ToString() appends the version (e.g. .NET Core 8.0).
  4. --cli <path> alone no longer creates a job — ignored if --runtimes and --corerun arguments are absent.

Breaking changes:

  1. RuntimeMoniker — the enum is replaced by string constants. [SimpleJob(RuntimeMoniker.Net80)] still compiles, but code that switched on the enum or used removed members (HostProcess, NotRecognized, MonoAOTLLVM*, WasmNet*) must change.
  2. Runtime.IsAOT and AotFilterAttribute removed — filter AOT-incompatible benchmarks with runtime is NativeAotRuntime instead.
  3. Runtime abstract members Name, Version, GetDefaultToolchain.
  4. --clrVersion removed (ClrRuntime.CreateForLocalFullNetFrameworkBuild, COMPLUS_Version) — the legacy private full-Framework-build feature is gone.
  5. --AOTCompilerMode removed.
  6. CustomRuntime and MonoAotLLVMRuntime removed.
  7. WasmRuntime/MonoRuntime split into the concrete types above; the MonoRuntime(name, path) ctor is gone — use WithToolchain(RoslynMonoToolchain.From(new MonoSettings { MonoPath = ... })).
  8. CoreRuntime factory changedCoreRuntime.CreateForNewVersion(msBuildMoniker, displayName) is replaced by CoreRuntime.From(Version version, string? platform = null).
  9. Equality contracts simplified — Dropped IEquatable<T> from runtimes and toolchains. Equality is still value-based via Equals(object).
  10. Settings objects changed from class with ctor with optional params to records with init properties
  11. Removed NativeAotToolchainBuilder.
  12. NativeAotSettings removed support for adding arbitrary nuget feeds
  13. Concrete toolchain classes renamed/split
    • NativeAotToolchain -> CsProjNativeAotToolchain
    • R2RToolchain -> CsProjR2RToolchain
    • MonoToolchain -> CsProjMonoCoreToolchain
    • WasmToolchain -> CsProjMonoWasmToolchain / CsProjMonoWasmAotToolchain / CsProjCoreWasmToolchain
    • MonoAotToolchain -> RoslynMonoAotToolchain
    • CsProjClassicNetToolchain -> CsProjFrameworkToolchain
    • CsProjCoreToolchain and settings namespace changed BenchmarkDotNet.Toolchains.CsProj -> BenchmarkDotNet.Toolchains.NetCoreApp
    • RoslynToolchain -> RoslynFrameworkToolchain / RoslynMonoToolchain
      • Roslyn.Builder/Roslyn.Generator -> RoslynBuilder/RoslynGenerator
  14. string paths changed to FileInfo/DirectoryInfo (on settings; ArtifactsPaths stays string).
  15. Build-component constructors take a settings objectCsProjGenerator (and its subclasses), DotNetCliBuilder, and DotNetCliPublisher now accept the toolchain's settings record (with the target framework moniker already resolved against the runtime) instead of separate targetFrameworkMoniker/cliPath parameters. DotNetCliGenerator exposes a single Settings property in place of TargetFrameworkMoniker/CliPath/PackagesPath.

Other Changes:

  1. Sealed concrete toolchain implementations instead of inheriting each other. Added intermediate abstract classes to reduce code duplication.
  2. Added UnknownRuntime instead of throwing when the current runtime is unrecognized.
  3. RuntimeFlavor enum and --wasmRuntimeFlavor removed.
  4. RuntimeValidator removed and replaced with BDN1800 analyzer rule and code fixer.
  5. New/expanded tests.
  6. Fixed and enabled MonoWasmAot tests.
  7. General dead code cleanup.

# Conflicts:
#	src/BenchmarkDotNet/Jobs/EnvironmentMode.cs
#	tests/BenchmarkDotNet.Tests/Perfonar/PerfonarTests.cs
@timcassell
timcassell merged commit fba7092 into master Sep 5, 2026
21 checks passed
@timcassell
timcassell deleted the refactor-toolchain-runtime branch September 5, 2026 10:12
timcassell added a commit that referenced this pull request Sep 5, 2026
Adapts the branch to the toolchain/runtime refactor (#3232):

- `InProcessEmitToolchain`'s constructor is now private behind the `From`
  factory; `AsyncEnumerableParamsSourceTests` updated accordingly.
- The branch's `9.0` LangVersion floor is re-applied in master's renamed
  `CsProjNativeAotGenerator`; `MonoAOTLLVMCsProj.txt` is gone with its toolchain.
- `BenchmarkCase` keeps both master's `GetToolchain()` and the branch's
  `DisposeAsync()`.
- `MockInProcessDiagnoser` is a new netstandard2.0 satellite that bound the
  branch's internal `System.Linq.AsyncEnumerable` polyfill, which is compiled out
  of BenchmarkDotNet's .NET 10 asset. It now iterates locally, matching
  `EtwProfiler` and `FinalizerBlockerDiagnoser`.

The analyzer resource files are restored to the repository's CRLF+BOM encoding.
The branch had rewritten both to LF, which made every line conflict.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant