[PERF] Performance Wasm AOT jobs are failing due to Dia2Lib issue

[LoopedBard3]

Description

Performance Wasm AOT jobs are failing due to Dia2Lib issue. WasmApp.Common.targets(681,5): error : Could not load signature of Dia2Lib.DiaStackWalkerClass:getEnumFrames2 due to: Could not resolve type with token 0100000f from typeref (expected class 'System.Enum' in assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089') assembly:mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 type:System.Enum member:(null)

FYI: @ilonatommy

Reproduction Steps

(Tested in WSL Ubuntu 22.4.3 LTS)
Clone performance repo
Install a v8 wasm engine with jsvu: jsvu project page (Take note of the install location)
cd into performance/scripts
run (replace path): sudo python3 benchmarks_local.py --commits b29efed9995da044d97f9d3f97bc3d9de80a2050 --run-types WasmAOT --filter *Span.IndexerBench.CoveredIndex2* *WriteReadAsync* --wasm-engine-path /home/<username>/.jsvu/engines/v8/v8

Expected behavior

Successful BDN run with results

Actual behavior

Full log available: https://dev.azure.com/dnceng/internal/_build/results?buildId=2386986&view=results under '...wasm aot...' failing job. Job is failing with:

[2024/02/24 12:14:11][INFO]   [Dia2Lib.dll] Exec (with response file contents expanded) in /home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in: MONO_PATH=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in:/home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NETCore.App.Runtime.Mono.browser-wasm/9.0.0-ci/runtimes/browser-wasm/native/:/home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NETCore.App.Runtime.Mono.browser-wasm/9.0.0-ci/runtimes/browser-wasm/lib/net9.0 MONO_ENV_OPTIONS= /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.linux-x64.Cross.browser-wasm/9.0.0-ci/Sdk/../tools/mono-aot-cross --wasm-exceptions --debug --llvm "--aot=no-opt,static,direct-icalls,deterministic,mattr=simd,dwarfdebug,llvm-path=/home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.Emscripten.3.1.34.Sdk.linux-x64/9.0.0-preview.2.24121.1/tools/bin/,static,dedup-skip,trimming-eligible-methods-outfile=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/tokens/Dia2Lib_dll_compiled_methods.txt,llvmonly,interp,asmonly,llvm-outfile=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/Dia2Lib.dll.bc.tmp,temp-path=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/Dia2Lib.dll.tmp" "Dia2Lib.dll"
[2024/02/24 12:14:11][INFO] 
[2024/02/24 12:14:11][INFO] /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(681,5): error : Precompiling failed for /home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in/Dia2Lib.dll with exit code 139. [/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-UGTVKY/BenchmarkDotNet.Autogenerated.csproj]
[2024/02/24 12:14:11][INFO] /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(681,5): error : Mono Ahead of Time compiler - compiling assembly /home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in/Dia2Lib.dll [/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-UGTVKY/BenchmarkDotNet.Autogenerated.csproj]
[2024/02/24 12:14:11][INFO] /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(681,5): error : Could not load signature of Dia2Lib.DiaStackWalkerClass:getEnumFrames2 due to: Could not resolve type with token 0100000f from typeref (expected class 'System.Enum' in assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089') assembly:mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 type:System.Enum member:(null) [/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-UGTVKY/BenchmarkDotNet.Autogenerated.csproj]

Regression?

No response

Known Workarounds

No response

Configuration

Dotnet/runtime hash: f32c428

 ----------------------------------------------
 Initializing logger 2024-02-24 12:13:13.950464
 ----------------------------------------------
 $ dotnet --info
 .NET SDK:
  Version:           9.0.100-preview.2.24116.21
  Commit:            e6b3ff2dff
  Workload version:  9.0.100-manifests.325f9588
 
 Runtime Environment:
  OS Name:     ubuntu
  OS Version:  22.04
  OS Platform: Linux
  RID:         linux-x64
  Base Path:   /home/helixbot/work/AF97099E/p/dotnet/sdk/9.0.100-preview.2.24116.21/
 
 .NET workloads installed:
  [wasm-experimental]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.current/WorkloadManifest.json
    Install Type:        FileBased
 
  [wasm-tools]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.current/WorkloadManifest.json
    Install Type:        FileBased
 
  [wasm-tools-net8]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.net8/WorkloadManifest.json
    Install Type:        FileBased
 
  [wasm-experimental-net8]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.net8/WorkloadManifest.json
    Install Type:        FileBased
 
 
 Host:
   Version:      9.0.0-preview.2.24115.1
   Architecture: x64
   Commit:       963626276e
 
 .NET SDKs installed:
   9.0.100-preview.2.24116.21 [/home/helixbot/work/AF97099E/p/dotnet/sdk]
 
 .NET runtimes installed:
   Microsoft.AspNetCore.App 9.0.0-preview.2.24114.9 [/home/helixbot/work/AF97099E/p/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.NETCore.App 8.0.3 [/home/helixbot/work/AF97099E/p/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 9.0.0-preview.2.24115.1 [/home/helixbot/work/AF97099E/p/dotnet/shared/Microsoft.NETCore.App]
 
 Other architectures found:
   None
 
 Environment variables:
   DOTNET_ROOT       [/home/helixbot/work/AF97099E/p/dotnet/]
 
 global.json file:
   Not found
 
 Learn more:
   https://aka.ms/dotnet/info
 
 Download .NET:
   https://aka.ms/dotnet/download

Other information

No response

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Performance Wasm AOT jobs are failing due to Dia2Lib issue. WasmApp.Common.targets(681,5): error : Could not load signature of Dia2Lib.DiaStackWalkerClass:getEnumFrames2 due to: Could not resolve type with token 0100000f from typeref (expected class 'System.Enum' in assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089') assembly:mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 type:System.Enum member:(null)

FYI: @ilonatommy

Reproduction Steps

(Tested in WSL Ubuntu 22.4.3 LTS)
Clone performance repo
Install a v8 wasm engine with jsvu: jsvu project page (Take note of the install location)
cd into performance/scripts
run (replace path): sudo python3 benchmarks_local.py --commits b29efed9995da044d97f9d3f97bc3d9de80a2050 --run-types WasmAOT --filter *Span.IndexerBench.CoveredIndex2* *WriteReadAsync* --wasm-engine-path /home/<username>/.jsvu/engines/v8/v8

Expected behavior

Successful BDN run with results

Actual behavior

Full log available: https://dev.azure.com/dnceng/internal/_build/results?buildId=2386986&view=results under '...wasm aot...' failing job. Job is failing with:

[2024/02/24 12:14:11][INFO]   [Dia2Lib.dll] Exec (with response file contents expanded) in /home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in: MONO_PATH=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in:/home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NETCore.App.Runtime.Mono.browser-wasm/9.0.0-ci/runtimes/browser-wasm/native/:/home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NETCore.App.Runtime.Mono.browser-wasm/9.0.0-ci/runtimes/browser-wasm/lib/net9.0 MONO_ENV_OPTIONS= /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.linux-x64.Cross.browser-wasm/9.0.0-ci/Sdk/../tools/mono-aot-cross --wasm-exceptions --debug --llvm "--aot=no-opt,static,direct-icalls,deterministic,mattr=simd,dwarfdebug,llvm-path=/home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.Emscripten.3.1.34.Sdk.linux-x64/9.0.0-preview.2.24121.1/tools/bin/,static,dedup-skip,trimming-eligible-methods-outfile=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/tokens/Dia2Lib_dll_compiled_methods.txt,llvmonly,interp,asmonly,llvm-outfile=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/Dia2Lib.dll.bc.tmp,temp-path=/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/Dia2Lib.dll.tmp" "Dia2Lib.dll"
[2024/02/24 12:14:11][INFO] 
[2024/02/24 12:14:11][INFO] /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(681,5): error : Precompiling failed for /home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in/Dia2Lib.dll with exit code 139. [/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-UGTVKY/BenchmarkDotNet.Autogenerated.csproj]
[2024/02/24 12:14:11][INFO] /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(681,5): error : Mono Ahead of Time compiler - compiling assembly /home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm/wasm/for-publish/aot-in/Dia2Lib.dll [/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-UGTVKY/BenchmarkDotNet.Autogenerated.csproj]
[2024/02/24 12:14:11][INFO] /home/helixbot/work/AF97099E/p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(681,5): error : Could not load signature of Dia2Lib.DiaStackWalkerClass:getEnumFrames2 due to: Could not resolve type with token 0100000f from typeref (expected class 'System.Enum' in assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089') assembly:mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 type:System.Enum member:(null) [/home/helixbot/work/AF97099E/w/AF580949/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-UGTVKY/BenchmarkDotNet.Autogenerated.csproj]

Regression?

No response

Known Workarounds

No response

Configuration

Dotnet/runtime hash: f32c428

 ----------------------------------------------
 Initializing logger 2024-02-24 12:13:13.950464
 ----------------------------------------------
 $ dotnet --info
 .NET SDK:
  Version:           9.0.100-preview.2.24116.21
  Commit:            e6b3ff2dff
  Workload version:  9.0.100-manifests.325f9588
 
 Runtime Environment:
  OS Name:     ubuntu
  OS Version:  22.04
  OS Platform: Linux
  RID:         linux-x64
  Base Path:   /home/helixbot/work/AF97099E/p/dotnet/sdk/9.0.100-preview.2.24116.21/
 
 .NET workloads installed:
  [wasm-experimental]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.current/WorkloadManifest.json
    Install Type:        FileBased
 
  [wasm-tools]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.current/WorkloadManifest.json
    Install Type:        FileBased
 
  [wasm-tools-net8]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.net8/WorkloadManifest.json
    Install Type:        FileBased
 
  [wasm-experimental-net8]
    Installation Source: SDK 9.0.100-preview.2
    Manifest Version:    9.0.0-ci/9.0.100-preview.2
    Manifest Path:       /home/helixbot/work/AF97099E/p/dotnet/sdk-manifests/9.0.100-preview.2/microsoft.net.workload.mono.toolchain.net8/WorkloadManifest.json
    Install Type:        FileBased
 
 
 Host:
   Version:      9.0.0-preview.2.24115.1
   Architecture: x64
   Commit:       963626276e
 
 .NET SDKs installed:
   9.0.100-preview.2.24116.21 [/home/helixbot/work/AF97099E/p/dotnet/sdk]
 
 .NET runtimes installed:
   Microsoft.AspNetCore.App 9.0.0-preview.2.24114.9 [/home/helixbot/work/AF97099E/p/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.NETCore.App 8.0.3 [/home/helixbot/work/AF97099E/p/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 9.0.0-preview.2.24115.1 [/home/helixbot/work/AF97099E/p/dotnet/shared/Microsoft.NETCore.App]
 
 Other architectures found:
   None
 
 Environment variables:
   DOTNET_ROOT       [/home/helixbot/work/AF97099E/p/dotnet/]
 
 global.json file:
   Not found
 
 Learn more:
   https://aka.ms/dotnet/info
 
 Download .NET:
   https://aka.ms/dotnet/download

Other information

No response

Author:	LoopedBard3
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[ilonatommy]

@kg, @vargaz it cannot find a method from Dia2Lib.dll in AOT.

[vargaz]

Should Dia2Lib.dll reference mscorlib ?

.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 4:0:0:0
}

[vargaz]

The PR referenced above will fix the aot compiler crash, but it will still error out. The Dia2Lib.dll referenced by the build appears to be built against mscorlib, and not System.Private.Corelib.

[LoopedBard3]

The PR referenced above will fix the aot compiler crash, but it will still error out. The Dia2Lib.dll referenced by the build appears to be built against mscorlib, and not System.Private.Corelib.

Is the next step to get the correct reference something that will be corrected somewhere in dotnet/runtime or do we need to look into BenchmarkDotNet more likely?

[vargaz]

The mscorlib.dll used by the build doesn't have the type forwarder for System.Enum. Maybe its a linker issue ?

[LoopedBard3]

@sbomer does this seem like a potential linker issue, or where a next good place to start looking may be?

[sbomer]

It very well could be. I will try to repro and investigate this tomorrow, or if someone else with a repro could share the linker inputs that would be helpful.

[LoopedBard3]

Is there a specific line/program I should look for to find the linker inputs? Otherwise, there is a link to a run with the job which has the full logs under the wasm aot job.

[sbomer]

Yes, the linker invocation would be running illink.dll, normally through an MSBuild task ILLink from the _RunILLink target.

[vargaz]

Some observations:

• after the failing build, the dotnet project can be found under artifacts/bin/MicroBenchmarks/Release/net9.0/<...>
• it can be built locally in that dir by doing:
  $REPO/scripts/runtime-testing-artifacts/dotnet/dotnet build -c Release --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:NuGetPackageRoot="$REPO/artifacts/packages"
  where REPO is the path to the perf repo
• after the build, the artifacts are in $REPO/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/net9.0/browser-wasm
• it looks like the 'linked' directory doesn't contain the Dia2Lib.dll, meaning the linker has linked it away
• but the the 'aot-in' directory contains an unlinked version of Dia2Lib.dll, which means the build is trying to AOT it, which leads to the error since types it depends on were linked away