Failed to run benchmarks on .net 5 RC1 and Preview LangVersion

[trampster]

When running benchmarks from a .net 5 RC1 console project I get the following error:

// Build Error: Standard output:

Standard error:
/home/daniel/Work/JsonSrcGen/Benchmarks/bin/Release/net5.0/f332688d-f4ca-426b-95f1-b4c8df64944f/BenchmarkDotNet.Autogenerated.csproj(20,18): error MSB4086: A numeric comparison was attempted on "$(LangVersion)" that evaluates to "preview" instead of a number, in condition "'$(LangVersion)' == '' Or '$(LangVersion)' < '7.3'".

// Build Error: Standard output:

Standard error:
/home/daniel/Work/JsonSrcGen/Benchmarks/bin/Release/net5.0/f332688d-f4ca-426b-95f1-b4c8df64944f/BenchmarkDotNet.Autogenerated.csproj(20,18): error MSB4086: A numeric comparison was attempted on "$(LangVersion)" that evaluates to "preview" instead of a number, in condition "'$(LangVersion)' == '' Or '$(LangVersion)' < '7.3'".

I'm running benchmarks for a .net 5 c# source generator, this requires that the LangVersion be set to Preview

[adamsitnik]

This is a duplicate of #1452 which has been solved in #1420 but just not released to NuGet yet.

To use it you need to add our CI feed to your nuget.config file and update to the latest version:

<packageSources>
  <add key="bdn-ci" value="https://ci.appveyor.com/nuget/benchmarkdotnet" />
</packageSources>

[adamsitnik]

I updated to 0.12.0.1228 but got the following error instead

Please try 0.12.1.1420

[trampster]

With 0.12.1.1420 I get the following

/snap/dotnet-sdk/100/sdk/5.0.100-rc.1.20452.10/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(241,5): error NETSDK1005: Assets file '/home/daniel/Work/JsonSrcGen/Benchmarks/obj/project.assets.json' doesn't have a target for 'net5.0'. Ensure that restore has run and that you have included 'net5.0' in the TargetFrameworks for your project. [/home/daniel/Work/JsonSrcGen/Benchmarks/Benchmarks.csproj]

The build failed. Fix the build errors and run again.

I'm using the linux snap package which requires dotnet to be run with a prefix:

dotnet-sdk.dotnet run -c Release

It looks like maybe BenchmarkDotNet isn't doing this

[adamsitnik]

Ensure that restore has run and that you have included 'net5.0' in the TargetFrameworks for your project. [/home/daniel/Work/JsonSrcGen/Benchmarks/Benchmarks.csproj]

What TargetFrameworks is the [/home/daniel/Work/JsonSrcGen/Benchmarks/Benchmarks.csproj targetting?

Are you using netcoreapp5.0 or net5.0 moniker?

[trampster]

I'm using the net5.0 moniker.

I just switched from the snap package to the deb packet (which doesn't need the dotnet-sdk prefix) and it works.

The snap package uses the prefix so that you can have the deb installed at the stable version and then have the snap running the previews/RC.

BenchmarkDotNet is trying to run the code using dotnet instead of with dotnet-sdk.dotnet, so what is happening is it was trying to use the dotnet core 3 installed by my deb package. Which doesn't know about net5.0.

So by moving to the deb package I've lost the ability to run dotnet core 3. But for now I can live with that.

[adamsitnik]

So by moving to the deb package I've lost the ability to run dotnet core 3.

Why is that? If you install .NET 5 SDK it just adds .NET 5 Runtime to the list of installed runtimes and it should be able to build and run the previous versions of .NET Core (it's not previous runtimes, just adding new one and upgrading SDK).

Is there any way to detect that the user is using a snap package? We could add support for it, but I would need some help with that.

[trampster]

I'm guessing you are not a linux user.

The deb package replace previous packages, side by side installs are not a thing.

Using the snap package is the preferred way of running previews and release candidates and Benchmark.net doesn't support them.

[adamsitnik]

I'm guessing you are not a linux user.

I am a Linux newbie ;)

The deb package replace previous packages, side by side installs are not a thing.

Thanks for the explanation!

So now it's about detecting that the user has started the process via dotnet-sdk.dotnet and using it for building the generated project.