[Feature Request] Support `dotnet run benchmark.cs -c Release`

[333fred]

.NET is introducing the ability to just run .cs files, as detailed in https://devblogs.microsoft.com/dotnet/announcing-dotnet-run-app/. It would be great if BDN could support this natively, so we can just write a one-off benchmark and run it without having to make a temp project.

[timcassell]

Currently we search for a csproj that matches the assembly name to build the generated benchmark project against. How would it work for cs file? Does the assembly name match the filename? If we <Compile Include="BenchmarkScript.cs">, does it just work, ignoring all the new script-focused # directives?

[DamianEdwards]

@timcassell yes the assembly name matches the file name, e.g. app.cs results in app.dll being run.

If you <Compile Include="BenchmarkScript.cs" /> in a project, it will be treated like any other .cs file, so can work depending on what else is included in the project (e.g. can't have multiple top-level statements files, etc.)

[timcassell]

If the script file uses top-level statements, and our benchmark project has a UniqueProgramName.Main that we set as the entry point using <StartupObject>BenchmarkDotNet.Autogenerated.UniqueProgramName</StartupObject>, do the top-level statements get ignored?

[333fred]

If we <Compile Include="BenchmarkScript.cs">, does it just work, ignoring all the new script-focused # directives?

Yes, *but

The but is important though: #:package BenchmarkDotNet@0.15.0 is going to produce an error in the program built with a csproj.

/cc @jaredpar and @jjonescz.

[333fred]

If the script file uses top-level statements, and our benchmark project has a UniqueProgramName.Main that we set as the entry point using <StartupObject>BenchmarkDotNet.Autogenerated.UniqueProgramName</StartupObject>, do the top-level statements get ignored?

Not sure why this would be relevant: I can use top-level statements with BDN today? Nothing changes in that aspect.

[timcassell]

If we <Compile Include="BenchmarkScript.cs">, does it just work, ignoring all the new script-focused # directives?

Yes, *but

The but is important though: #:package BenchmarkDotNet@0.15.0 is going to produce an error in the program built with a csproj.

That's a very big but. Do you have any suggestions to make it work?

[DamianEdwards]

I think BDN will need to recognize this app type and generate the project differently, or indeed generate another file-based app instead and run that.

[DamianEdwards]

Or perhaps it could do dotnet project convert app.cs to some other location and then run as usual. This might require us to support a mode on dotnet project convert that leaves the original files in place (right now it removes them).

[333fred]

Alternatively, we could consider making the #: error a warning instead? I'd like to hear @jaredpar and @jjonescz's thoughts on that as well.

[DamianEdwards]

Alternatively, we could consider making the #: error a warning instead? I'd like to hear @jaredpar and @jjonescz's thoughts on that as well.

That won't help much if the app requires those references though right?

[timcassell]

If the script file uses top-level statements, and our benchmark project has a UniqueProgramName.Main that we set as the entry point using <StartupObject>BenchmarkDotNet.Autogenerated.UniqueProgramName</StartupObject>, do the top-level statements get ignored?

Not sure why this would be relevant: I can use top-level statements with BDN today? Nothing changes in that aspect.

I think it's relevant because today they are in 2 separate projects, while compile include would make them 1 project, and I'm not 100% sure if the compiler is fine with that.

[333fred]

Yeah, that is fair; I usually just need BDN itself, but if there's some other 3rd party dependency I'm benchmarking it would fail. Likely the cleanest option would be to have BDN be able to do dotnet run app.cs, but I don't know what exactly BDN needs to customize here.

[333fred]

I think it's relevant because today they are in 2 separate projects, while compile include would make them 1 project, and I'm not 100% sure if the compiler is fine with that.

I see. Yeah, you'll get errors, TLS is going to override anything else.