dotnet test - allow to specify msbuild properties

[tapika]

At the moment if you run dotnet test myexample.csproj - by default it would try to build and run that test.
I however would like to pass extra msbuild parameters to project file, so I could mimic project presence in solution.

Normally you would like to set SolutionName and SolutionDir properties, but it's possible that even custom properties needs to be defined.

Command line syntax could be extended to be something like this:

dotnet test -pSolutioName=mySolution myexample.csproj

(Pure example, dev team can decide best approach)

[KalleOlaviNiemitalo]

If NuGet/Home#11408 for dotnet list package is also implemented, I'd like these to use the same option syntax.

[tapika]

Hmm... There exists a way to walkaround missing property setting. It's possible to pass property over environment variable, but of course such things as SolutionName gets overridden by msbuild itself. One approach is just to declare custom property name if it's not defined (via environment variable).

  <PropertyGroup Condition=" '$(MySolutionName)' == '' ">
    <MySolutionName>$(SolutionName)</MySolutionName>
  </PropertyGroup>

and then continue using MySolutionName everywhere.

And override that as environment variable from build system before it, for example:

Cake: Environment.SetEnvironmentVariable("MySolutionName", ...);
Batch: set MySolutionName=...

And so on...

[nohwnd]

This already works:

dotnet test .\mstest40.csproj -p:SolutionName=mySolution.sln -bl

[KalleOlaviNiemitalo]

I don't see -p listed in dotnet test --help though. Neither the --help output nor the dotnet test documentation says that unlisted MSBuild options can be used.

[KalleOlaviNiemitalo]

I didn't find any code that explicitly forwards unrecognized options of dotnet test to MSBuild. Do the options end up in result.GetValueForArgument(TestCommandParser.SlnOrProjectArgument) here? In which case, this might be working by accident.

sdk/src/Cli/dotnet/commands/dotnet-test/Program.cs

Lines 34 to 36 in 957ae5c

	msbuildArgs.AddRange(result.OptionValuesToBeForwarded(TestCommandParser.GetCommand()));
	msbuildArgs.AddRange(result.GetValueForArgument(TestCommandParser.SlnOrProjectArgument) ?? Array.Empty<string>());

[nohwnd]

It does not end up there. You can provide whatever properties you want including the other msbuild parameters (see me use -bl above to get binary msbuild log). dotnet test + csproj and dotnet test + sln (but not dotnet test + dll) will translate all the provided parameters they can recognize into msbuild properties and forward the rest to msbuild.

[baronfel]

A bit of context - you should be looking at the Run method in the dotnet-test\Program.cs file. These Run methods are invoked from the System.CommandLine Command Handlers in the matching Parser classes that generally are siblings of the Programs.

The key thing you want to look at is line 82, parseResults.GetArguments(). In System.CommandLine, this returns the entire arguments array for the command in raw form, and we forward those along to VSTest directly. The Parser configuration in TestCommandParser.cs is used primarily for fast help generation and tab-completion in the SDK's CLI itself. WE aim over time to encourage teams that we integrate with in the SDK to ship System.CommandLine Commands to us directly, so that those teams control help and tab-completion in their entirety. We're currently trialling this process with the dotnet/templating team (who contribute the dotnet new templating command) and hope to do more along these lines in the future.

[baronfel]

@KalleOlaviNiemitalo what you're seeing here is that the test help output isn't exhaustive - I would encourage any contributions to bring the SDK's definition in-line with that of the raw VSTest command, but due to the separation (and not having a Command from the VSTest library) this sort of thing is not uncommon. I've fixed a few for NuGet in a recent PR, for example.

[KalleOlaviNiemitalo]

The key thing you want to look at is line 82, parseResults.GetArguments().

The result of parseResults.GetArguments() goes to the args variable, which then is used only if (ContainsBuiltTestSources(args)). Do you mean ContainsBuiltTestSources returns true in dotnet test .\mstest40.csproj -p:SolutionName=mySolution.sln -bl?

[baronfel]

Figured it out - this is actually an unanticipated interaction of the current argument configuration of the parser. Line 36, where the SlnOrProjectArgument is added to the options to forward, is where the -p argument is added. This is...unanticipated...to say the least.

[KalleOlaviNiemitalo]

Well, that explains why it is not documented.