Use Testing platform server mode breaks testing

[daryllabar]

Describe the bug

Upon updating to the latest version of VS, my test explorer has stopped running tests.

Steps To Reproduce

Enable Use Testing platform server mode

Expected behavior

My tests run

Actual behavior

Here are the logs:

[1/27/2025 11:53:08.392 AM]  Launching test runner 'C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.dll'
[1/27/2025 11:53:08.393 AM]  Logging diagnostics under 'C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\Log'
[1/27/2025 11:53:08.393 AM]  Created TCP listener with port 0 for program 1
[1/27/2025 11:53:08.394 AM]  Starting test runner 'C:\Program Files\dotnet\dotnet.exe C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.dll  --server --diagnostic --diagnostic-verbosity Debug --diagnostic-output-directory "C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\Log" --results-directory "C:\_dev\MyClient\dynamics-box\CRM\Server\TestResults" --client-port 50491'
[1/27/2025 11:53:08.409 AM] [Error] Unable to connect to testing platform runner process 'C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.dll'.

When trying to run the test in the command line, I get a different error, not sure if it is because I need to start a TCP listener first?

C:\Users\Me>"C:\Program Files\dotnet\dotnet.exe" "C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plug
in.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.dll"  --server --diagnostic --diagnostic-verbosity Debug --diagnostic-output-directory "C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\Log" --results-directory "C:\_dev\MyClient\dynamics-box\CRM\Server\TestResults" --client-port 50491
A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\'.
Failed to run as a self-contained app.
  - The application was run as a self-contained app because 'C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.runtimeconfig.json' was not found.
  - If this should be a framework-dependent app, add the 'C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.runtimeconfig.json' file and specify the appropriate framework.

Additional context

I'm testing a .Net 4.6.2 project with the following nuget packages with "test" in their name installed:

Unchecking the preview feature and restarting VS resolves the issue.

[Youssef1313]

cc @drognanar

[drognanar]

If this is a net4.6.2 project then it cannot be run via dotnet.exe, @Evangelink are testing platform projects generating an executable for netfx by default?

otherwise @daryllabar could you check if setting <OutputType>Exe</OutputType> in the csproj ends up generating an executable? (C:_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\bin\Debug\MyClient.Dataverse.Plugin.Tests.exe). if the exe is present it should be run instead

[daryllabar]

@drognanar it is not generating an exe.

[Evangelink]

@daryllabar I am assuming your project looks roughly like:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net462</TargetFramework>
    <LangVersion>latest</LangVersion>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <EnableMSTestRunner>true</EnableMSTestRunner>
	<!--<OutputType>Exe</OutputType>-->
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="MSTest" Version="3.7.3" />
  </ItemGroup>

</Project>

If that's the case then the behavior is "expected". As explained in our documentations and blogposts you need to set both EnableMSTestRunner and OutputType for the behavior to be correct.

@Youssef1313 This is an example of where adding a target that warns when output type is not set to exe and enable mstest runner is set could be useful.

Closing as by-design.

[Youssef1313]

@Evangelink Interesting. At least for .NET Framework seems like dotnet exec on libraries don't work (that seems to work for .NET Core)

[Evangelink]

Yes that's what I have seen. Not sure if that's expected dotnet behavior but I wouldn't be surprised there is a fundamental change in assembly layout.

[daryllabar]

@Youssef1313 What blog posts / documentation are you referring to? All I know is I updated my Visual Studio and my unit tests failed to run.

I've updated my test project to this:

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
		<AssemblyName>MyClient.Dataverse.Plugin.Tests</AssemblyName>
		<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
		<DebugType>pdbonly</DebugType>
		<DebugSymbols>true</DebugSymbols>
		<RootNamespace>MyClient.Dataverse.Plugin.Tests</RootNamespace>
		<TargetFramework>net462</TargetFramework>
		<SignAssembly>false</SignAssembly>
		<LangVersion>9</LangVersion>
		<EnableMSTestRunner>true</EnableMSTestRunner>
		<OutputType>Exe</OutputType>
	</PropertyGroup>
	<ItemGroup>
	  <ProjectReference Include="..\MyClient.Dataverse.Plugin\MyClient.Dataverse.Plugin.csproj" />
	  <ProjectReference Include="..\MyClient.Dataverse.Test\MyClient.Dataverse.Test.csproj" />
	</ItemGroup>
</Project>

which does generate the exe,

But now I get a new error:

Unhandled Exception: System.InvalidOperationException: The test framework adapter has not been registered. Use 'ITestApplicationBuilder.RegisterTestFramework' to register it
   at Microsoft.Testing.Platform.Builder.TestApplicationBuilder.<BuildAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at TestingPlatformEntryPoint.<Main>d__0.MoveNext() in C:\_dev\MyClient\dynamics-box\CRM\Server\MyClient.Dataverse.Plugin.Tests\obj\Debug\TestPlatformEntryPoint.cs:line 14
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at TestingPlatformEntryPoint.<Main>(String[] args)

[Evangelink]

What blog posts / documentation are you referring to

https://devblogs.microsoft.com/dotnet/introducing-ms-test-runner/
https://learn.microsoft.com/en-us/dotnet/core/testing/unit-testing-mstest-runner-intro

All I know is I updated my Visual Studio and my unit tests failed to run.

How did you onboard to MSTest runner? The csproj change is not linked to VS update and does require some manual onboarding which you probably did at some point.

But now I get a new error:

@daryllabar Where are MSTest package references?