Intermittent failure starting a process

[Youssef1313]

Description

In microsoft/testfx repo, I'm seeing intermittent failures that look like this:

  failed Invoke_TestTarget_With_Incompatible_Arch (24s 257ms)
    Assert.Contains failed. String '  Determining projects to restore...
      Restored D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj (in 2.47 sec).
      MSBuild Tests -> D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.dll
      Run tests: 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.dll' [net10.0|arm64]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: The specified task executable "MSBuild Tests.exe" could not be run. System.ComponentModel.Win32Exception (0x80004005): An error occurred trying to start process 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.exe' with working directory 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests'. The operation completed successfully. [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) The specified task executable "MSBuild Tests.exe" could not be run. System.ComponentModel.Win32Exception (0x80004005): An error occurred trying to start process 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.exe' with working directory 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests'. The operation completed successfully. [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at Microsoft.Build.Utilities.ToolTask.Execute() [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at Microsoft.Build.Utilities.ToolTask.Execute() [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
    Build FAILED.
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: The specified task executable "MSBuild Tests.exe" could not be run. System.ComponentModel.Win32Exception (0x80004005): An error occurred trying to start process 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.exe' with working directory 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests'. The operation completed successfully. [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) The specified task executable "MSBuild Tests.exe" could not be run. System.ComponentModel.Win32Exception (0x80004005): An error occurred trying to start process 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.exe' with working directory 'D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests'. The operation completed successfully. [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
D:\a\_work\1\s\artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: at Microsoft.Build.Utilities.ToolTask.Execute() [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj] [D:\a\_work\1\s\test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj]
##[error]artifacts\tmp\Release\testsuite\tOOCM\.packages\microsoft.testing.platform.msbuild\2.1.0-preview.25580.1\buildMultiTargeting\Microsoft.Testing.Platform.MSBuild.targets(347,5): error MSB6003: (NETCORE_ENGINEERING_TELEMETRY=Build) at Microsoft.Build.Utilities.ToolTask.Execute() [D:\a\_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\MSBuild Tests.csproj]
        0 Warning(s)
        1 Error(s)
    Time Elapsed 00:00:20.43' does not contain string 'The specified executable is not a valid application for this OS platform.'.

The error:

System.ComponentModel.Win32Exception (0x80004005): An error occurred trying to start process 'D:\a_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests\bin\Debug\net10.0\win-arm64\MSBuild Tests.exe' with working directory 'D:\a_work\1\s\artifacts\tmp\Release\testsuite\dvgGn\MSBuildTests'. The operation completed successfully.

doesn't make sense to me. It feels like Win32 API for starting process indicated failure, but last exit code was zero, which is how it ended up in "The operation completed successfully" failure. That feels unexpected.

Reproduction Steps

None that I have to reliably reproduce. But the error makes no sense.

Expected behavior

More sensible error message, or just works.

Actual behavior

Error messages that doesn't make a lot of sense.

Regression?

Not sure.

Known Workarounds

No response

Configuration

No response

Other information

No response

[adamsitnik]

Hi @Youssef1313

Which .NET version are you using? Do you try to start the process with UseShellExecute set to true? Can you point me to the logic responsible for starting the process?

https://github.com/dotnet/runtime/blob/d0bc904454e2b9ad8fc25316dc3dd7f0b920c955/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Win32.cs#L25C30-L27

[Youssef1313]

@adamsitnik We just use MSBuild's ToolTask. The logic there is owned by MSBuild: https://github.com/dotnet/msbuild/blob/dd27cf39334b66329ff8eea71db6e8153e0929c6/src/Utilities/ToolTask.cs#L745

We do dotnet build -t:Test and embed our own Test target that runs our ToolTask. Not sure in that context if it's running .NET Framework or modern .NET though.