dotnet-watch tests failing

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=629827
Build error leg or test failing: dotnet-watch.Tests.dll.1.WorkItemExecution
Pull request: #39888

Error Message

##[error].packages/microsoft.dotnet.helix.sdk/8.0.0-beta.24177.1/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(89,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item dotnet-watch.Tests.dll.1 in job 82728a56-9430-4356-ab82-1c21670792a7 has failed

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item dotnet-watch\\.Tests\\.dll\\.1 in job [a-z0-9\\-]+ has failed",
  "BuildRetry": true,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=629827
Error message validated: [Work item dotnet-watch\.Tests\.dll\.1 in job [a-z0-9\-]+ has failed]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/4/2024 9:51:10 AM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1021108	dotnet/sdk	🟣 Run FullFramework Tests	Log	#48028
1020922	dotnet/sdk	Run FullFramework Tests	Log	#48589
1020237	dotnet/sdk	🟣 Run FullFramework Tests	Log	#48436
1019981	dotnet/sdk	Run TestBuild Tests	Log	#48556
2689266	dotnet-sdk	Run TestBuild Tests	Log	#49269
2689020	dotnet-sdk	Run TestBuild Tests	Log	#49252
1018194	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48477
1017988	dotnet/sdk	Run TestBuild Tests	Log	#48494
2688317	dotnet-sdk	Run FullFramework Tests	Log	#49223
2688152	dotnet-sdk	Run TestBuild Tests	Log	#49128
1017353	dotnet/sdk	Run TestBuild Tests	Log	#48486
1016809	dotnet/sdk	Run FullFramework Tests	Log	#48472
1016293	dotnet/sdk	Run TestBuild Tests	Log	#48447
2686950	dotnet-sdk	Run TestBuild Tests	Log	#49128
1015325	dotnet/sdk	Run FullFramework Tests	Log	#48430
1015239	dotnet/sdk	Run TestBuild Tests	Log	#48393
1015036	dotnet/sdk	Run TestBuild Tests	Log	#48425
2685339	dotnet-sdk	Run FullFramework Tests	Log	#48918
2685332	dotnet-sdk	Run TestBuild Tests	Log	#49129
1014045	dotnet/sdk	Run FullFramework Tests	Log	#48405
1013477	dotnet/sdk	Run TestBuild Tests	Log	#48274
1012858	dotnet/sdk	Run TestBuild Tests	Log	#48365
1012929	dotnet/sdk	Run TestBuild Tests	Log	#48266
1012824	dotnet/sdk	Run TestBuild Tests	Log	#48363
1012580	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48353
1012515	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48353
1012496	dotnet/sdk	Run TestBuild Tests	Log	#48274
1012414	dotnet/sdk	Run TestBuild Tests	Log	#48356
1012268	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48353
1011910	dotnet/sdk	Run TestBuild Tests	Log	#48291
1011706	dotnet/sdk	Run TestBuild Tests	Log	#48341
2683475	dotnet-sdk	Run TestBuild Tests	Log	#48610
1010510	dotnet/sdk	Run TestBuild Tests	Log	#48298
1010475	dotnet/sdk	Run TestBuild Tests	Log	#48305
1010372	dotnet/sdk	Run TestBuild Tests	Log	#48298
1010369	dotnet/sdk	Run TestBuild Tests	Log	#48297
2682841	dotnet-sdk	Run TestBuild Tests	Log	#48610
1010094	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48277
1009755	dotnet/sdk	Run FullFramework Tests	Log	#48005
1009247	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48246
1009236	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48087
1008785	dotnet/sdk	Run FullFramework Tests	Log	#48005
2681751	dotnet-sdk	Run FullFramework Tests	Log	#48610
2681707	dotnet-sdk	Run FullFramework Tests	Log	#48611
1008115	dotnet/sdk	Run TestBuild Tests	Log	#45520
1007537	dotnet/sdk	Run TestBuild Tests	Log	#48149
1007531	dotnet/sdk	Run TestBuild Tests	Log	#48151
1007356	dotnet/sdk	Run TestBuild Tests	Log	#48185
1006402	dotnet/sdk	Run TestBuild Tests	Log	#48196
1006419	dotnet/sdk	Run FullFramework Tests	Log	#48005
1005844	dotnet/sdk	Run TestBuild Tests	Log	#48181
1005741	dotnet/sdk	Run TestBuild Tests	Log	#48180
1003629	dotnet/sdk	Run FullFramework Tests	Log	#48005
1003508	dotnet/sdk	Run FullFramework Tests	Log	#48125
2678133	dotnet-sdk	Run FullFramework Tests	Log	#48610
1003249	dotnet/sdk	Run FullFramework Tests	Log	#48005
2678053	dotnet-sdk	Run TestBuild Tests	Log	#48918
2678012	dotnet-sdk	Run FullFramework Tests	Log	#48611
2677925	dotnet-sdk	Run TestBuild Tests	Log	#48626
1002556	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48028
1002021	dotnet/sdk	Run FullFramework Tests	Log	#48005
1001451	dotnet/sdk	Run FullFramework Tests	Log	#48005
1001366	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48081
1000507	dotnet/sdk	Run FullFramework Tests	Log	#48005
1000375	dotnet/sdk	Run FullFramework Tests	Log	#48005
999041	dotnet/sdk	Run FullFramework Tests	Log	#48005
998918	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48012
996348	dotnet/sdk	Run TestBuild Tests	Log	#47691
996237	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47956
994847	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47873
994703	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47773
994648	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47869
994570	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47914
994014	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47893
994049	dotnet/sdk	Run TestBuild Tests	Log	#47900
993357	dotnet/sdk	Run TestBuild Tests	Log	#47819
993281	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47833
992837	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47792
992300	dotnet/sdk	Run TestBuild Tests	Log	#47850
991243	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47833
991176	dotnet/sdk	Run TestBuild Tests	Log	#47736
990992	dotnet/sdk	🟣 Run TestBuild Tests	Log	#47833
990830	dotnet/sdk	Run TestBuild Tests	Log	#47827
990254	dotnet/sdk	Run FullFramework Tests	Log	#47818

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	17	84

[marcpopMSFT]

@tmat we still see this fairly frequently. Any luck? Here's a recent one from a codeflow PR:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-41616-merge-58511284b77c40778a/dotnet-watch.Tests.dll.1/1/console.418de7ca.log?helixlogtype=result

[tmat]

No luck yet. This might help: https://github.com/dotnet/sdk/blob/d45e40d665e647804de03d2b4eec2aae9c488cb5/test/dotnet-watch.Tests/xunit.runner.json

It will print out test info every 20 seconds a test is running.

[marcpopMSFT]

@tmat another failure this morning shows a bunch of build processes for watch apps still running... https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-sdk-refs-pull-41958-merge-84357342ee154a5088/dotnet-watch.Tests.dll.2/1/console.3dddc463.log?helixlogtype=result

[tmat]

Looks like that's in 8.0 branch. Porting test diagnostics from 9: #41963

[Forgind]

This has been making one of my PRs fail today, so I looked into it a bit. Based on when this issue was created and which tests are failing, I suspect this PR:
#39618

I asked jasonmalinowski whether the MSBuildWorkspace is supposed to support F# because I'd found a code snippet that suggested that it shouldn't, and he said it doesn't and never did. Did that PR make us use MSBuildWorkspace in a different way than we had before?

[Forgind]

The changes to HotReloadDotNetWatcher look most suspicious to me. It looks like a substantial change to how we call MSBuild.