The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1272659	dotnet/runtime	Send to Helix	Log
1272866	dotnet/runtime	Build	Log	dotnet/runtime#123779
1272666	dotnet/maui	Run Integration Tests - RunOniOS_MauiRelease macOS	Log
1272847	dotnet/runtime	Build	Log	dotnet/runtime#123781
2890820	dotnet-maui	Controls CollectionView	Log
1272640	dotnet/maui	Run Integration Tests - RunOniOS_BlazorRelease macOS	Log	dotnet/maui#32296
1272660	dotnet/runtime	Send tests to Helix (Unix)	Log
2890823	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Binary)	Log
1272667	dotnet/maui	Controls Layout	Log
1272764	dotnet/runtime	Build	Log	dotnet/runtime#123775
1272642	dotnet/maui	Controls CollectionView	Log
2890841	dotnet-maui	Run Integration Tests - RunOniOS_MauiNativeAOT macOS	Log
1272737	dotnet/runtime	Build	Log	dotnet/runtime#123734
1272398	dotnet/runtime	Send tests to Helix (Unix)	Log
1272541	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123763
1272423	dotnet/runtime	Send to Helix	Log
1271887	dotnet/maui	Install Simulator Runtimes	Log
2890835	dotnet-sign	Build and run tests	Log
1272625	dotnet/runtime	Build	Log	dotnet/runtime#123726
1272445	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123768
1272429	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123766
1272563	dotnet/runtime	Send tests to Helix (Windows)	Log
2890817	dotnet-performance	Send job to Helix (Unix)	Log
1272407	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33756
2890760	dotnet-dotnet	Build	Log
1272483	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123701
1272408	dotnet/maui	Controls Layout	Log	dotnet/maui#33756
1272474	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123695
1272459	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123727
1272425	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123763
1272436	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123765
1272487	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123754
1272419	dotnet/runtime	Send tests to Helix (Windows)	Log
1272418	dotnet/runtime	Send tests to Helix (Windows)	Log
1272415	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123593
1272411	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123761
2890746	dotnet-runtime	Send job to Helix (Windows)	Log
1272422	dotnet/runtime	Send to Helix	Log
1272387	dotnet/runtime	Send tests to Helix (Windows)	Log
1270869	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123509
1272449	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123455
1272364	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123762
1272432	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33618
1272342	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123500
1270002	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123386
1272332	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123759
1272392	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123601
1272307	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123483
1272315	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#121985
1272298	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123738
1271975	dotnet/maui	Run Integration Tests - RunOnAndroid macOS	Log	dotnet/maui#33723
1272263	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33198
2890600	dotnet-maui	Controls Layout	Log
1272303	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123642
1272261	dotnet/maui	Controls CollectionView	Log	dotnet/maui#33737
2890682	dotnet-performance	Send job to Helix (Unix)	Log
1272200	dotnet/maui	Run Integration Tests - RunOniOS_MauiNativeAOT macOS	Log
1272192	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123033
1272171	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123757
1272159	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123183
1272150	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123755
1272201	dotnet/maui	Controls CollectionView	Log
2890599	dotnet-maui	Run Integration Tests - RunOniOS_MauiRelease macOS	Log
1272125	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123323
1272185	dotnet/maui	Controls CollectionView	Log
1270637	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123142
1272111	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123611
1272082	dotnet/runtime	Send tests to Helix (Windows)	Log
2890593	dotnet-dotnet	Secure Supply Chain Analysis (auto-injected by policy)	Log
2890463	dotnet-maui	Controls Layout	Log
1272083	dotnet/runtime	Send tests to Helix (Windows)	Log
1272043	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123752
1272086	dotnet/runtime	Send to Helix	Log
1272019	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33774
2890514	dotnet-runtime	Send job to Helix (Windows)	Log
1271951	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123717
1271959	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33759
1271912	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#122638
1271765	dotnet/runtime	Send to Helix	Log
2890481	dotnet-aspnetcore	Run build.cmd helix target	Log
1271945	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33765
1271939	dotnet/aspnetcore	Run build.cmd helix target	Log
2890490	dotnet-aspnetcore	Run build.cmd helix target	Log
1271895	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123719
2890462	dotnet-maui	Run Integration Tests - RunOniOS_MauiRelease macOS	Log
1271902	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123342
2890475	dotnet-performance	Send job to Helix (Unix)	Log
1271846	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123138
1271940	dotnet/performance	Send job to Helix (Windows)	Log
1271766	dotnet/runtime	Send to Helix	Log
1271821	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123546
1271886	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33618
1271875	dotnet/runtime	Send to Helix	Log
1271823	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33689
1271784	dotnet/runtime	Send tests to Helix (Windows)	Log
1271748	dotnet/runtime	Send to Helix	Log
1271767	dotnet/runtime	Publish Build Assets	Log
1271756	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123726
1271433	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123716
1271763	dotnet/aspnetcore	Run build.cmd helix target	Log
Displaying 100 of 2128 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
140	1106	2128

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez