The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1274111	dotnet/roslyn	Test	Log	dotnet/roslyn#82230
2892357	dotnet-maui	Controls (v18.4) TabbedPage,TableView,TimePicker,TitleView,ToolbarItem
1275126	maestro-auth-test/helix-queue-insights-test	Run Tests (Windows)	Log
1275070	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65283
1275071	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65283
1275037	dotnet/runtime	Send tests to Helix (Unix)	Log
1275042	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33817
1274997	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
2892356	dotnet-maui	Controls Layout	Log
1275001	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33814
1275007	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33815
1274998	dotnet/maui	Run Integration Tests - Build windows	Log
1274995	dotnet/maui	Run Integration Tests - Build windows	Log
1266961	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65226
1266936	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65224
1266937	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65224
1266963	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65226
1274975	dotnet/maui	Controls Layout	Log	dotnet/maui#33808
1274942	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33808
1273684	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33234
1274916	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65283
1274917	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65283
1274817	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123645
1274763	dotnet/runtime	Send to Helix	Log
2892169	dotnet-runtime	Send tests to Helix (Unix)	Log
2892029	dotnet-runtime	Send job to Helix (Unix)	Log
1274764	dotnet/runtime	Send tests to Helix (Unix)	Log
2892155	dotnet-runtime	Send to Helix	Log	#57010
1274746	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65283
1274747	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#65283
1274581	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33808
1274577	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33809
2891910	dotnet-maui	Controls ScrollView	Log
2891914	dotnet-maui	Controls (v18.4) TabbedPage,TableView,TimePicker,TitleView,ToolbarItem	Log
2891903	dotnet-maui	Controls Layout	Log
1274499	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29101
1274495	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33799
1274498	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33160
1274494	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33792
1274496	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33524
1274497	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33432
1274469	dotnet/maui	Run Integration Tests - Build windows	Log
1274474	dotnet/maui	Controls Layout	Log
1274356	maestro-auth-test/helix-queue-insights-test	Run Tests (Windows)	Log
1274336	dotnet/aspnetcore	Run build.cmd helix target	Log
1274333	dotnet/aspnetcore	Run build.cmd helix target	Log
1274334	dotnet/aspnetcore	Run build.cmd helix target	Log
1274082	dotnet/runtime	Send to Helix	Log
1274269	dotnet/roslyn	Test	Log
1274236	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33618
1274264	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7575
2891701	dotnet-dotnet	🛡 Guardian: ARMory	Log
2891580	dotnet-dotnet	🛡 Generate SBoM Manifest	Log
1273952	dotnet/runtime	Build product	Log	dotnet/runtime#123810
1273949	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7573
1273806	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123779
1273832	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
1273860	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123142
2891579	dotnet-sdk	SDL Binary Analysis Tools Pre-Checks (1ES PT)	Log	#57370
2891582	dotnet-dotnet	Secure Supply Chain Analysis (auto-injected by policy)	Log	#57371
2891578	dotnet-sdk	Build	Log	#57370
1273781	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123775
1273830	dotnet/maui	Run Integration Tests - Build windows	Log
1273825	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33618
1273802	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33799
1272799	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123774
2891523	dotnet-dotnet	🛡 Generate SBoM Manifest	Log
1273773	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123342
2891511	dotnet-dotnet	Build (azurelinuxArm64CrossContainer)	Log
1273679	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123044
1273661	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#122638
2891488	dotnet-dotnet	Checkout dotnet-dotnet@release/11.0.1xx-preview1 to s	Log
1273644	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123726
1273708	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65263
2891445	dotnet-dotnet	Run Scancode (azurelinuxSourceBuildTestContainer)	Log
1272781	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123273
2891439	dotnet-dotnet	Run Scancode (azurelinuxSourceBuildTestContainer)	Log
1273598	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123753
2891430	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
1273611	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123800
1273581	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123033
1273497	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29101
1273574	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123251
2891422	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
1273557	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123794
2891407	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
2891399	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
1270002	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123386
2891396	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
1273556	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33792
1273306	dotnet/runtime	Send to Helix	Log
1273485	dotnet/aspnetcore	Run build.cmd helix target	Log
2891332	dotnet-aspnetcore	Run build.cmd helix target	Log
2891331	dotnet-aspnetcore	Run build.cmd helix target	Log
1273379	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123791
1273373	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#123500
2891313	dotnet-arcade-services	Publish Using Darc	Log
1273427	dotnet/runtime	Build	Log	dotnet/runtime#123142
1273339	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#123753
1273355	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#122970
Displaying 100 of 2277 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
30	1065	2277

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez