Two Helix failure entries for a single test failure

[matouskozak]

In some cases, failures are incorrectly duplicated, thus causing confusion while triaging.

For example, https://dev.azure.com/dnceng-public/public/_build/results?buildId=828263&view=ms.vss-test-web.build-test-results-tab&runId=21538738&resultId=144084&paneView=debug

The highlighted failures are the same, yet they appear as two separate ones. They both contain the same log files as well.

I'm unsure whether this is an issue of how xHarness is reporting failures to Helix or somewhere else (AzDo/BuildAnalysis). Possibly related to the same issue as described dotnet/runtime#106319 (comment).

[akoeplinger]

That's a general Helix/AzDO behavior. The work item failing is reported as a separate "test result" because if you have some catastrophic error that results in no regular test results we still want something to show up in AzDO.

[matouskozak]

That's a general Helix/AzDO behavior. The work item failing is reported as a separate "test result" because if you have some catastrophic error that results in no regular test results we still want something to show up in AzDO.

Is it designed to work this way only for mobile jobs? For example, https://dev.azure.com/dnceng-public/public/_build/results?buildId=829545&view=ms.vss-test-web.build-test-results-tab&runId=21579120&paneView=debug

For the windows failure, I can only see System.Runtime.Serialization.Formatters.Tests.BinaryFormatterTests.ValidateAgainstBlobs test failure entry but no entry for the test suit System.Runtime.Serialization.Formatters.Tests.

[akoeplinger]

Hmm interesting. There might be some difference we should investigate then.