Skip to content

Commit 41fb6ae

Browse files
[tests] Stream on-device test results to MTP as they finish (#11833)
## Summary The MTP test adapter (`Microsoft.Android.Run`) reports on-device test results **all-or-nothing at the very end** of a run, which means a mid-run crash discards the entire run. ### The bug While investigating a CoreCLRTrimmable CI failure (build 1487907, PR #11802) that reported `Zero tests ran` after 52s, I found the app had actually **run 263 tests (251 passed)** and then crashed with a native SIGSEGV in the CoreCLR GC. Despite that, MTP reported nothing. Root cause — reporting is buffered until the end: - **Device** (`TestInstrumentation.OnStart`): runs the whole suite, buffers every result, and only writes the TRX + reports `resultsPath` in the final `Finish()` call. - **Host** (`AndroidTestAdapter`): blocks on `am instrument -w`, parses only the final bundle, then publishes all `TestNodeUpdateMessage`s in one batch. So when the process crashes mid-run, `Finish()` never runs → no TRX, no `resultsPath` → host throws *"Instrumentation did not report a resultsPath in the bundle."* → **Zero tests ran**, with every passed test discarded and no sign a crash happened. An `INSTRUMENTATION_STATUS` was already sent per test (that's the `[PASSED]`/`[FAILED]` in logcat), but the host used `-w` without `-r` and ignored it. ### The fix — stream results live - **Device** (`TestInstrumentation.TestListener`): emit an enriched `INSTRUMENTATION_STATUS` block per test on **start** and **finish**, carrying class/name/outcome and (Base64-encoded, to stay single-line) failure message + stack trace. - **Host** (`AndroidTestAdapter`): run `am instrument -w -r`, parse status blocks **line-by-line**, and publish each test to MTP **as it finishes**. A `LineWriter` splits process output into lines; a `StatusStreamParser` hands completed blocks to an async consumer via a channel (the synchronous read loop never blocks on `PublishAsync`). Now a mid-run crash only loses the in-flight test — every test completed beforehand is already reported. When the run doesn't finish cleanly, a **synthetic failed test node** is published so the run is clearly marked failed instead of silently empty. The TRX-pull path is retained as a fallback for instrumentation that doesn't stream. ### Also `[Ignore]` the `AndroidMessageHandlerTests.ServerCertificateCustomValidationCallback_*` tests, which crash the test process with a native SIGSEGV (#8608). The root cause will be diagnosed separately; this unblocks the CoreCLR/Trimmable device test runs. ## Testing Host adapter builds clean. Full validation requires the on-device CI matrix (device build not run locally).
1 parent ad8afc0 commit 41fb6ae

5 files changed

Lines changed: 488 additions & 45 deletions

File tree

build-tools/automation/yaml-templates/apk-instrumentation.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ steps:
7474
$env:PATH = "${DOTNET_ROOT};$env:PATH"
7575
$dotnetPath = "${DOTNET_ROOT}\dotnet.exe"
7676
}
77-
& $dotnetPath test $projectFile --no-build -bl:${{ parameters.xaSourcePath }}/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog -c ${{ parameters.configuration }} --results-directory $resultsDir --report-trx --report-trx-filename ${{ parameters.testName }}.trx ${{ parameters.extraBuildArgs }}
77+
& $dotnetPath test $projectFile --no-build -bl:${{ parameters.xaSourcePath }}/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog -c ${{ parameters.configuration }} --results-directory $resultsDir --report-trx --report-trx-filename ${{ parameters.testName }}.trx --output Detailed ${{ parameters.extraBuildArgs }}
7878
Pop-Location
7979
displayName: run ${{ parameters.testName }}
8080
condition: ${{ parameters.condition }}

0 commit comments

Comments
 (0)