Commit 41fb6ae
authored
[tests] Stream on-device test results to MTP as they finish (#11833)
## Summary
The MTP test adapter (`Microsoft.Android.Run`) reports on-device test results **all-or-nothing at the very end** of a run, which means a mid-run crash discards the entire run.
### The bug
While investigating a CoreCLRTrimmable CI failure (build 1487907, PR #11802) that reported `Zero tests ran` after 52s, I found the app had actually **run 263 tests (251 passed)** and then crashed with a native SIGSEGV in the CoreCLR GC. Despite that, MTP reported nothing.
Root cause — reporting is buffered until the end:
- **Device** (`TestInstrumentation.OnStart`): runs the whole suite, buffers every result, and only writes the TRX + reports `resultsPath` in the final `Finish()` call.
- **Host** (`AndroidTestAdapter`): blocks on `am instrument -w`, parses only the final bundle, then publishes all `TestNodeUpdateMessage`s in one batch.
So when the process crashes mid-run, `Finish()` never runs → no TRX, no `resultsPath` → host throws *"Instrumentation did not report a resultsPath in the bundle."* → **Zero tests ran**, with every passed test discarded and no sign a crash happened.
An `INSTRUMENTATION_STATUS` was already sent per test (that's the `[PASSED]`/`[FAILED]` in logcat), but the host used `-w` without `-r` and ignored it.
### The fix — stream results live
- **Device** (`TestInstrumentation.TestListener`): emit an enriched `INSTRUMENTATION_STATUS` block per test on **start** and **finish**, carrying class/name/outcome and (Base64-encoded, to stay single-line) failure message + stack trace.
- **Host** (`AndroidTestAdapter`): run `am instrument -w -r`, parse status blocks **line-by-line**, and publish each test to MTP **as it finishes**. A `LineWriter` splits process output into lines; a `StatusStreamParser` hands completed blocks to an async consumer via a channel (the synchronous read loop never blocks on `PublishAsync`).
Now a mid-run crash only loses the in-flight test — every test completed beforehand is already reported. When the run doesn't finish cleanly, a **synthetic failed test node** is published so the run is clearly marked failed instead of silently empty. The TRX-pull path is retained as a fallback for instrumentation that doesn't stream.
### Also
`[Ignore]` the `AndroidMessageHandlerTests.ServerCertificateCustomValidationCallback_*` tests, which crash the test process with a native SIGSEGV (#8608). The root cause will be diagnosed separately; this unblocks the CoreCLR/Trimmable device test runs.
## Testing
Host adapter builds clean. Full validation requires the on-device CI matrix (device build not run locally).1 parent ad8afc0 commit 41fb6ae
5 files changed
Lines changed: 488 additions & 45 deletions
File tree
- build-tools/automation/yaml-templates
- src/Microsoft.Android.Run
- tests/TestRunner.Core
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
77 | | - | |
| 77 | + | |
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| |||
0 commit comments