Skip to content

Commit 7bcd003

Browse files
kazo0claude
andcommitted
ci: run the runtime tests on desktop, WebAssembly, Android and iOS
Layer 1/4 of the MSAL authentication work (#3139): the device lanes, without any MSAL code. Build stages publish each head as a pipeline artifact; test stages boot an emulator / simulator / headless browser, run the engine with a namespace filter and harvest NUnit results. Ported from Studio Live's setup. - Four lanes (stage-runtime-tests-{desktop,wasm,android,ios}.yml) fed by stage-build-runtimetests-mobile.yml; scripts under build/test-scripts with bounded waits, inline emulator logs and a results validator that fails on an empty or all-skipped run - Runtime-test head: MobileRuntimeTestsAutostart drives UnitTestsControl and writes the NUnit document on Android/iOS (the engine's own runner throws PlatformNotSupportedException in a sandboxed app); WebAssemblyRuntimeShim supplies the __WASM__ symbol and InvokeJS the source-shipped engine needs; MainActivity uses GetExternalFilesDir for the results path; the iOS head carries the keychain entitlement MSAL's cache accessor demands - src/Directory.Build.targets exempts AndroidApplication heads from the library-only Private=false rule: .NET Android rewrites an app head's OutputType to Library, which emptied ReferenceCopyLocalPaths and shipped APKs without their ProjectReference closure - src/Directory.Build.props turns the trim/AOT analyzers off for test and sample projects; they only turned "this test calls the API it exists to test" into IL2026 errors on the WebAssembly heads - RuntimeTestsFilter is Storage.UI.Tests: green on every lane today. The MSAL namespace joins it in the next layer; '!_HotReload' waits on 15 rotted cases in the wider suite AGENTS.md records the new Window() rule (mobile rejects secondary windows); specs/lessons.md the CI lessons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqfQh5ovrPEvFGcERLiLhH
1 parent 0960026 commit 7bcd003

27 files changed

Lines changed: 1769 additions & 23 deletions

.gitignore

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -339,4 +339,10 @@ DebugPlatforms.props
339339

340340
!build/ci/publish
341341

342-
*.lscache
342+
*.lscache
343+
# Runtime-test results/logs written into the tree by build/ci/stage-runtime-tests-*.yml
344+
/build/*-runtime-tests/
345+
/build/*-runtime-tests-app/
346+
/build/playwright-browsers/
347+
# --tool-path used by build/test-scripts/wasm-runtime-tests.sh for the WebAssembly runner
348+
/build/.dotnet-tools/

AGENTS.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# AI Agents Contribution & Coding Instructions
1+
# AI Agents Contribution & Coding Instructions
22
<!-- cspell:ignore PKCE -->
33

44
This document defines strict guardrails for any AI-assisted or automated agent contributions (including Copilot, custom prompt runners, or scripted refactors) working in the **Uno.Extensions** repository. Human contributors must also ensure generated changes comply before merge. It is the single source of truth for repo-wide orientation *and* the rules agents must follow; `CLAUDE.md` at the repo root is a thin pointer that includes this file.
@@ -248,6 +248,10 @@ The repo has three distinct test surfaces — each with a different runner:
248248

249249
Test-placement gotcha: a UI-host-requiring test placed in a `*.Tests` (not `*.UI.Tests`) project will be **picked up** locally by `dotnet test` (and likely fail or flake) but **excluded** by package CI — producing platform-dependent results. Match the project type to the host requirement.
250250

251+
### Windows in UI / runtime tests
252+
253+
`new Window()` is not portable. Android and iOS reject secondary windows outright (`InvalidOperationException: Creating secondary windows on this platform is not allowed`), so a test that constructs one passes on desktop and fails every mobile lane — this is what turned all 10 `Given_MsalAuthentication` cases red on the iOS simulator in build 228808. Take the host's window via `UnitTestsUIContentHelper.CurrentTestWindow` instead, and if you assign `Content`, bracket it with `SaveOriginalContent`/`RestoreOriginalContent`.
254+
251255
### Hot-reload tests
252256

253257
Files in `src/Uno.Extensions.Navigation.UI.Tests/` named `Given_HotReload.cs` (and the supporting `HotReload*.cs` targets) use `[RunsInSecondaryApp]` and `HotReloadHelper.UpdateSourceFile(...)`. Constraints — see `src/Uno.Extensions.Navigation.UI.Tests/HotReload.Spec.md` for the living spec:

build/ci/.azure-pipelines.yml

Lines changed: 52 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
trigger:
1+
trigger:
22
branches:
33
include:
44
- main
@@ -14,6 +14,30 @@ pr:
1414
- release/*
1515
- legacy/*
1616

17+
variables:
18+
# iOS agent image, Xcode root and simulator target are kept together on purpose: the simulator
19+
# runtime the .app is linked against in the build job must exist on the agent that runs it, and
20+
# splitting these across files is how that pairing silently drifts.
21+
IosVmImage: 'macOS-15'
22+
# Xcode and the simulator runtime are a matched pair: actool refuses to compile the asset
23+
# catalog unless a simulator runtime matching the selected Xcode's iphonesimulator SDK is
24+
# installed. Build 227563 pinned Xcode 16.2 (iOS 18.2 SDK, build 22C146) on an agent whose only
25+
# installed runtimes were 18.5, 18.6 and 26.0-26.2, and failed with
26+
# "No simulator runtime version from [...] available to use with iphonesimulator SDK version".
27+
# Xcode 16.4 ships the iOS 18.5 SDK, which matches the installed 18.5 runtime (22F77).
28+
IosXCodeRoot: '/Applications/Xcode_16.4.app'
29+
IosSimulatorDevice: 'iPhone 16'
30+
IosSimulatorRuntimePrefix: 'com.apple.CoreSimulator.SimRuntime.iOS-18-5'
31+
32+
# Which runtime tests the four device stages run. Scoped to a namespace whose suite is green on
33+
# every lane - Storage.UI.Tests: default key-value store selection and the browser-store round
34+
# trip, 8 cases. The wider non-hot-reload suite has 15 failing tests (verified identical at the
35+
# pre-change baseline, so they are rot rather than regressions); widen to '!_HotReload' once those
36+
# are fixed - that is the whole point of turning these stages back on. The engine filter is a
37+
# substring match on the fully-qualified test name with '|' for OR, so this MUST stay namespace
38+
# fragments and never a class name, which silently drops sibling classes.
39+
RuntimeTestsFilter: 'Storage.UI.Tests'
40+
1741
stages:
1842
- stage: Determine_Changes
1943
displayName: Determine Changes
@@ -68,9 +92,35 @@ stages:
6892
condition: and(succeeded(), ne(dependencies.Determine_Changes.outputs['evaluate_changes.DetermineChanges.docsOnly'], 'true'))
6993
jobs:
7094
- template: stage-build-uitests-wasm.yml
71-
# Disabled waiting on uno fix: https://github.com/unoplatform/uno/pull/17668
95+
# Still disabled, but not for the original reason (unoplatform/uno#17668, long since merged).
96+
# Verified 2026-08-13: the full non-hot-reload runtime suite has 15 failing tests on this branch
97+
# AND at the pre-change baseline - navigation/ComboBox/logger cases that have rotted while the
98+
# stage was off. Enabling it as-is turns CI red on day one. The device stages below run the same
99+
# engine against a scoped filter instead; widening that filter is blocked on fixing those 15.
72100
# - template: stage-build-runtimetests-skia.yml
73101
- template: stage-build-runtimetests-skia-hotreload.yml
102+
# Builds the mobile / wasm runtime-test apps as artifacts; the jobs below consume them.
103+
- template: stage-build-runtimetests-mobile.yml
104+
105+
- stage: Runtime_Tests_Devices
106+
displayName: Runtime Tests - Devices
107+
dependsOn: Build_Tests
108+
jobs:
109+
- template: stage-runtime-tests-desktop.yml
110+
# Depends on a workaround: Uno.UI.RuntimeTests.Engine (source package, still true in 2.0.0-dev.81)
111+
# guards its Console.CancelKeyPress registration with `#if !__WASM__`, and Uno.Sdk no longer defines
112+
# __WASM__ for consumer projects, so the browser head used to throw PlatformNotSupportedException
113+
# before the first test (build 228808). Uno.Extensions.RuntimeTests.Core defines the symbol for the
114+
# browserwasm TFM and supplies the InvokeJS its other branches need - see WebAssemblyRuntimeShim.cs.
115+
- template: stage-runtime-tests-wasm.yml
116+
# Was disabled because the APK shipped without its ProjectReference closure and died at
117+
# Application.OnCreate with `open_from_bundles: failed to load bundled assembly
118+
# Uno.Extensions.Reactive.dll`. Root cause was ours: .NET Android rewrites an app head's
119+
# OutputType to 'Library', so the head matched the library-only `Private=false` rule in
120+
# src/Directory.Build.targets and every project reference stopped being copy-local on Android
121+
# alone. See the lesson in specs/lessons.md before touching that ItemDefinitionGroup.
122+
- template: stage-runtime-tests-android.yml
123+
- template: stage-runtime-tests-ios.yml
74124

75125
##
76126
## Publishing

build/ci/cspell.json

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{
1+
{
22
"version": "0.2",
33
"language": "en",
44
"words": [
@@ -7,11 +7,13 @@
77
"Avalonia",
88
"ambiently",
99
"binlog",
10+
"binlogs",
1011
"Blazor",
1112
"blockquotes",
1213
"Bootstrapper",
1314
"brotli",
1415
"browsersync",
16+
"browserwasm",
1517
"bytecode",
1618
"Callout",
1719
"chatops",
@@ -27,9 +29,14 @@
2729
"devs",
2830
"Dismissable",
2931
"Docfx",
32+
"DPAPI",
3033
"ellipsize",
3134
"Entra",
3235
"Expando",
36+
"cref",
37+
"libsecret",
38+
"nuspecs",
39+
"unmaterialized",
3340
"flyouts",
3441
"Framebuffer",
3542
"Gamepad",
@@ -48,7 +55,10 @@
4855
"LLRM",
4956
"mergeable",
5057
"MESCIUS",
58+
"maccatalyst",
5159
"MSAL",
60+
"monodroid",
61+
"msauth",
5262
"msbuild",
5363
"MVUX",
5464
"MVVM",
@@ -85,6 +95,7 @@
8595
"UI's",
8696
"unmanaged",
8797
"Uno's",
98+
"UXAML",
8899
"unoplatform",
89100
"UWP's",
90101
"VSIX",

build/ci/scripts/host-cleanup-linux.sh

Lines changed: 57 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,23 +16,73 @@ fi
1616
echo "Disk space before cleanup:"
1717
df -h /
1818

19-
rm -rf ~/.cargo ~/.rustup ~/.dotnet || true
19+
# ── Docker images — the biggest single win on hosted runners (~30 GB) ───────
20+
# Omitting this is why the runtime-test build jobs still ran out of disk while
21+
# archiving the SDK cache ("tar: Wrote only 8192 of 10240 bytes").
22+
if command -v docker >/dev/null 2>&1; then
23+
docker system prune -af --volumes 2>/dev/null || true
24+
fi
25+
26+
# ── Home-directory toolchains ──────────────────────────────────────────────
27+
# Wipes ~/.dotnet — safe on ephemeral CI agents only. On a developer machine
28+
# set ALLOW_DOTNET_WIPE=1 to opt in.
29+
if [ "${ALLOW_DOTNET_WIPE:-0}" = "1" ] || [ "${CI:-}" = "true" ] || [ "${TF_BUILD:-}" = "True" ]; then
30+
rm -rf ~/.cargo ~/.rustup ~/.dotnet || true
31+
else
32+
echo "Skipping ~/.dotnet wipe — not a CI agent (set ALLOW_DOTNET_WIPE=1 to override)."
33+
rm -rf ~/.cargo ~/.rustup || true
34+
fi
2035

36+
# ── System-wide pre-installed software not needed for .NET builds ──────────
2137
$SUDO rm -rf /usr/share/swift || true
2238
$SUDO rm -rf /opt/microsoft/msedge || true
2339
$SUDO rm -rf /usr/local/.ghcup || true
2440
$SUDO rm -rf /usr/lib/mono || true
25-
$SUDO rm -rf /usr/local/lib/android || true
2641
$SUDO rm -rf /opt/ghc || true
27-
$SUDO rm -rf /opt/hostedtoolcache/CodeQL || true
2842

29-
if command -v snap >/dev/null 2>&1; then
30-
timeout 60s $SUDO snap remove lxd || true
31-
timeout 60s $SUDO snap remove core20 || true
43+
# The agent's Android SDK is ~10 GB and worth reclaiming for jobs that don't
44+
# build Android - but a job that does must keep it, or the build fails with
45+
# XA5300 ("The Android SDK directory could not be found"). Set
46+
# KEEP_ANDROID_SDK=1 in such a job.
47+
if [ "${KEEP_ANDROID_SDK:-0}" = "1" ]; then
48+
echo "Keeping /usr/local/lib/android (KEEP_ANDROID_SDK=1)."
49+
else
50+
$SUDO rm -rf /usr/local/lib/android || true
3251
fi
52+
$SUDO rm -rf /opt/hostedtoolcache/CodeQL || true
3353

54+
# ── Hosted tool cache (non-.NET runtimes) ──────────────────────────────────
55+
# Node is deliberately left in place: several stages use NodeTool@0 and would
56+
# just re-download it.
57+
$SUDO rm -rf /opt/hostedtoolcache/Ruby || true
58+
$SUDO rm -rf /opt/hostedtoolcache/PyPy || true
59+
$SUDO rm -rf /opt/hostedtoolcache/Python || true
60+
$SUDO rm -rf /opt/hostedtoolcache/go || true
61+
62+
# ── apt ────────────────────────────────────────────────────────────────────
63+
# Only snapd is purged here, plus a cache clean.
64+
#
65+
# studio.live's version of this script additionally purges firefox,
66+
# google-chrome-stable, libllvm*, clang-*, llvm-* and php*/ruby*. Do NOT copy
67+
# that list into this repo:
68+
#
69+
# * Purging `libllvm*` cascades. apt removes reverse-dependencies even
70+
# without --auto-remove, and Mesa's software rasteriser (llvmpipe) links
71+
# LLVM, so `xvfb` goes with it. Both Skia desktop stages here run their
72+
# tests under `xvfb-run`, which then fails with exit 127
73+
# ("xvfb-run: command not found") - build 227556.
74+
# * `google-chrome-stable` is needed by the WebAssembly UI tests, which drive
75+
# a real Chrome through chromedriver.
76+
#
77+
# The Docker prune above already reclaims far more than this list would.
3478
if command -v apt-get >/dev/null 2>&1; then
35-
DEBIAN_FRONTEND=noninteractive timeout 120s $SUDO apt-get purge -y snapd || true
79+
DEBIAN_FRONTEND=noninteractive timeout 120s $SUDO apt-get purge -y snapd 2>/dev/null || true
80+
DEBIAN_FRONTEND=noninteractive timeout 60s $SUDO apt-get clean 2>/dev/null || true
81+
fi
82+
83+
if command -v snap >/dev/null 2>&1; then
84+
timeout 60s $SUDO snap remove lxd 2>/dev/null || true
85+
timeout 60s $SUDO snap remove core20 2>/dev/null || true
3686
fi
3787

3888
echo "Disk space after cleanup:"

0 commit comments

Comments
 (0)