[tests] Fix CreateAndBuildProjectTemplate hang/flakiness for Mac Catalyst - #26564
Conversation
…lyst Two issues in the 'Execute'/'DeleteSavedState' helpers used by the Mac Catalyst 'CreateAndBuildProjectTemplate' tests could cause the test process to hang for 30 seconds (or longer) waiting on an unattended 'Do you want to try to reopen its windows again?' AppKit modal dialog: * 'DeleteSavedState' only deleted '<bundleid>.savedState', but macOS stores Mac Catalyst apps' saved state under '<bundleid>~iosmac.savedState' (the 'iosmac' personality suffix), so the saved state was never actually being cleaned up. * Even with a clean Saved Application State folder, AppKit also tracks a separate persistent 'crash history' per bundle identifier that isn't stored in that folder and isn't cleared by deleting it. After many crashed/killed test runs sharing the same default bundle identifier, this alone could still trigger the modal. Fix both: delete the '~iosmac' saved-state variant too, and launch the test app with '-ApplePersistenceIgnoreState YES', which tells AppKit to skip the restore-prompt flow entirely regardless of any tracked crash history. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0d41380a-e6ee-470e-b837-68411050a4f6
There was a problem hiding this comment.
🟡 Changes recommended
The new launch argv modifies args.Length, and many test apps return args.Length as their exit code, which will turn previously-successful runs into failures.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR targets flakiness/hangs in the Mac Catalyst CreateAndBuildProjectTemplate-style execution helpers by preventing macOS/AppKit state-restore prompts from blocking unattended test runs.
Changes:
- Launches the built app with
-ApplePersistenceIgnoreState YESto bypass AppKit’s “reopen windows” restore prompt flow. - Improves saved-state cleanup by also deleting the
~iosmac.savedStatevariant used by Mac Catalyst apps.
File summaries
| File | Description |
|---|---|
| tests/dotnet/UnitTests/TestBaseClass.cs | Adjusts app execution arguments to avoid AppKit restore prompts and expands saved-state deletion to include the ~iosmac personality suffix. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Use the defaults command to configure ApplePersistenceIgnoreState for the test app, then remove the override after execution. Keep the app launch argument list empty so tests that use args.Length as their exit code continue to pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…-more-saved-state
There was a problem hiding this comment.
🟡 Changes recommended
The new defaults write/delete is redundantly executed twice per run (inside a loop) and the implementation diverges from the PR description, both of which should be resolved to avoid introducing new flakiness and confusion.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
tests/dotnet/UnitTests/TestBaseClass.cs:536
⚠️ defaults write/deleteis executed inside theforeachloop, so it runs twice (once per identifier variant) with identical arguments. This adds unnecessary process launches and can extend timeouts (each call allows 30s), increasing test runtime/flakiness. Move thedefaultscall outside the loop (and consider logging failures via ExitCode/output for the write case).
if (cleanup) {
Execution.RunAsync ("/usr/bin/defaults", new [] { "delete", bundleIdentifier, "ApplePersistenceIgnoreState" }, timeout: TimeSpan.FromSeconds (30)).Result;
} else {
Execution.RunAsync ("/usr/bin/defaults", new [] { "write", bundleIdentifier, "ApplePersistenceIgnoreState", "-bool", "YES" }, timeout: TimeSpan.FromSeconds (30)).Result;
}
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
🚀 [CI Build #edd4d3e] Test results 🚀Test results✅ All tests passed on VSTS: test results. 🎉 All 264 tests passed 🎉 Tests counts✅ assembly-processing: All 1 tests passed. Html Report (VSDrops) Download macOS tests✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
Two issues in the 'Execute'/'DeleteSavedState' helpers used by the Mac Catalyst 'CreateAndBuildProjectTemplate' tests could cause the test process to hang for 30 seconds (or longer) waiting on an unattended 'Do you want to try to reopen its windows again?' AppKit modal dialog:
'DeleteSavedState' only deleted '.savedState', but macOS stores Mac Catalyst apps' saved state under '~iosmac.savedState' (the 'iosmac' personality suffix), so the saved state was never actually being cleaned up.
Even with a clean Saved Application State folder, AppKit also tracks a separate persistent 'crash history' per bundle identifier that isn't stored in that folder and isn't cleared by deleting it. After many crashed/killed test runs sharing the same default bundle identifier, this alone could still trigger the modal.
Fix both: delete the '~iosmac' saved-state variant too, and set the application default 'ApplePersistenceIgnoreState' to 'YES', which tells AppKit to skip the restore-prompt flow entirely regardless of any tracked crash history.
Copilot-Session: 0d41380a-e6ee-470e-b837-68411050a4f6