Commit b494233
Enforce exactly one Category attribute per UI test (#33242)
## Description
This PR enforces that every UI test has **exactly one** `[Category]`
attribute through a build-time analyzer. Previously, tests could have
multiple categories which caused them to run multiple times in CI.
Each test needs to have a category so they are ran in the CI pipeline.
With this analyzer we ensure that this is not forgotten during
development.
## Changes
### Analyzer Updates (`NUnitTestMissingCategoryAnalyzer`)
- **MAUI0001**: Build error when a test has **no** category
- **MAUI0002**: Build error when a test has **more than one** category
- Changed severity from `Warning` to `Error`
- Excludes platform-specific ignore attributes
(`FailsOnAndroidWhenRunningOnXamarinUITest`, etc.) from the category
count since they conditionally derive from `CategoryAttribute`
### Fixed 314 Tests
All tests with multiple categories have been fixed by keeping the most
appropriate single category:
- **Control-specific categories** preferred (Button, Entry,
CollectionView, ListView, etc.)
- **Feature categories** kept when no control-specific category applied
(Navigation, Shell, Gestures, etc.)
- **Removed generic categories** like `Compatibility` in favor of more
specific ones
### Added Analyzer Unit Tests
New test project `UITest.Analyzers.Tests` with 20 tests covering:
- Missing category detection
- Multiple categories detection
- Custom `CategoryAttribute` derivatives
- Platform ignore attribute exclusions
- Edge cases
## CI Impact
Tests with multiple categories were running multiple times because the
CI uses category-based test filtering with OR logic. For example, a test
with `[Category("Button")]` and `[Category("Compatibility")]` would run
in both the Button job and the Compatibility job.
**Before this PR:**
- ~314 tests had multiple categories
- Each ran 2-3x per platform depending on category count
- Estimated **~1,256 duplicate test executions** per CI run (314 × ~1
extra × 4 platforms)
**After this PR:**
- Every test runs exactly once per platform
- Build fails if a developer adds multiple categories
- Significant CI time savings
## Testing
- [x] All 314 affected tests updated
- [x] Test project builds successfully with 0 errors
- [x] Analyzer unit tests pass (20/20)
- [x] Verified analyzer catches both missing and multiple category
violations
---------
Co-authored-by: Copilot <[email protected]>
Co-authored-by: rmarinho <[email protected]>
Co-authored-by: Shane Neuville <[email protected]>1 parent a213790 commit b494233
File tree
167 files changed
+1377
-200
lines changed- src
- TestUtils
- src/UITest.Analyzers/NUnit
- tests/UITest.Analyzers.Tests
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
167 files changed
+1377
-200
lines changedLarge diffs are not rendered by default.
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
337 | 337 | | |
338 | 338 | | |
339 | 339 | | |
340 | | - | |
341 | 340 | | |
342 | 341 | | |
343 | 342 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
23 | 22 | | |
24 | 23 | | |
25 | 24 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | 9 | | |
11 | 10 | | |
12 | 11 | | |
| |||
Lines changed: 1 addition & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
19 | | - | |
| 18 | + | |
20 | 19 | | |
21 | 20 | | |
22 | 21 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
| |||
0 commit comments