Skip to content

Commit b494233

Browse files
jfversluisCopilotrmarinhoPureWeen
authored
Enforce exactly one Category attribute per UI test (#33242)
## Description This PR enforces that every UI test has **exactly one** `[Category]` attribute through a build-time analyzer. Previously, tests could have multiple categories which caused them to run multiple times in CI. Each test needs to have a category so they are ran in the CI pipeline. With this analyzer we ensure that this is not forgotten during development. ## Changes ### Analyzer Updates (`NUnitTestMissingCategoryAnalyzer`) - **MAUI0001**: Build error when a test has **no** category - **MAUI0002**: Build error when a test has **more than one** category - Changed severity from `Warning` to `Error` - Excludes platform-specific ignore attributes (`FailsOnAndroidWhenRunningOnXamarinUITest`, etc.) from the category count since they conditionally derive from `CategoryAttribute` ### Fixed 314 Tests All tests with multiple categories have been fixed by keeping the most appropriate single category: - **Control-specific categories** preferred (Button, Entry, CollectionView, ListView, etc.) - **Feature categories** kept when no control-specific category applied (Navigation, Shell, Gestures, etc.) - **Removed generic categories** like `Compatibility` in favor of more specific ones ### Added Analyzer Unit Tests New test project `UITest.Analyzers.Tests` with 20 tests covering: - Missing category detection - Multiple categories detection - Custom `CategoryAttribute` derivatives - Platform ignore attribute exclusions - Edge cases ## CI Impact Tests with multiple categories were running multiple times because the CI uses category-based test filtering with OR logic. For example, a test with `[Category("Button")]` and `[Category("Compatibility")]` would run in both the Button job and the Compatibility job. **Before this PR:** - ~314 tests had multiple categories - Each ran 2-3x per platform depending on category count - Estimated **~1,256 duplicate test executions** per CI run (314 × ~1 extra × 4 platforms) **After this PR:** - Every test runs exactly once per platform - Build fails if a developer adds multiple categories - Significant CI time savings ## Testing - [x] All 314 affected tests updated - [x] Test project builds successfully with 0 errors - [x] Analyzer unit tests pass (20/20) - [x] Verified analyzer catches both missing and multiple category violations --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: rmarinho <[email protected]> Co-authored-by: Shane Neuville <[email protected]>
1 parent a213790 commit b494233

File tree

167 files changed

+1377
-200
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

167 files changed

+1377
-200
lines changed

Microsoft.Maui-dev.sln

Lines changed: 735 additions & 0 deletions
Large diffs are not rendered by default.

src/Controls/tests/TestCases.Shared.Tests/Tests/FeatureMatrix/GraphicsViewFeatureTests.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -337,7 +337,6 @@ public void GraphicsView_EndInteraction_VerifyEventTriggered()
337337

338338
[Test]
339339
[Category(UITestCategories.GraphicsView)]
340-
[Category(UITestCategories.Gestures)]
341340
public void GraphicsView_DragInteraction_VerifyEventTriggered()
342341
{
343342
App.WaitForElement("ClearEventsButton");

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla23942.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ public Bugzilla23942(TestDevice testDevice) : base(testDevice)
1414

1515
[Test]
1616
[Category(UITestCategories.LifeCycle)]
17-
[Category(UITestCategories.Compatibility)]
1817
public void Bugzilla23942Test()
1918
{
2019
App.WaitForElement("success");

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla25943.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ public Bugzilla25943(TestDevice testDevice) : base(testDevice)
1919

2020
[Test]
2121
[Category(UITestCategories.Gestures)]
22-
[Category(UITestCategories.Compatibility)]
2322
public void VerifyNestedStacklayoutTapsBubble()
2423
{
2524
App.WaitForElement(InnerLayout);

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla28570.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@
66
namespace Microsoft.Maui.TestCases.Tests.Issues
77
{
88
[Category(UITestCategories.ScrollView)]
9-
[Category(UITestCategories.Compatibility)]
109
public class Bugzilla28570UITests : _IssuesUITest
1110
{
1211
public Bugzilla28570UITests(TestDevice device)

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla29128.cs

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,7 @@ public Bugzilla29128(TestDevice testDevice) : base(testDevice)
1515
public override string Issue => "Slider background lays out wrong Android";
1616

1717
[Test]
18-
[Category(UITestCategories.LifeCycle)]
19-
[Category(UITestCategories.Compatibility)]
18+
[Category(UITestCategories.Slider)]
2019
public void Bugzilla29128Test()
2120
{
2221
App.WaitForElement("SliderId");

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla29363.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ public Bugzilla29363(TestDevice testDevice) : base(testDevice)
1414

1515
[Test]
1616
[Category(UITestCategories.Navigation)]
17-
[Category(UITestCategories.Compatibility)]
1817
public void PushButton()
1918
{
2019
App.WaitForElement("ModalPushPopTest");

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla29453.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ public Bugzilla29453(TestDevice testDevice) : base(testDevice)
1414

1515
[Test]
1616
[Category(UITestCategories.Navigation)]
17-
[Category(UITestCategories.Compatibility)]
1817
public void Bugzilla29453Test()
1918
{
2019
App.WaitForElement("Page1");

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla30166.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ public Bugzilla30166(TestDevice testDevice) : base(testDevice)
1414

1515
[Test]
1616
[Category(UITestCategories.Navigation)]
17-
[Category(UITestCategories.Compatibility)]
1817
public void Bugzilla30166Test()
1918
{
2019
App.WaitForElement("PushModal");

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla30935.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ public Bugzilla30935(TestDevice testDevice) : base(testDevice)
1414

1515
[Test]
1616
[Category(UITestCategories.Page)]
17-
[Category(UITestCategories.Compatibility)]
1817
public void Bugzilla30935DoesntThrowException()
1918
{
2019
App.WaitForNoElement("IssuePageLabel");

0 commit comments

Comments
 (0)