Repository Quality Improvement — Public API XML Documentation Coverage

[Evangelink]

🎯 Repository Quality Improvement Report — Public API XML Documentation

Analysis Date: 2026-06-11 | Focus Area: Documentation (Standard) | Strategy: Standard category selected to diversify after 7 consecutive custom runs

Executive Summary

Analysis of XML doc comment (///) coverage on public APIs reveals a critical gap in MSTest's Assert class — the single most heavily used API in any test suite. With only 1.4% coverage (6 of 434 Assert methods documented), developers get no IntelliSense tooltips on Assert.AreEqual, Assert.IsTrue, Assert.ThrowsException, etc. Core attributes (TestClassAttribute, TestMethodAttribute, IgnoreAttribute) are also undocumented. GenerateDocumentationFile is already enabled globally and CS1591 warnings are active but not build-breaking, so improvements can be made incrementally. The Microsoft.Testing.Platform component is in much better shape (89.2% type coverage), confirming the team can and does write quality XML docs.

Full Analysis Report

Current State

Metric	Value	Status
TestFramework overall member coverage	162 / 779 (20.8%)	❌
TestFramework type-level coverage	64 / 131 types (48.9%)	⚠️
Assert / CollectionAssert method coverage	6 / 434 (1.4%)	❌
Core attribute types documented	13 / 31 (42%)	⚠️
MTP type-level coverage	116 / 130 (89.2%)	✅
MTP overall member coverage	372 / 1,757 (21.2%)	❌
CS1591 warnings enabled	true (SkipArcadeNoWarnCS1591)	✅
CS1591 treated as build error	false	⚠️

Top undocumented Assert files (by method count):

• Assert.IsExactInstanceOfType.cs — 46 undocumented
• Assert.IsInstanceOfType.cs — 46 undocumented
• Assert.AreEqual.InterpolatedStringHandlers.cs — 40 undocumented
• Assert.ThrowsException.cs — 39 undocumented
• Assert.Count.cs — 29 undocumented; Assert.AreSame.cs, Assert.IsNull.cs, Assert.IsTrue.cs — 24 each
• StringAssert.cs — 18 undocumented

Undocumented critical attribute types (18 total): TestClassAttribute, TestMethodAttribute, IgnoreAttribute, RetryAttribute, RetryBaseAttribute, TestCategoryBaseAttribute, ConditionBaseAttribute, CIConditionAttribute, OSConditionAttribute, DataTestMethodAttribute, DataSourceAttribute, GlobalTestInitializeAttribute, GlobalTestCleanupAttribute, DiscoverInternalsAttribute, InheritanceBehavior, AssemblyFixtureProviderAttribute, TestDataSourceOptionsAttribute, TestResult

Undocumented MTP types (14): IBannerMessageOwnerCapability, IGracefulStopTestExecutionCapability, ArgumentArity, CompositeExtensionFactory, CancelledTestNodeStateProperty, TestNodeUpdateMessage, EnvironmentVariable, OwnedEnvironmentVariable, ExecuteRequestContext, HostProperties, RequestProperties, ApplicationMode, TestNodeDetails, CommandLineParseOption

---

🤖 Suggested Improvement Tasks

Task 1: Document Core Assert Methods (IsTrue/IsFalse, IsNull/IsNotNull, AreSame/AreNotSame)

Priority: High | Effort: Small

Add <summary>, <param>, and <exception> XML doc comments to the ~72 overloads in Assert.IsTrue.cs, Assert.IsNull.cs, Assert.AreSame.cs. Use TestContext.cs as the style model. These are the most called Assert methods in the ecosystem.

---

Task 2: Document AreEqual, ThrowsException, and StringAssert Families

Priority: High | Effort: Medium

Document the next-highest-impact groups: Assert.AreEqual.cs (8 overloads), Assert.AreEqual.Numerics.cs (16 — delta overloads), Assert.AreEqual.String.cs (8), Assert.AreEqual.InterpolatedStringHandlers.cs (40 handler types), Assert.ThrowsException.cs (39 overloads incl. async), StringAssert.cs (18). For numeric delta overloads, clarify: "Maximum permitted difference between expected and actual."

---

Task 3: Document All Core MSTest Attribute Types

Priority: High | Effort: Small–Medium

Add <summary> to the 18 undocumented attribute types listed above. Prioritize: TestClassAttribute, TestMethodAttribute, IgnoreAttribute, RetryAttribute (used in every project), then lifecycle attributes (GlobalTestInitializeAttribute, GlobalTestCleanupAttribute), then data attributes (DataTestMethodAttribute, DataSourceAttribute).

---

Task 4: Document Remaining Assert Methods and 14 MTP Types

Priority: Medium | Effort: Medium

4a — MTP (14 types in src/Platform/Microsoft.Testing.Platform/): Focus on contract-forming types first: ArgumentArity, CompositeExtensionFactory, EnvironmentVariable, OwnedEnvironmentVariable, ExecuteRequestContext, CancelledTestNodeStateProperty.

4b — Remaining Assert families: CollectionAssert.cs/Membership.cs/Subset.cs, Assert.Count.cs, Assert.Contains.cs, Assert.DoesNotContain.cs, Assert.ContainsSingle.cs, Assert.StartsWith.cs, Assert.EndsWith.cs, Assert.Matches.cs, Assert.IsInRange.cs.

---

Task 5: Enforce CS1591 as a Build Error for Public API Projects

Priority: Medium | Effort: Small (config)

After Tasks 1–4 substantially close the gap, prevent future regressions by adding to Microsoft.VisualStudio.TestTools.UnitTesting.csproj and Microsoft.Testing.Platform.csproj:

<WarningsAsErrors>$(WarningsAsErrors);CS1591</WarningsAsErrors>

Track progress by counting remaining CS1591 warnings after each task.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type
2026-05-22	test-framework-api-ergonomics	Custom
2026-05-25	agentic-workflow-maintainability	Custom
2026-05-26	workflow-ecosystem-health	Custom
2026-05-27	test-diagnostic-experience	Custom
2026-06-08	todo-comment-policy-compliance	Custom
2026-06-09	mstest-diagnostic-id-governance	Custom
2026-06-10	analyzer-code-fix-coverage-gap	Custom
2026-06-11	public-api-xml-documentation	Standard

Next analysis: 2026-06-12 — Focus area selected based on diversity algorithm

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Repository Quality Improver workflow. · 264.6 AIC · ⌖ 14 AIC · [◷]( · ◷)

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Jun 13, 2026, 10:57 PM UTC