Skip to content

Repository Quality: MSTest Analyzer Ecosystem Health ImprovementsΒ #7951

@Evangelink

Description

@Evangelink

🎯 Repository Quality Improvement Report β€” MSTest Analyzer Ecosystem Health

Analysis Date: 2026-04-30
Focus Area: MSTest Analyzer Ecosystem Health
Strategy Type: Custom (repository-specific)

Executive Summary

This analysis examined the health of the MSTest Roslyn analyzer ecosystem β€” including release note accuracy, test file naming consistency, and release note discrepancies. The analyzer suite is impressively large (54 analyzers, 37 code fixers, 4 suppressors, ~63 shipped rules), reflecting a mature and well-maintained testing guidance system.

Several low-effort, high-impact improvements were identified. Three category typos (USage instead of Usage) appear in the AnalyzerReleases.Shipped.md "Changed Rules" table for Release 4.1.0. One rule entry uses backtick-wrapped formatting (`Usage`) inconsistent with surrounding entries. Two analyzer names in the release notes do not match the actual C# class names. One test file is missing the standard Tests suffix in its filename.

These are all small, targeted fixes that improve the discoverability and trustworthiness of the analyzer documentation used both internally and by tooling like the Roslyn analyzer infrastructure.

Full Analysis Report

Focus Area: MSTest Analyzer Ecosystem Health

Current State Assessment

Metrics Collected:

Metric Value Status
Total analyzer files 54 βœ…
Total code fixer files 37 ⚠️
Total suppressor files 4 βœ…
Shipped rules in release notes 63+ βœ…
Category name typos in release notes 3 (USage) ❌
Formatting inconsistencies in release notes 1 (backtick) ⚠️
Incorrect analyzer class names in release notes 2 ⚠️
Misnamed test files (missing Tests suffix) 1 ⚠️

Findings

Strengths

  • Excellent rule density: 63 shipped rules covering a wide range of MSTest usage patterns
  • Nearly all rules have corresponding unit tests
  • Code fixes provided for the majority of actionable diagnostics
  • All resource strings are properly localized (no raw string literals in diagnostic messages)
  • Clear documentation links on every rule entry

Areas for Improvement

❌ Critical β€” Typos in AnalyzerReleases.Shipped.md

In the "Changed Rules" table for Release 4.1.0 (lines 19–21), the New Category column shows USage instead of Usage for three rules:

MSTEST0023 | USage | Info  | Usage | Warning | DoNotNegateBooleanAssertionAnalyzer
MSTEST0037 | USage | Info  | Usage | Warning | UseProperAssertMethodsAnalyzer
MSTEST0045 | USage | Info  | Usage | Warning | UseCooperativeCancellationForTimeoutAnalyzer
```

**⚠️ Formatting inconsistency in `AnalyzerReleases.Shipped.md`**

In the "New Rules" table for **Release 3.7.0** (line 118), the category value uses backticks:

```
MSTEST0037 | `Usage` | Info | UseProperAssertMethodsAnalyzer, ...
```

All other entries use plain `Usage` without backticks.

**⚠️ Incorrect analyzer class names in `AnalyzerReleases.Shipped.md`**

Two entries in the release notes reference class names that don't match the actual source files:

| Rule | Name in release notes | Actual class name |
|------|-----------------------|-------------------|
| MSTEST0052 (Release 3.11.0) | `PreferDynamicDataSourceTypeAutoDetectAnalyzer` | `AvoidExplicitDynamicDataSourceTypeAnalyzer` |
| MSTEST0059 (Release 4.1.0) | `DoNotUseParallelizeAndDoNotParallelizeTogetherAnalyzer` | `UseParallelizeAttributeAnalyzer` (shared) |

**⚠️ Misnamed test file**

`test/UnitTests/MSTest.Analyzers.UnitTests/TypeContainingTestMethodShouldBeATestClassAnalyzer.cs`

The file contains the `TypeContainingTestMethodShouldBeATestClassAnalyzerTests` class and is a tests file, but is missing the `Tests` suffix in its filename. All other test files in this directory follow the `*AnalyzerTests.cs` / `*Tests.cs` convention.

</details>

---

### πŸ€– Suggested Improvement Tasks

The following actionable tasks address the findings above.

#### Task 1: Fix `USage` typos in `AnalyzerReleases.Shipped.md`

**Priority**: Medium
**Estimated Effort**: Small

In `src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Shipped.md`, the Release 4.1.0 "Changed Rules" table has three rows where the `New Category` column reads `USage` instead of `Usage`:

- Line 19: `MSTEST0023 | USage | Info | Usage | Warning | DoNotNegateBooleanAssertionAnalyzer`
- Line 20: `MSTEST0037 | USage | Info | Usage | Warning | UseProperAssertMethodsAnalyzer`
- Line 21: `MSTEST0045 | USage | Info | Usage | Warning | UseCooperativeCancellationForTimeoutAnalyzer`

Each `USage` should be corrected to `Usage`.

---

#### Task 2: Fix backtick formatting for MSTEST0037 in Release 3.7.0

**Priority**: Low
**Estimated Effort**: Small

In `src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Shipped.md`, the Release 3.7.0 entry for MSTEST0037 uses backtick-wrapped formatting:

```
MSTEST0037 | `Usage` | Info | UseProperAssertMethodsAnalyzer, ...

This should be changed to plain Usage (no backticks) to be consistent with all other entries in the file.


Task 3: Correct analyzer class names in release notes

Priority: Low
Estimated Effort: Small

Two entries in src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Shipped.md reference class names that do not match the actual source files:

  1. Release 3.11.0, MSTEST0052: Listed as PreferDynamicDataSourceTypeAutoDetectAnalyzer β€” should be AvoidExplicitDynamicDataSourceTypeAnalyzer (the actual class in src/Analyzers/MSTest.Analyzers/AvoidExplicitDynamicDataSourceTypeAnalyzer.cs).

  2. Release 4.1.0, MSTEST0059: Listed as DoNotUseParallelizeAndDoNotParallelizeTogetherAnalyzer β€” this diagnostic is actually defined within UseParallelizeAttributeAnalyzer (in src/Analyzers/MSTest.Analyzers/UseParallelizeAttributeAnalyzer.cs). The notes entry should reflect the actual analyzer class.


Task 4: Rename misnamed analyzer test file

Priority: Low
Estimated Effort: Small

test/UnitTests/MSTest.Analyzers.UnitTests/TypeContainingTestMethodShouldBeATestClassAnalyzer.cs should be renamed to TypeContainingTestMethodShouldBeATestClassAnalyzerTests.cs to follow the naming convention used by all other test files in the same directory. The class inside the file is already correctly named TypeContainingTestMethodShouldBeATestClassAnalyzerTests.


πŸ“Š Historical Context

Previous Focus Areas
Date Focus Area Type
2026-04-30 MSTest Analyzer Ecosystem Health Custom

🎯 Recommendations

Immediate Actions (This Week)

  1. Fix the 3 USage β†’ Usage typos in AnalyzerReleases.Shipped.md (Release 4.1.0) β€” Priority: Medium
  2. Rename TypeContainingTestMethodShouldBeATestClassAnalyzer.cs to TypeContainingTestMethodShouldBeATestClassAnalyzerTests.cs β€” Priority: Low

Short-term Actions (This Month)

  1. Correct the analyzer class names for MSTEST0052 and MSTEST0059 in the release notes β€” Priority: Low
  2. Fix the backtick formatting for MSTEST0037 in Release 3.7.0 β€” Priority: Low

Generated by Repository Quality Improvement Agent
Next analysis: 2026-05-01 β€” Focus area selected based on diversity algorithm

Generated by Repository Quality Improver Β· ● 1.6M Β· β—·

  • expires on May 2, 2026, 6:18 AM UTC

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions