Try Microsoft testing platform integration #6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Unit Tests macos-latest0 tests 0 ✅ 0s ⏱️ Results for commit 02b41f9. ♻️ This comment has been updated with latest results. |
Unit Tests ubuntu-latest0 tests 0 ✅ 0s ⏱️ Results for commit 02b41f9. ♻️ This comment has been updated with latest results. |
Unit Tests windows-latest 8 files 8 suites 5m 34s ⏱️ Results for commit a2f3c97. ♻️ This comment has been updated with latest results. |
There was a problem hiding this comment.
Pull Request Overview
Integrate the Microsoft Testing Platform into Coverlet by adding extension hooks, environment variable and command-line providers, and a test host process lifetime handler for coverage collection. Update CI/CD configurations and documentation to support .NET 8/9.
- Introduce
coverlet.MTPproject with builder hook and logger adapter - Implement environment variable and command-line providers plus a process lifetime handler
- Update global SDK version, CI pipelines, and documentation
Reviewed Changes
Copilot reviewed 63 out of 63 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coverlet.MTP/TestingPlatformBuilderHook.cs | Add static hook to register Coverlet extension in test builder |
| src/coverlet.MTP/CoverletExtensionProvider.cs | Register environment, process handler, and CLI providers |
| src/coverlet.MTP/Logging/CoverletLoggerAdapter.cs | Adapter for Microsoft Testing Platform logger |
| src/coverlet.MTP/CoverletExtensionEnvironmentVariableProvider.cs | Stub environment variable provider |
| src/coverlet.MTP/CoverletExtensionConfiguration.cs | Configuration model and helper class for CLI options |
| src/coverlet.MTP/CoverletExtensionCommandLineProvider.cs | Define and validate command-line options |
| src/coverlet.MTP/CoverletExtensionCollector.cs | Implement coverage instrumentation before/after test run |
| global.json | Bump SDK version to 9.0.300 |
| eng/*.yml | Update build and publish pipelines to .NET 8/9 and log patterns |
| Directory.*.props | Update package versions and introduce new TFMs |
| .github/workflows/dotnet.yml | Add GitHub Actions workflow for .NET 8/9 |
| README.md and Documentation/*.md | Update docs for .NET 8/9 support and new settings |
Comments suppressed due to low confidence (7)
src/coverlet.MTP/Collector/CoverletExtensionCollector.cs:113
- Checking for null on 'OutputDirectory' doesn't catch empty strings; if it's empty, 'Path.GetDirectoryName' returns null and causes a crash. Use 'string.IsNullOrEmpty' or a default fallback.
string dOutput = _configuration.OutputDirectory != null ? _configuration.OutputDirectory : Directory.GetCurrentDirectory() + Path.DirectorySeparatorChar.ToString();
src/coverlet.MTP/TestingPlatformBuilderHook.cs:15
- [nitpick] The parameter name '_' is ambiguous and reduces readability; consider renaming it to something descriptive like 'args' or 'commandLineArgs'.
public static void AddExtensions(ITestApplicationBuilder testApplicationBuilder, string[] _)
src/coverlet.MTP/CoverletExtensionConfiguration.cs:20
- [nitpick] Property 'sourceMappingFile' does not follow PascalCase convention; rename to 'SourceMappingFile' for consistency with other properties.
public string sourceMappingFile { get; set; } = string.Empty;
src/coverlet.MTP/CoverletExtensionConfiguration.cs:22
- [nitpick] Property 'formats' should be PascalCase (e.g., 'Formats') to match .NET naming conventions.
public string[] formats { get; set; } = ["json"];
eng/build.yml:30
- [nitpick] Mixed forward and backward slashes in the project path can be confusing; use one consistent style (preferably forward slashes in YAML).
dotnet test test/coverlet.msbuild.tasks.tests\coverlet.msbuild.tasks.tests.csproj -c $(BuildConfiguration) --no-build -bl:test.msbuild.tasks.binlog ...
eng/publish-coverlet-result-files.yml:42
- Switching from '**/.binlog' to '.binlog' will miss binlog files in subdirectories. Revert to the recursive pattern or adjust as needed.
Contents: '*.binlog'
.github/workflows/dotnet.yml:34
- Installing .NET 9.0 then immediately installing .NET 8.0 overrides the first setup. If multi-version validation is intended, run tests in separate jobs or remove one install step.
- name: Setup .NET 9.0
- uses: actions/setup-dotnet@v4
- with:
- dotnet-version: 9.0.x
+ - name: Setup .NET 8.0
Comment on lines
17
to
24
| CoverletExtensionConfiguration coverletExtensionConfiguration = new(); | ||
| if (ignoreIfNotSupported) | ||
| { | ||
| #if !NETCOREAPP | ||
| coverletExtensionConfiguration.Enable =false; | ||
| #endif | ||
| } | ||
|
|
There was a problem hiding this comment.
The 'ignoreIfNotSupported' flag sets 'Enable' on the configuration but the configuration is never used; either apply the flag to skip registration or remove the unused parameter.
Suggested change
| CoverletExtensionConfiguration coverletExtensionConfiguration = new(); | |
| if (ignoreIfNotSupported) | |
| { | |
| #if !NETCOREAPP | |
| coverletExtensionConfiguration.Enable =false; | |
| #endif | |
| } |
Comment on lines
+12
to
+14
| #pragma warning disable CS9113 // Parameter is unread. | ||
| internal sealed class CoverletExtensionEnvironmentVariableProvider(IConfiguration configuration, Platform.CommandLine.ICommandLineOptions commandLineOptions, ILoggerFactory loggerFactory) : ITestHostEnvironmentVariableProvider | ||
| #pragma warning restore CS9113 // Parameter is unread. |
There was a problem hiding this comment.
Several constructor parameters are unused and suppressed via pragmas; remove unused parameters or implement the environment-variable logic for clarity.
Suggested change
| #pragma warning disable CS9113 // Parameter is unread. | |
| internal sealed class CoverletExtensionEnvironmentVariableProvider(IConfiguration configuration, Platform.CommandLine.ICommandLineOptions commandLineOptions, ILoggerFactory loggerFactory) : ITestHostEnvironmentVariableProvider | |
| #pragma warning restore CS9113 // Parameter is unread. | |
| internal sealed class CoverletExtensionEnvironmentVariableProvider : ITestHostEnvironmentVariableProvider |
| public void InitializeFromCommandLineArgs() | ||
| { | ||
| IServiceCollection serviceCollection = new ServiceCollection(); | ||
| ServiceProvider serviceProvider = serviceCollection.BuildServiceProvider(); |
There was a problem hiding this comment.
The locally built 'serviceProvider' is never used; the method then uses '_serviceProvider' instead. Either use or remove the local provider.
Suggested change
| ServiceProvider serviceProvider = serviceCollection.BuildServiceProvider(); | |
| // Removed unused local variable 'serviceProvider'. |
global.json
Outdated
| { | ||
| "sdk": { | ||
| "version": "8.0.407" | ||
| "version": "9.0.300" |
There was a problem hiding this comment.
The global.json SDK version (9.0.300) doesn’t match the SDK used in build YAML (8.0.408); align these versions to avoid confusion.
Suggested change
| "version": "9.0.300" | |
| "version": "8.0.408" |
SummarySummary
Coveragecoverletsample.integration.determisticbuild - 100%
|
Bertk
force-pushed
the
coverlet-MTP
branch
2 times, most recently
from
04156f6 to
941929d
Compare
Bertk
force-pushed
the
coverlet-MTP
branch
2 times, most recently
from
592ff6a to
010a409
Compare
- Updated target frameworks to net472 in coverlet.core and coverlet.msbuild.tasks projects - Adjusted CoverletToolsPath for multi-targeting support in buildMultiTargeting props and targets - Created unit tests for Coverlet.MTP command line options validation - Added documentation for Coverlet.MTP integration
- Added CoverletExtensionCollector to handle test session lifecycle for coverage collection. - Introduced CoverletExtensionCommandLineProvider for command line options. - Created CoverletExtensionConfiguration to manage configuration settings. - Developed CoverletLoggerAdapter for logging integration with Microsoft Testing Platform. - Implemented CoverletExtensionEnvironmentVariableProvider for environment variable management. - Added CoverletExtensionProvider to register the Coverlet extension with the testing platform. - Created TestingPlatformBuilderHook to facilitate extension registration. - Updated project files to include necessary dependencies and configurations for Coverlet. - Added support for multiple target frameworks (net8.0 and net9.0). - Included build and packaging configurations for Coverlet.MTP. - Implemented command line options for coverage report formats and exclusions. - Established logging mechanisms for better traceability during coverage collection.
…ng and add coverage cleanup messages
…ationString for consistency and lowercase output
… consistency across environments
- Expanded MTP integration docs and added usage ToDo - Enabled package generation in coverlet.MTP.csproj - Added strong name signing for validation tests - Updated validation test project: new dependencies, utility refs, and test project copying - Updated InternalsVisibleTo for new test assemblies
Introduce a comprehensive validation test suite for the Coverlet Microsoft Testing Platform (MTP) extension, including new integration and CLI option tests. Add isolated test infrastructure with sample projects and custom MSBuild props/targets to ensure tests run in a pure MTP environment. Update packaging logic in coverlet.MTP to improve dependency handling and NuGet layout, and adjust build system to better support MTP scenarios. Also includes minor bug fixes and ensures correct local package versioning in tests.
- Enhance test output with detailed diagnostics and error context
- Use robust assertions with informative failure messages
- Create temp test projects under artifacts/tmp for isolation
- Add retries and file attribute handling to test dir cleanup
- Check for test executable and coverlet.MTP.dll before running
- Refactor HelpCommandTests for consistent path handling
- Fix sample class naming mismatch in test project
- Add condition to MSBuild import for props file robustness
- Update NuGet config and project file handling for clarity
Renamed the constant sutName to SutName in HelpCommandTests to follow .NET PascalCase naming conventions for constants. Updated all references to use the new name for consistency.
Switch all test projects to xunit.v3.mtp-v2 for Microsoft Testing Platform (MTP) integration. Bump Microsoft.Testing.Platform to 2.0.2 and manage Moq version via property. Make Coverlet MTP extension opt-in by default and register as TestingPlatformExtension. Minor formatting and encoding adjustments included.
Replaced return path; with return path!; in two locations to suppress nullable reference type warnings. This clarifies to the compiler that path is guaranteed non-null at those points. No changes to program logic.
Updated test execution scripts to always use --diagnostic-verbosity trace for MTP test runs, ensuring maximum diagnostic detail. Adjusted dotnet test --diag output paths to include the build configuration subdirectory for better log organization and to prevent file overwrites. Set diagnostic verbosity to trace for coverlet.core.coverage.tests. These changes standardize and enhance diagnostic logging, making CI troubleshooting easier.
Update test projects to use the Microsoft Testing Platform runner by setting <UseMicrosoftTestingPlatformRunner> to true and adding necessary package references. Also, add xunit.v3 to Directory.Packages.props and update .gitignore to exclude MTPTest.props. These changes modernize the test infrastructure and improve compatibility with the latest tooling.
Added Microsoft.Testing.Extensions.TrxReport.Abstractions to central and project dependencies. Updated test project to use xunit.v3.mtp-v2 and explicitly reference Microsoft.Testing.Platform 2.0.2 for compatibility with MTP v2.x.
- Rename --coverage to --coverlet-coverage and update all related option names for consistency and clarity (e.g., --formats, --exclude, --include). - Introduce CoverletOptionNames static class to centralize option name constants. - Update command-line providers to use new option names and improve descriptions. - Add validation for output formats; deprecate/comment out output path option. - Ensure help output always shows options; only collect coverage if --coverlet-coverage is set. - Update tests to use new flags and add help output verification. - Remove redundant code and improve comments for maintainability. - Update csproj to clean up package references and add local NuGet restore source.
Refactor CoverletExtensionCollector to use ITestHostProcessLifetimeHandler, moving instrumentation and report generation to new lifecycle methods. Modernize configuration handling and command-line parsing. Move InternalsVisibleTo attributes to AssemblyInfo.cs. Update tests for new APIs and improved diagnostics. Clean up obsolete code and improve maintainability.
- Allow multiple --formats options (ArgumentArity.OneOrMore) instead of comma-separated values - Enforce single, supported format per --formats argument - Update tests to use repeated --formats options - Expand and clarify process exit code to error message mapping
- Add BasicNoCoverage_CollectsDataForCoveredLines test to verify test execution without coverage collection. - Enhance Assert.True error messages to include process ErrorText for better debugging. - Update generated test project to reference TrxReport and explicitly include Tests.cs. - Refactor BranchTests to use explicit string type for result variables. - Minor code style and message consistency improvements.
Add detailed diagnostic logging and debugger attach support via environment variables. Enhance test module path resolution and configuration logging. Refactor DI container creation and introduce MtpProcessExitHandler for in/out-of-process execution. Add default exclude filter, improve report output handling, and expand command line parsing. Integrate Microsoft.Extensions.Configuration and update test project output path for consistency.
Introduce InstrumentationDiagnostics.cs to provide centralized, async diagnostic logging for code instrumentation and coverage collection. Logs instrumented modules, excluded modules, coverage result generation, and errors using ILogger, improving transparency and troubleshooting for Coverlet users. No changes to existing code; this is a new utility class.
Refactor to introduce CoverletCommandLineOptionDefinitions as the single source of truth for all command-line options, eliminating duplication and inconsistencies. Remove CoverletCommandLineOptionsProvider and update CoverletExtensionCommandLineProvider to use the centralized definitions and validation. Rename and clean up option constants in CoverletOptionNames. Update CoverageConfiguration and related logging for new source mapping file handling. Adjust tests and namespaces for consistency with new option names and structure.
- Refactor InstrumentationHelper backup/restore logic: - Skip and log if a module or symbol file is already in the backup list instead of throwing. - Check for backup existence before restore; log warnings or verbose messages as needed. - Add IsCurrentlyRunningAssembly to avoid restoring the running assembly. - Log the number of modules to restore and skip missing or running assemblies. - Update CoverletMTPCommandLineTests to match the current set of supported options. - Update HelpCommandTests to reflect new, clearer option descriptions.
Refactor build pipeline to use dotnet test for coverlet.MTP.unit.tests with MSBuild-based code coverage collection and opencover output. Increase minimum required line coverage from 70% to 90%. Update coverlet.MTP.unit.tests.csproj to include necessary imports and package references for coverage integration. Add comments for alternative coverage approaches and modernize test execution paths.
- Add support for reading Coverlet settings from appsettings.json or any Microsoft.Extensions.Configuration source. - Introduce CoverletMTPSettings, CoverletMTPConstants, and CoverletMTPSettingsParser for strongly-typed configuration and parsing. - Add CoverletConfigurationExtensions for easy registration with MTP. - Implement CoverletTestSessionHandler to use parsed configuration. - Rename all command line options from coverage-* to coverlet-* for consistency. - Enable and test the SourceMappingFile option. - Update and expand tests to cover configuration parsing, defaults, and new option names. - Update build/test pipeline and project files to support new config and cross-platform test execution.
- Add detailed Coverlet.MTP integration documentation and usage guide - Refactor CoverletExtensionCollector for clarity, logging, and env var handling - Overhaul test infra: generate SUT + test projects in solution, robust cleanup, and real-world structure - Improve coverage validation: assert on actual data, not just file presence - Add/configure .mcp.json for NuGet MCP server - Minor fixes: exit code handling, error messages, and CoverletTestSessionHandler interface compliance
Explicitly use ./ and .\ prefixes for test executables in build.yml scripts to ensure correct execution on all platforms. This improves reliability of unit test runs by avoiding path resolution issues on different operating systems.
- Add comprehensive CoverageConfigurationTests.cs for all config options using Moq - Fix devcontainer.json syntax and ensure GitHub Actions extension is included - Enable Dependabot for GitHub Actions and NuGet with custom schedule - Improve Coverlet.MTP.Integration.md: typo fix, clarify usage, update sample project and coverage command - Simplify build.yml by unifying test execution to cross-platform dotnet exec - Make CoverletDataCollector version dynamic from assembly - Remove default CoverletMTPEnabled property and make props import conditional - Set IsTestingPlatformApplication to false in csproj and tidy formatting
- Eliminate --coverlet-output and --coverlet-source-mapping-file options from Coverlet.MTP integration, code, and docs. - Coverage output directory now determined by Microsoft Testing Platform's test result directory. - Update documentation and help output to reflect new behavior and remove obsolete references. - Remove CoverletDataCollector class; rely on CoverletExtensionCollector and MTP extension model. - Update build script and test projects for new output handling and configuration. - Improve documentation examples for merging and reporting coverage. - Refine table formatting and minor test project settings.
Introduce CoverletExtensionCollectorTests to comprehensively verify construction, option parsing, environment variable handling, and integration with Microsoft Testing Platform extension interfaces. Tests cover normal and edge cases, ensuring correct behavior without modifying production code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.