Class name inconsistency in Code Coverage Report

[Glober777]

1. General summary of the issue

While testing out new Code Coverage Reporting capability in Azure DevOps with a dummy PowerShell module, I've noticed, that if my module has just a single cmdlet implemented, the class name property is generated based of the module path, and as soon as I add more cmdlets to the module, the class name in the code coverage report, becomes prefixed with the parent folder's name of the module:

• Single module member:
  

• Multiple module members:
  

2. Describe Your Environment

Pester version     : 4.8.1 C:\Users\seriavio\Documents\WindowsPowerShell\Modules\Pester\4.8.1\Pester.psd1
PowerShell version : 5.1.17763.503
OS version         : Microsoft Windows NT 10.0.17763.0

3. Expected Behavior

Expected behavior would be to report the class name consistently (either with or without the parent folder name)

4.Current Behavior

Class name is affected by the number of module members

5. Possible Solution

6. Context

[mikeclayton]

I've not really used the Azure DevOps Code Coverage feature, so I can't tell if this is a Pester issue or maybe something to do with the Azure DevOps Code Coverage feature itself.

However, I've tried to reproduce this locally using Pester's Code Coverage reports, and failed :-(.

Could you check your scenario against the sample files below and see how they compare? Note that the "Class" attribute in the resulting code coverage json is an empty string in my examples so I'm maybe not using equivalent code to yours...

test-single-cmdlet.psm1

function Invoke-MySingleCmdlet
{
    return $true;
}

test-single-cmdlet.tests.ps1

Describe "single cmdlet tests" {
    It "should run single" {
        Invoke-MySingleCmdlet | Should Be $true;
    }
}

test-multiple-cmdlets.psm1

function Invoke-MyFirstCmdlet
{
    return $true;
}
function Invoke-MySecondCmdlet
{
    return $true;
}

test-multiple-cmdlets.tests.ps1

Describe "multiple cmdlets tests" {
    It "should run first" {
        Invoke-MyFirstCmdlet | Should Be $true;
    }
    It "should run second" {
        Invoke-MySecondCmdlet | Should Be $true;
    }
}

go.ps1

$ErrorActionPreference = "Stop";
Set-StrictMode -Version "Latest";

Import-Module -Name ".\pester\pester.psd1" -ErrorAction "Stop";

Import-Module -Name ".\test-single-cmdlet.psm1" -ErrorAction "Stop";
$results = Invoke-Pester -Script ".\test-single-cmdlet.tests.ps1" -CodeCoverage ".\test-single-cmdlet.psm1" -PassThru;
write-host (ConvertTo-Json -InputObject $results.CodeCoverage);

Import-Module -Name ".\test-multiple-cmdlets.psm1" -ErrorAction "Stop";
$results = Invoke-Pester -Script ".\test-multiple-cmdlets.tests.ps1" -CodeCoverage ".\test-multiple-cmdlets.psm1" -PassThru;
write-host (ConvertTo-Json -InputObject $results.CodeCoverage);

results:

c:\src\scratch\pester\issue-1348>powershell .\go.ps1
    ____            __
   / __ \___  _____/ /____  _____
  / /_/ / _ \/ ___/ __/ _ \/ ___/
 / ____/  __(__  ) /_/  __/ /
/_/    \___/____/\__/\___/_/
Pester v4.8.1
Executing all tests in '.\test-single-cmdlet.tests.ps1'

Executing script .\test-single-cmdlet.tests.ps1

  Describing single cmdlet tests
    [+] should run single 534ms
Tests completed in 2.27s
Tests Passed: 1, Failed: 0, Skipped: 0, Pending: 0, Inconclusive: 0

Code coverage report:
Covered 100.00% of 1 analyzed Command in 1 File.
{
    "NumberOfCommandsAnalyzed":  1,
    "NumberOfFilesAnalyzed":  1,
    "NumberOfCommandsExecuted":  1,
    "NumberOfCommandsMissed":  0,
    "MissedCommands":  [

                       ],
    "HitCommands":  [
                        {
                            "File":  "C:\\src\\scratch\\pester\\issue-1348\\test-single-cmdlet.psm1",
                            "Line":  3,
                            "StartLine":  3,
                            "EndLine":  3,
                            "StartColumn":  12,
                            "EndColumn":  17,
                            "Class":  "",
                            "Function":  "Invoke-MySingleCmdlet",
                            "Command":  "return $true",
                            "HitCount":  1
                        }
                    ],
    "AnalyzedFiles":  [
                          "C:\\src\\scratch\\pester\\issue-1348\\test-single-cmdlet.psm1"
                      ]
}
    ____            __
   / __ \___  _____/ /____  _____
  / /_/ / _ \/ ___/ __/ _ \/ ___/
 / ____/  __(__  ) /_/  __/ /
/_/    \___/____/\__/\___/_/
Pester v4.8.1
Executing all tests in '.\test-multiple-cmdlets.tests.ps1'

Executing script .\test-multiple-cmdlets.tests.ps1

  Describing multiple cmdlets tests
    [+] should run first 263ms
    [+] should run second 268ms
Tests completed in 1.73s
Tests Passed: 2, Failed: 0, Skipped: 0, Pending: 0, Inconclusive: 0

Code coverage report:
Covered 100.00% of 2 analyzed Commands in 1 File.
{
    "NumberOfCommandsAnalyzed":  2,
    "NumberOfFilesAnalyzed":  1,
    "NumberOfCommandsExecuted":  2,
    "NumberOfCommandsMissed":  0,
    "MissedCommands":  [

                       ],
    "HitCommands":  [
                        {
                            "File":  "C:\\src\\scratch\\pester\\issue-1348\\test-multiple-cmdlets.psm1",
                            "Line":  3,
                            "StartLine":  3,
                            "EndLine":  3,
                            "StartColumn":  12,
                            "EndColumn":  17,
                            "Class":  "",
                            "Function":  "Invoke-MyFirstCmdlet",
                            "Command":  "return $true",
                            "HitCount":  1
                        },
                        {
                            "File":  "C:\\src\\scratch\\pester\\issue-1348\\test-multiple-cmdlets.psm1",
                            "Line":  7,
                            "StartLine":  7,
                            "EndLine":  7,
                            "StartColumn":  12,
                            "EndColumn":  17,
                            "Class":  "",
                            "Function":  "Invoke-MySecondCmdlet",
                            "Command":  "return $true",
                            "HitCount":  1
                        }
                    ],
    "AnalyzedFiles":  [
                          "C:\\src\\scratch\\pester\\issue-1348\\test-multiple-cmdlets.psm1"
                      ]
}

[mikeclayton]

Update - Ok, I've worked out how to produce the JaCoCo report with the "-CodeCoverageOutputFile" parameter, but I can't reproduce the "Class name inconsistency" issue...

go.ps1

$ErrorActionPreference = "Stop";
Set-StrictMode -Version "Latest";

Import-Module -Name ".\pester\pester.psd1" -ErrorAction "Stop";

Import-Module -Name ".\test-single-cmdlet.psm1" -ErrorAction "Stop";
$results = Invoke-Pester -Script       ".\test-single-cmdlet.tests.psm1" `
                         -CodeCoverage ".\test-single-cmdlet.psm1" `
                         -CodeCoverageOutputFile ".\test-single-cmdlet.jacoco" `
                         -OutputFile   ".\test-single-cmdlet.xml" `
                         -OutputFormat "NUnitXML" `
                         -PassThru;
write-host (ConvertTo-Json -InputObject $results.CodeCoverage);

Import-Module -Name ".\test-multiple-cmdlets.psm1" -ErrorAction "Stop";
$results = Invoke-Pester -Script       ".\test-multiple-cmdlets.tests.psm1" `
                         -CodeCoverage ".\test-multiple-cmdlets.psm1" `
                         -CodeCoverageOutputFile ".\test-multiple-cmdlet.jacoco" `
                         -OutputFile   ".\test-multiple-cmdlet.xml" `
                         -OutputFormat "NUnitXML" `
                         -PassThru;
write-host (ConvertTo-Json -InputObject $results.CodeCoverage);

gives class names:

single

        <class name="issue-1348/test-single-cmdlet" sourcefilename="test-single-cmdlet.psm1">

multipke

        <class name="issue-1348/test-multiple-cmdlets" sourcefilename="test-multiple-cmdlets.psm1">

@Glober777 - could you provide the "single cmdlet" and "multiple cmdlet" versions of your scripts (or a cut-down version of them) to see if I can reproduce the issue?