Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Cobertura coverage format #2298

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
932d9b4
wip: Implement Cobertura coverage format
joeskeen Feb 10, 2023
36af738
fix unit test
joeskeen Feb 10, 2023
14e36af
revert editor settings change
joeskeen Feb 10, 2023
658b358
remove ?? operator
joeskeen Feb 10, 2023
1f6525b
fix attribute ordering
joeskeen Feb 10, 2023
a2e5eb0
make coverage report test work on all platforms
joeskeen Feb 10, 2023
61eefc1
Fix windows paths
joeskeen Feb 10, 2023
6c3db56
fix unit test for Windows paths
joeskeen Feb 10, 2023
3b6e06b
kick the build
joeskeen Feb 10, 2023
14e8ef7
re-implement Cobertura coverage report generation
joeskeen Feb 14, 2023
ab0fa57
fix compatibility issues
joeskeen Feb 14, 2023
70ed47a
fix tests
joeskeen Feb 14, 2023
386cd3c
removing Cobertura from v4 parameter options
joeskeen Feb 14, 2023
f0ac943
fix compatibility with ReportGenerator
joeskeen Feb 14, 2023
a272120
Update src/functions/Coverage.ps1
joeskeen Feb 14, 2023
f9ed4a0
fix whitespace
joeskeen Feb 14, 2023
c20f260
Merge branch 'Cobertura' of github.com:joeskeen/Pester into Cobertura
joeskeen Feb 14, 2023
6a1ddc3
fix output
joeskeen Feb 14, 2023
9d78186
fix windows paths
joeskeen Feb 14, 2023
0bd5244
order packages,classes,methods by name
joeskeen Feb 14, 2023
719047a
change Cobertura DTD to loose
joeskeen Feb 14, 2023
9116f06
Tune coverage report for performance
joeskeen Feb 15, 2023
45bdc68
Merge remote-tracking branch 'upstream/main' into pr/joeskeen/2298
fflaten Jul 10, 2024
13c13a7
Remove outdated condition
fflaten Jul 10, 2024
505afef
Merge remote-tracking branch 'upstream/main' into pr/joeskeen/2298
fflaten Jul 12, 2024
a12b1c7
Add Cobertura DTD file
fflaten Jul 12, 2024
13f0b75
Apply suggestions from code review
fflaten Jul 12, 2024
b1f7f2e
Fix typo and update JaCoCo starttime
fflaten Jul 12, 2024
c643f5e
Fix tests
fflaten Jul 12, 2024
80f2120
Use epoch time for Cobertura and JaCoCo
fflaten Jul 12, 2024
58590c9
Merge remote-tracking branch 'upstream/main' into pr/joeskeen/2298
fflaten Oct 15, 2024
98b7881
Update test
fflaten Oct 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ if ($Clean) {
, ("$PSScriptRoot/src/schemas/JUnit4/*.xsd", "$PSScriptRoot/bin/schemas/JUnit4/")
, ("$PSScriptRoot/src/schemas/NUnit25/*.xsd", "$PSScriptRoot/bin/schemas/NUnit25/")
, ("$PSScriptRoot/src/schemas/NUnit3/*.xsd", "$PSScriptRoot/bin/schemas/NUnit3/")
, ("$PSScriptRoot/src/schemas/Cobertura/*.dtd", "$PSScriptRoot/bin/schemas/Cobertura/")
, ("$PSScriptRoot/src/schemas/JaCoCo/*.dtd", "$PSScriptRoot/bin/schemas/JaCoCo/")
, ("$PSScriptRoot/src/csharp/Pester/bin/$Configuration/net462/Pester.dll", "$PSScriptRoot/bin/bin/net462/")
, ("$PSScriptRoot/src/csharp/Pester/bin/$Configuration/net6.0/Pester.dll", "$PSScriptRoot/bin/bin/net6.0/")
Expand Down
1 change: 1 addition & 0 deletions publish/filesToPublish.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
'bin/net6.0/Pester.dll'
'en-US/about_Pester.help.txt'
'en-US/about_PesterConfiguration.help.txt'
'schemas/Cobertura/coverage-loose.dtd'
'schemas/JaCoCo/report.dtd'
'schemas/JUnit4/junit_schema_4.xsd'
'schemas/NUnit25/nunit_schema_2.5.xsd'
Expand Down
2 changes: 1 addition & 1 deletion src/csharp/Pester/CodeCoverageConfiguration.cs
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ public static CodeCoverageConfiguration ShallowClone(CodeCoverageConfiguration c
public CodeCoverageConfiguration() : base("Options to enable and configure Pester's code coverage feature.")
{
Enabled = new BoolOption("Enable CodeCoverage.", false);
OutputFormat = new StringOption("Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters", "JaCoCo");
OutputFormat = new StringOption("Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters, Cobertura", "JaCoCo");
OutputPath = new StringOption("Path relative to the current directory where code coverage report is saved.", "coverage.xml");
OutputEncoding = new StringOption("Encoding of the output file.", "UTF8");
Path = new StringArrayOption("Directories or files to be used for code coverage, by default the Path(s) from general settings are used, unless overridden here.", new string[0]);
Expand Down
2 changes: 1 addition & 1 deletion src/en-US/about_PesterConfiguration.help.txt
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ SECTIONS AND OPTIONS
Type: bool
Default value: $false

OutputFormat: Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters
OutputFormat: Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters, Cobertura
Type: string
Default value: 'JaCoCo'

Expand Down
14 changes: 7 additions & 7 deletions src/functions/Coverage.Plugin.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -145,11 +145,11 @@

$configuration = $run.PluginConfiguration.Coverage

if ($configuration.OutputFormat -in 'JaCoCo', 'CoverageGutters') {
[xml] $jaCoCoReport = [xml] (Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format $configuration.OutputFormat)
}
else {
throw "CodeCoverage.CoverageFormat '$($configuration.OutputFormat)' is not valid, please review your configuration."
$coverageXmlReport = switch ($configuration.OutputFormat) {
'JaCoCo' { [xml](Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format 'JaCoCo') }
'CoverageGutters' { [xml](Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format 'CoverageGutters') }
'Cobertura' { [xml](Get-CoberturaReportXml -CoverageReport $coverageReport -TotalMilliseconds $totalMilliseconds) }
default { throw "CodeCoverage.CoverageFormat '$($configuration.OutputFormat)' is not valid, please review your configuration." }
}

$settings = [Xml.XmlWriterSettings] @{
Expand All @@ -163,7 +163,7 @@
$stringWriter = [Pester.Factory]::CreateStringWriter()
$xmlWriter = [Xml.XmlWriter]::Create($stringWriter, $settings)

$jaCocoReport.WriteContentTo($xmlWriter)
$coverageXmlReport.WriteContentTo($xmlWriter)

$xmlWriter.Flush()
$stringWriter.Flush()
Expand Down Expand Up @@ -216,7 +216,7 @@
}

function Resolve-CodeCoverageConfiguration {
$supportedFormats = 'JaCoCo', 'CoverageGutters'
$supportedFormats = 'JaCoCo', 'CoverageGutters', 'Cobertura'
if ($PesterPreference.CodeCoverage.OutputFormat.Value -notin $supportedFormats) {
throw (Get-StringOptionErrorMessage -OptionPath 'CodeCoverage.OutputFormat' -SupportedValues $supportedFormats -Value $PesterPreference.CodeCoverage.OutputFormat.Value)
}
Expand Down
242 changes: 234 additions & 8 deletions src/functions/Coverage.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -802,13 +802,12 @@ function Get-JaCoCoReportXml {

$isGutters = "CoverageGutters" -eq $Format

if ($null -eq $CoverageReport -or ('None' -eq $pester.Show) -or $CoverageReport.NumberOfCommandsAnalyzed -eq 0) {
if ($null -eq $CoverageReport -or $CoverageReport.NumberOfCommandsAnalyzed -eq 0) {
return [string]::Empty
}

$now = & $SafeCommands['Get-Date']
$nineteenSeventy = & $SafeCommands['Get-Date'] -Date "01/01/1970"
[long] $endTime = [math]::Floor((New-TimeSpan -start $nineteenSeventy -end $now).TotalMilliseconds)
# Report uses unix epoch time format (milliseconds since midnight 1/1/1970 UTC)
[long] $endTime = [System.DateTimeOffset]::UtcNow.ToUnixTimeMilliseconds()
Comment on lines +817 to +818
Copy link
Collaborator

@fflaten fflaten Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nohwnd I included this bugfix for JaCoCo as well. [DateTime]"01/01/1970" is not proper epoch origin (missing UTC kind), so off by one hour in my timezone.

Source for epoch time format:
https://github.com/jacoco/jacoco/blob/77d2af5ec31faf04419d945e1a0c34da49f8a702/org.jacoco.core/src/org/jacoco/core/data/SessionInfo.java#L33-L38

Backport version will need to use:

$nineteenSeventy = & $SafeCommands['New-Object'] 'System.DateTime' -ArgumentList @(1970, 1, 1, 0, 0, 0, [System.DateTimeKind]::Utc)
$now = [DateTime]::Now.ToUniversalTime()
[long] $endTime = [math]::Floor(($now - $nineteenSeventy).TotalMilliseconds)
[long] $startTime = [math]::Floor($endTime - $TotalMilliseconds)

[long] $startTime = [math]::Floor($endTime - $TotalMilliseconds)

$folderGroups = $CommandCoverage | & $SafeCommands["Group-Object"] -Property {
Expand Down Expand Up @@ -1030,6 +1029,224 @@ function Get-JaCoCoReportXml {
return $xml
}

function Get-CoberturaReportXml {
param (
[parameter(Mandatory = $true)]
[object] $CoverageReport,
[parameter(Mandatory = $true)]
[long] $TotalMilliseconds
)

if ($null -eq $CoverageReport -or $CoverageReport.NumberOfCommandsAnalyzed -eq 0) {
return [string]::Empty
}

# Report uses unix epoch time format (milliseconds since midnight 1/1/1970 UTC)
[long] $endTime = [System.DateTimeOffset]::UtcNow.ToUnixTimeMilliseconds()
[long] $startTime = [math]::Floor($endTime - $TotalMilliseconds)
Comment on lines +1052 to +1054
Copy link
Collaborator

@fflaten fflaten Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Source for epoch time format: https://github.com/cobertura/cobertura/blob/0ff963284cecaace30f409a977dccb07c41a5a8f/cobertura/src/main/java/net/sourceforge/cobertura/reporting/xml/XMLReport.java#L89

Backport version will need to use:

$nineteenSeventy = & $SafeCommands['New-Object'] 'System.DateTime' -ArgumentList @(1970, 1, 1, 0, 0, 0, [System.DateTimeKind]::Utc)
$now = [DateTime]::Now.ToUniversalTime()
[long] $endTime = [math]::Floor(($now - $nineteenSeventy).TotalMilliseconds)
[long] $startTime = [math]::Floor($endTime - $TotalMilliseconds)


$commonRoot = Get-CommonParentPath -Path $CoverageReport.AnalyzedFiles

$allLines = [System.Collections.Generic.List[object]]@()
$allLines.AddRange($CoverageReport.MissedCommands)
$allLines.AddRange($CoverageReport.HitCommands)
$packages = @{}
foreach ($command in $allLines) {
$package = & $SafeCommands["Split-Path"] $command.File -Parent
if (!$packages[$package]) {
$packages[$package] = @{
Classes = @{}
}
}

$class = $command.File
if (!$packages[$package].Classes[$class]) {
$packages[$package].Classes[$class] = @{
Methods = @{}
Lines = @{}
}
}

if (!$packages[$package].Classes[$class].Lines[$command.Line]) {
$packages[$package].Classes[$class].Lines[$command.Line] = [ordered]@{ number = $command.Line ; hits = 0 }
}
$packages[$package].Classes[$class].Lines[$command.Line].hits += $command.HitCount

$method = $command.Function
if (!$method) {
continue
}

if (!$packages[$package].Classes[$class].Methods[$method]) {
$packages[$package].Classes[$class].Methods[$method] = @{}
}

if (!$packages[$package].Classes[$class].Methods[$method][$command.Line]) {
$packages[$package].Classes[$class].Methods[$method][$command.Line] = [ordered]@{ number = $command.Line ; hits = 0 }
}
$packages[$package].Classes[$class].Methods[$method][$command.Line].hits += $command.HitCount
}

$packages = foreach ($packageGroup in $packages.GetEnumerator()) {
$classGroups = $packageGroup.Value.Classes
$classes = foreach ($classGroup in $classGroups.GetEnumerator()) {
$methodGroups = $classGroup.Value.Methods
$methods = foreach ($methodGroup in $methodGroups.GetEnumerator()) {
$lines = ([object[]]$methodGroup.Value.Values) | New-LineNode
$coveredLines = foreach ($line in $lines) { if (0 -lt $line.attributes.hits) { $line } }

$method = [ordered]@{
name = 'method'
attributes = [ordered]@{
name = $methodGroup.Name
signature = '()'
}
children = [ordered]@{
lines = $lines | & $SafeCommands["Sort-Object"] { [int]$_.attributes.number }
}
totalLines = $lines.Length
coveredLines = $coveredLines.Length
}

$method
}

$lines = ([object[]]$classGroup.Value.Lines.Values) | New-LineNode
$coveredLines = foreach ($line in $lines) { if (0 -lt $line.attributes.hits) { $line } }

$lineRate = Get-LineRate -CoveredLines $coveredLines.Length -TotalLines $lines.Length
$filename = $classGroup.Name.Substring($commonRoot.Length).Replace('\', '/').TrimStart('/')

$class = [ordered]@{
name = 'class'
attributes = [ordered]@{
name = (& $SafeCommands["Split-Path"] $classGroup.Name -Leaf)
filename = $filename
'line-rate' = $lineRate
'branch-rate' = 1
}
children = [ordered]@{
methods = $methods | & $SafeCommands["Sort-Object"] { $_.attributes.name }
lines = $lines | & $SafeCommands["Sort-Object"] { [int]$_.attributes.number }
}
totalLines = $lines.Length
coveredLines = $coveredLines.Length
}

$class
}

$totalLines = ($classes.totalLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$coveredLines = ($classes.coveredLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$lineRate = Get-LineRate -CoveredLines $coveredLines -TotalLines $totalLines
$packageName = $packageGroup.Name.Substring($commonRoot.Length).Replace('\', '/').TrimStart('/')

$package = [ordered]@{
name = 'package'
attributes = [ordered]@{
name = $packageName
'line-rate' = $lineRate
'branch-rate' = 0
}
children = [ordered]@{
classes = $classes | & $SafeCommands["Sort-Object"] { $_.attributes.name }
}
totalLines = $totalLines
coveredLines = $coveredLines
}

$package
}

$totalLines = ($packages.totalLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$coveredLines = ($packages.coveredLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$lineRate = Get-LineRate -CoveredLines $coveredLines -TotalLines $totalLines

$coverage = [ordered]@{
name = 'coverage'
attributes = [ordered]@{
'lines-valid' = $totalLines
'lines-covered' = $coveredLines
'line-rate' = $lineRate
'branches-valid' = 0
'branches-covered' = 0
'branch-rate' = 1
timestamp = $startTime
version = 0.1
}
children = [ordered]@{
sources = [ordered]@{
name = 'source'
value = $commonRoot.Replace('\', '/')
}
packages = $packages | & $SafeCommands["Sort-Object"] { $_.attributes.name }
}
}

$xmlDeclaration = '<?xml version="1.0" ?>'
fflaten marked this conversation as resolved.
Show resolved Hide resolved
$docType = '<!DOCTYPE coverage SYSTEM "coverage-loose.dtd">'
$coverageXml = ConvertTo-XmlElement -Node $coverage
$document = "$xmlDeclaration`n$docType`n$($coverageXml.OuterXml)"

$document
}

function New-LineNode {
param(
[parameter(Mandatory = $true, ValueFromPipeline = $true)] [object] $LineObject
)

process {
[ordered]@{
name = 'line'
attributes = $LineObject
}
}
}

function Get-LineRate {
param(
[parameter(Mandatory = $true)] [int] $CoveredLines,
[parameter(Mandatory = $true)] [int] $TotalLines
)

[double]$denominator = if ($TotalLines) { $TotalLines } else { 1 }

joeskeen marked this conversation as resolved.
Show resolved Hide resolved
$CoveredLines / $denominator
}

function ConvertTo-XmlElement {
param(
[parameter(Mandatory = $true)] [object] $Node
)

$element = ([xml]"<$($Node.name)/>").DocumentElement
if ($node.attributes) {
$attributes = $node.attributes
foreach ($attr in $attributes.GetEnumerator()) {
$element.SetAttribute($attr.Name, $attr.Value)
}
}
if ($node.children) {
$children = $node.children
foreach ($child in $children.GetEnumerator()) {
$childElement = ([xml]"<$($child.Name)/>").DocumentElement
foreach ($value in $child.Value) {
$childXml = ConvertTo-XmlElement $value
$importedChildXml = $childElement.OwnerDocument.ImportNode($childXml, $true)
$null = $childElement.AppendChild($importedChildXml)
}
$importedChild = $element.OwnerDocument.ImportNode($childElement, $true)
$null = $element.AppendChild($importedChild)
}
}
if ($node.value) {
$element.InnerText = $node.value
}

$element
}

function Add-XmlElement {
param (
[parameter(Mandatory = $true)] [System.Xml.XmlNode] $Parent,
Expand All @@ -1038,14 +1255,23 @@ function Add-XmlElement {
)
$element = $Parent.AppendChild($Parent.OwnerDocument.CreateElement($Name))
if ($Attributes) {
foreach ($key in $Attributes.Keys) {
$attribute = $element.Attributes.Append($Parent.OwnerDocument.CreateAttribute($key))
$attribute.Value = $Attributes.$key
}
Add-XmlAttribute -Element $element -Attributes $Attributes
}
return $element
}

function Add-XmlAttribute {
param(
[parameter(Mandatory = $true)] [System.Xml.XmlNode] $Element,
[parameter(Mandatory = $true)] [System.Collections.IDictionary] $Attributes
)

foreach ($key in $Attributes.Keys) {
$attribute = $Element.Attributes.Append($Element.OwnerDocument.CreateAttribute($key))
$attribute.Value = $Attributes.$key
}
}

function Add-JaCoCoCounter {
param (
[parameter(Mandatory = $true)] [ValidateSet('Instruction', 'Line', 'Method', 'Class')] [string] $Type,
Expand Down
Loading
Loading