Generated Clover XML logfiles do not validate against clover.xsd

[sebastianbergmann]

$ git clone [email protected]:sebastianbergmann/money.git

$ cd money

$ phpunit -c build

$ wget https://bitbucket.org/atlassian/clover/raw/bd0ebb6cc8cc3e3fdb878b2640078461b8902a32/etc/schema/clover.xsd

$ xmllint --noout --schema /tmp/clover.xsd build/logs/clover.xml 
build/logs/clover.xml:2: element coverage: Schemas validity error : Element 'coverage': The attribute 'clover' is required but missing.
build/logs/clover.xml:4: element package: Schemas validity error : Element 'package': This element is not expected. Expected is ( metrics ).
build/logs/clover.xml:2: element coverage: Schemas validity error : Element 'coverage': Missing child element(s). Expected is ( testproject ).
build/logs/clover.xml fails to validate

[sebastianbergmann]

This could be a breaking change.

[sebastianbergmann]

Started to work on this in https://github.com/sebastianbergmann/php-code-coverage/tree/issue-578/clover-format.

[sebastianbergmann]

Thanks to @jenschude for https://gist.github.com/jenschude/9619ba13a9e7743368ed07b21dad6179#gistcomment-2328139:

public class Foo
{
    public boolean bar(int a, int b)
    {
        if (a == b) {
            return false;
        }

        return true;
    }
}

import org.junit.Test;

import static org.assertj.core.api.Assertions.assertThat;

public class FooTest
{
    @Test
    public void one() {
        Foo foo = new Foo();

        assertThat(foo.bar(1, 2));
        assertThat(!foo.bar(1, 1));
    }
}

<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1516617047954" clover="4.2.0">
   <project name="foo" timestamp="1516617047145">
      <metrics coveredelements="6" complexity="2" loc="11" methods="1" classes="1" statements="3" packages="1" coveredconditionals="2" coveredmethods="1" elements="6" ncloc="10" files="1" conditionals="2" coveredstatements="3"/>
      <package name="default-pkg">
         <metrics coveredelements="6" complexity="2" loc="11" methods="1" classes="1" statements="3" coveredconditionals="2" coveredmethods="1" elements="6" ncloc="10" files="1" conditionals="2" coveredstatements="3"/>
         <file path="/Users/jensschulze/workspace/foo/src/main/java/Foo.java" name="Foo.java">
            <metrics coveredelements="6" coveredconditionals="2" complexity="2" loc="11" coveredmethods="1" methods="1" elements="6" classes="1" ncloc="10" statements="3" conditionals="2" coveredstatements="3"/>
            <class name="Foo">
               <metrics coveredelements="6" coveredconditionals="2" complexity="2" coveredmethods="1" methods="1" elements="6" statements="3" conditionals="2" coveredstatements="3"/>
            </class>
            <line complexity="2" visibility="public" signature="bar(int,int) : boolean" num="3" count="2" type="method"/>
            <line num="5" count="2" type="stmt"/>
            <line falsecount="1" truecount="1" num="5" type="cond"/>
            <line num="6" count="1" type="stmt"/>
            <line num="9" count="1" type="stmt"/>
         </file>
      </package>
   </project>
   <testproject name="foo" timestamp="1516617047145">
      <metrics coveredelements="4" complexity="1" loc="15" methods="1" classes="1" statements="3" packages="1" coveredconditionals="0" coveredmethods="1" elements="4" ncloc="11" files="1" conditionals="0" coveredstatements="3"/>
      <package name="default-pkg">
         <metrics coveredelements="4" complexity="1" loc="15" methods="1" classes="1" statements="3" coveredconditionals="0" coveredmethods="1" elements="4" ncloc="11" files="1" conditionals="0" coveredstatements="3"/>
         <file path="/Users/jensschulze/workspace/foo/src/test/java/FooTest.java" name="FooTest.java">
            <metrics coveredelements="4" coveredconditionals="0" complexity="1" loc="15" coveredmethods="1" methods="1" elements="4" classes="1" ncloc="11" statements="3" conditionals="0" coveredstatements="3"/>
            <class name="FooTest">
               <metrics coveredelements="4" complexity="1" methods="1" testruns="1" statements="3" coveredconditionals="0" coveredmethods="1" elements="4" testfailures="0" testduration="0.057999998331069946" conditionals="0" coveredstatements="3" testpasses="1"/>
            </class>
            <line complexity="1" visibility="public" signature="one() : void" num="8" testsuccess="true" count="1" testduration="0.057999998331069946" type="method"/>
            <line num="10" count="1" type="stmt"/>
            <line num="12" count="1" type="stmt"/>
            <line num="13" count="1" type="stmt"/>
         </file>
      </package>
   </testproject>
</coverage>

[RykHawthorn]

Hi, are you still working on it?

[jimdelois]

Is there an estimated timeline for this? Are there any works-in-progress that could use assistance to get this across the finish line, or can/should it be taken up anew?

There is a great deal of tooling that relies on correct Clover formatting - it is unfortunate that when using PHP with such tooling, the incompatibility with the Clover spec means that coverage features cannot be used (for example, AWS CodeBuild).

[sebastianbergmann]

I am not working on this and there is no timeline.

[vnrmc]

@jimdelois use cobertura as small workaround (--coverage-cobertura, available since phpunit 9.4) and modify your buildpec:
... reports: codecoverage: ... file-format: 'COBERTURAXML'

[amottier]

One issue that I have with current Clover report format is that the name attribute of file tag is actually used to store the full file path (If I understand correctly the code) whereas, according to XSD, it should only store the file name and the full file path should be stored in a path attribute (path attribute is not set in current report implementation).
Such behavior is confirmed by Clover Java implementation that set attributes values using only file name for the name attribute and absolute path for the path attribute.

In SonarQube, parsing of the report by Sonar PHP plugin has be done to handle correctly the current implementation that only include name attribute.
Also in SonarQube, the Clover plugin is using only the path attribute those making it incompatible with current implementation but supporting the XSD specification. (Edit: I realized that SonarQube Clover plugin is only targeting Java and Groovy).