lambdaisland/kaocha-junit-xml

Kaocha plugin to generate a JUnit XML version of the test results.

 

 

Support Lambda Island Open Source

kaocha-junit-xml is part of a growing collection of quality Clojure libraries and tools released on the Lambda Island label. If you are using this project commercially then you are expected to pay it forward by becoming a backer on Open Collective, so that we may continue to enjoy a thriving Clojure ecosystem.

 

 

Usage

• Add kaocha-junit-xml as a dependency

;; deps.edn
{:aliases
 {:test
  {:extra-deps {lambdaisland/kaocha {...}
                lambdaisland/kaocha-junit-xml {:mvn/version "1.17.101"}}}}}

or

;; project.clj
(defproject ,,,
  :dependencies [,,,
                 [lambdaisland/kaocha-junit-xml "1.17.101"]])

• Enable the plugin and set an output file

;; tests.edn
#kaocha/v1
{:plugins [:kaocha.plugin/junit-xml]
 :kaocha.plugin.junit-xml/target-file "junit.xml"}

Or from the CLI

bin/kaocha --plugin kaocha.plugin/junit-xml --junit-xml-file junit.xml

Optionally you can omit captured output from junit.xml

;; tests.edn
#kaocha/v1
{:plugins [:kaocha.plugin/junit-xml]
 :kaocha.plugin.junit-xml/target-file      "junit.xml"
 :kaocha.plugin.junit-xml/omit-system-out? true}

Or from the CLI

bin/kaocha --plugin kaocha.plugin/junit-xml --junit-xml-file junit.xml --junit-xml-omit-system-out

Requirements

Requires at least Kaocha 0.0-306 and Clojure 1.9.

CI Integration

Some CI tooling supports the junit xml output in various flavours.

CircleCI

One of the services that can use this output is CircleCI. Your .circleci/config.yml could look like this:

version: 2
jobs:
  build:
    docker:
      - image: circleci/clojure:tools-deps-1.9.0.394
    steps:
      - checkout
      - run: mkdir -p test-results/kaocha
      - run: bin/kaocha --plugin kaocha.plugin/junit-xml --junit-xml-file test-results/kaocha/results.xml --junit-xml-add-location-metadata --junit-xml-use-relative-path-in-location
      - store_test_results:
          path: test-results

GitHub Actions

The following configuration will create annotations for test failures on files of the relevant commit/PR. First enable the plugin with the add-location-metadata? flag in your tests.edn:

#kaocha/v1
{:plugins [:kaocha.plugin/junit-xml]
 :kaocha.plugin.junit-xml/target-file "junit.xml"
 :kaocha.plugin.junit-xml/add-location-metadata? true}

Then, an example .github/workflows/build.yml may look like:

name: Build
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    container:
      image: clojure:openjdk-8-tools-deps-1.11.1.1113
    steps:
      - uses: actions/checkout@v2
      - name: test
        run: |
          bin/kaocha
      - name: Annotate failure
        if: failure()
        uses: mikepenz/action-junit-report@41a3188dde10229782fd78cd72fc574884dd7686
        with:
          report_paths: junit.xml

Gitlab

Configuring Gitlab to parse JUnit XML is easy; just add a report artifact that points to the XML file:

test:
  only:
    -tags
  script:
    - make test
  artifacts:
    reports:
      junit: junit.xml

See the Gitlab documentation on reports using JUnit for more information.

Caveats

For timing information (timestamp and running time) this plugin relies on the kaocha.plugin/profiling plugin. If the plugin is not present then a running time of 0 will be reported.

For output capturing the kaocha.plugin/capture-output must be present. If it is not present <system-out> will always be empty.

Resources

It was hard to find a definitive source of the Ant Junit XML format. I mostly went with this page for documentation.

For information on how to configure CircleCI to use this information, see store_test_results.

After reports that the output was not compatible with Azure Devops Pipeline the output was changed to adhere to this schema.

The --junit-xml-add-location-metadata flag was added to enhance testcase output with test location metadata à la pytest. This allows for integration with various tools on GitHub Actions for producing annotations on files in commits/PRs with test failure data. For example, the JUnit Report Action.

The --junit-xml-use-relative-path-in-location flasg was added to enhance the --junit-xml-add-location-metadata output for test ecosystems that rely on relative file-pathing. Notably, CircleCI's parallelization assumes file paths in JUnit files are relative to the runner's working directory.

Contributing

Everyone has a right to submit patches to kaocha-junit-xml, and thus become a contributor.

Contributors MUST

• adhere to the LambdaIsland Clojure Style Guide
• write patches that solve a problem. Start by stating the problem, then supply a minimal solution. *
• agree to license their contributions as EPL 1.0.
• not break the contract with downstream consumers. **
• not break the tests.

Contributors SHOULD

• update the CHANGELOG and README.
• add tests for new functionality.

If you submit a pull request that adheres to these rules, then it will almost certainly be merged immediately. However some things may require more consideration. If you add new dependencies, or significantly increase the API surface, then we need to decide if these changes are in line with the project's goals. In this case you can start by writing a pitch, and collecting feedback on it.

* This goes for features too, a feature needs to solve a problem. State the problem it solves, then supply a minimal solution.

** As long as this project has not seen a public release (i.e. is not on Clojars) we may still consider making breaking changes, if there is consensus that the changes are justified.

License

Copyright © 2018-2020 Arne Brasseur and contributors

Available under the terms of the Eclipse Public License 1.0, see LICENSE.txt