Skip to content

Latest commit

 

History

History
381 lines (272 loc) · 12.1 KB

CONTRIBUTING.rst

File metadata and controls

381 lines (272 loc) · 12.1 KB
Raw

Contributing

This document contains:

How to report bugs

When reporting a bug, first search our issues to avoid duplicates. In your bug report, please describe what you expected gcovr to do, and what it actually did. Also try to include the following details:

  • how you invoked gcovr, i.e. the exact flags and from which directory
  • your project layout
  • your gcovr version
  • your compiler version
  • your operating system
  • and any other relevant details.

Ideally, you can provide a short script and the smallest possible source file to reproduce the problem.

How to help

If you would like to help out, please take a look at our open issues and pull requests. The issues labeled help wanted and needs review would have the greatest impact.

There are many ways how you can help:

  • assist other users with their problems
  • share your perspective as a gcovr user in discussions
  • test proposed changes in your real-world projects
  • improve our documentation
  • submit pull requests with bug fixes and enhancements

How to submit a Pull Request

Thank you for helping with gcovr development! Please follow this checklist for your pull request:

  • Is this a good approach? Fixing open issues is always welcome! If you want to implement an enhancement, please discuss it first as a GitHub issue.

  • Does it work? Please run the tests locally:

    make test

    (see also: :ref:`test suite`)

    In any case, the tests will run automatically when you open the pull request. But please prevent unnecessary build failures and run the tests yourself first. If you cannot run the tests locally, you can activate GitHub or Appveyor for your fork, or run the tests with Docker.

    If you add new features, please try to add a test case.

  • Does it conform to the style guide? The source code should conform to the PEP 8 standard. Please check your code:

    make lint

# or:

python3 -m flake8 doc gcovr --ignore E501,W503

    The command make qa will run the linter, run the tests, and check that the docs can be built.

  • Add yourself as an author. If this is your first contribution to gcovr, please add yourself to the AUTHORS.txt file.

  • One change at a time. Please keep your commits and your whole pull request fairly small, so that the changes are easy to review. Each commit should only contain one kind of change, e.g. refactoring or new functionality.

  • Why is this change necessary? When you open the PR, please explain why we need this change and what your PR does. If this PR fixes an open issue, reference that issue in the pull request description.

Once you submit the PR, it will be automatically tested on Windows and Linux, and code coverage will be collected. Your code will be reviewed. This can take a week. Please fix any issues that are discovered during this process. Feel free to force-push your updates to the pull request branch.

If you need assistance for your pull request, you can

  • chat in our Gitter room
  • discuss your problem in an issue
  • open an unfinished pull request as a work in progress (WIP), and explain what you've like to get reviewed

How to set up a development environment

For working on gcovr, you will need a supported version of Python 3, and GCC version 5. Other GCC versions are supported by gcovr, but will cause spurious test failures.

  • (Optional) Fork the project on GitHub.

  • Clone the git repository.

  • (Optional) Set up a virtualenv (e.g. with python3 -m venv my-venv)

  • Install gcovr in development mode, and install the test requirements:

    make setup-dev  # install all test + doc dependencies

# or:

pip install -e .
pip install -r requirements.txt

    You can then run gcovr as gcovr or python3 -m gcovr.

    Run the tests to verify that everything works (see :ref:`test suite`).

  • (Optional) Install documentation requirements:

    # would be already done by `make setup-dev`
pip install -r doc/requirements.txt

    See doc/README.txt for details on working with the documentation.

  • (Optional) Activate GitHub and Appveyor for your forked GitHub repository, so that the cross-platform compatibility tests get run whenever you push your work to your repository. These tests will also be run when you open a pull request to the main gcovr repository.

Tip: If you have problems getting everything set up, consider looking at these files:

  • for Linux: .github/workflows/test.yml and admin/Dockerfile.qa
  • for Windows: .github/workflows/test.yml and appveyor.yml

On Windows, you will need to install a GCC toolchain as the tests expect a Unix-like environment. You can use MinGW-W64 or MinGW. To run the tests, please make sure that the make and cmake from your MinGW distribution are in the system PATH.

If setting up a local toolchain is too complicated, you can also run the tests in a Docker container (see :ref:`test suite`).

Project Structure

Path Description
/ project root
/gcovr/ the gcovr source code (Python module)
/gcovr/__main__.py command line interface + top-level behaviour
/gcovr/templates/ HTML report templates
/gcovr/tests/ unit tests + integration test corpus
/setup.py Python package configuration
/doc/ documentation
/doc/sources/ user guide + website
/doc/examples/ runnable examples for the user guide

The program entrypoint and command line interface is in gcovr/__main__.py. The coverage data is parsed in the gcovr.gcov module. The HTML, XML, text, and summary reports are in gcovr.html_generator and respective modules.

Test suite

The QA process (make qa) consists of multiple parts:

  • linting (make lint)

  • tests (make test)

    • unit tests in gcovr/tests
    • integration tests in gcovr/tests
    • documentation examples in doc/examples

  • documentation build (make doc)

The tests are in the gcovr/tests directory. You can run the tests with make test or python3 -m pytest gcovr.

There are unit tests for some parts of gcovr, and a comprehensive corpus of example projects that are executed as the test_gcovr.py integration test. Each gcovr/tests/* directory is one such example project.

The next sections discuss the :ref:`structure of integration tests <integration tests>`, how to :ref:`run and filter tests <run tests>`, and how to :ref:`run tests with Docker <docker tests>`.

Structure of integration tests

Each project in the corpus contains a Makefile and a reference directory:

gcovr/tests/sometest/
  reference/
  Makefile
  README
  example.cpp

The Makefile controls how the project is built, and how gcovr should be invoked. The reference directory contains baseline files against which the gcovr output is compared. Tests can be executed even without baseline files.

Each Makefile contains the following targets:

  • all: builds the example project. Can be shared between gcovr invocations.
  • run: lists available targets which must be a subset of the available output formats.
  • clean: remove any generated files after all tests of the scenario have finished.
  • output formats (txt, html, json, sonarqube, ...): invoke gcovr to produce output files of the correct format. The test runner automatically finds the generated files (if any) and compares them to the baseline files in the reference directory. All formats are optional, but using at least JSON is recommended.
  • clean-each: if provided, will be invoked by the test runner after testing each format.

Run and filter tests

To run all tests, use make test or make qa. The tests currently assume that you are using GCC 5 and have set up a :ref:`development environment <development environment>`.

You can run the tests with additional options by setting TEST_OPTS variable. Run all tests after each change is a bit slow, therefore you can limit the tests to a specific test file, example project, or output format. For example:

# run only XML tests
make test TEST_OPTS="-k 'xml'"

# run the simple1 tests
make test TEST_OPTS="-k 'simple1'"

# run the simple1 tests only for XML
make test TEST_OPTS="-k 'xml and simple1'"

To see which tests would be run, add the --collect-only option:

#see which tests would be run
make test TEST_OPTS="--collect-only"

Sometimes during development you need to create reference files for new test or update the current reference files. To do this you have to add --generate_reference or --update-reference option to the TEST_OPTS variable. By default generated output files are automatically removed after test run. To skip this process you can add --skip_clean option the TEST_OPTS. For example:

# run tests and generate references for simple1 example
make test TEST_OPTS="-k 'simple1' --generate_reference"

# run tests and update xml references for simple1 example
make test TEST_OPTS="-k 'xml and simple1' --update_reference"

# run only XML tests and do not remove generated files
make test TEST_OPTS="-k 'xml' --skip_clean"

When the currently generated output reports differ to the reference files you can create a ZIP archive named diff.zip in the tests directory by using --archive_differences option. Currently in gcovr it is used by AppVeyor CI to create a ZIP file with the differences as an artifact.

# run tests and generate a ZIP archive when there were differences
make test TEST_OPTS="--archive_differences"
.. versionadded:: NEXT
   Added test options `--generate_reference`, `--update_reference`,
   `--skip_clean`, '--archive_differences' and changed way to call tests
   only by ``make test``.

Run tests with Docker

If you can't set up a toolchain locally, you can run the QA process via Docker. First, build the container image:

make docker-qa-build

Then, run the container, which executes make qa within the container:

make docker-qa

You can select the gcc version to use inside the docker by setting the make variable CC to gcc-5 (default) or gcc-8

Become a gcovr developer

After you've contributed a bit (whether with discussions, documentation, or code), consider becoming a gcovr developer. As a developer, you can:

  • manage issues and pull requests (label and close them)
  • review pull requests (a developer must approve each PR before it can be merged)
  • participate in votes

Just open an issue that you're interested, and we'll have a quick vote.