Skip to content

Latest commit

 

History

History
169 lines (118 loc) · 10.3 KB

CONTRIBUTING.md

File metadata and controls

169 lines (118 loc) · 10.3 KB
Raw

Contributing to continuous-eval

First off, thanks for taking the time to contribute! ❤️

All types of contributions are encouraged and valued. See the Table of Contents for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉

And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:

  • Star the project
  • Tweet about it
  • Refer this project in your project's readme
  • Mention the project at local meetups and tell your friends/colleagues

Table of Contents

Code of Conduct

This project and everyone participating in it is governed by the continuous-eval Code of Conduct. By participating, you are expected to uphold this code.

I Have a Question

If you want to ask a question, we assume that you have read the available Documentation.

Before you ask a question, it is best to search for existing Issues that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first.

If you then still feel the need to ask a question and need clarification, we recommend the following:

  • Open an Issue.
  • Provide as much context as you can about what you're running into.

We will then take care of the issue as soon as possible.

I Want To Contribute

Legal Notice

When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license.

Reporting Bugs

Before Submitting a Bug Report

A good bug report shouldn't leave others needing to chase you up for more information. Therefore, we ask you to investigate carefully, collect information and describe the issue in detail in your report. Please complete the following steps in advance to help us fix any potential bug as fast as possible.

  • Make sure that you are using the latest version.
  • Determine if your bug is really a bug and not an error on your side e.g. using incompatible environment components/versions (Make sure that you have read the documentation. If you are looking for support, you might want to check this section).
  • To see if other users have experienced (and potentially already solved) the same issue you are having, check if there is not already a bug report existing for your bug or error in the bug tracker.
  • Also make sure to search the internet (including Stack Overflow) to see if users outside of the GitHub community have discussed the issue.
  • Collect information about the bug:
    • Stack trace (Traceback)
    • OS, Platform and Version (Windows, Linux, macOS, x86, ARM)
    • Python version
    • Possibly your input and the output
    • Can you reliably reproduce the issue?

How Do I Submit a Good Bug Report?

You must never report security related issues, vulnerabilities or bugs including sensitive information to the issue tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to [email protected].

We use GitHub issues to track bugs and errors. If you run into an issue with the project:

  • Open an Issue. (Since we can't be sure at this point whether it is a bug or not, we ask you not to talk about a bug yet and not to label the issue.)
  • Explain the behavior you would expect and the actual behavior.
  • Please provide as much context as possible and describe the reproduction steps that someone else can follow to recreate the issue on their own. This usually includes your code. For good bug reports you should isolate the problem and create a reduced test case.
  • Provide the information you collected in the previous section.

Once it's filed:

  • The project team will label the issue accordingly.
  • A team member will try to reproduce the issue with your provided steps. If there are no reproduction steps or no obvious way to reproduce the issue, the team will ask you for those steps and mark the issue as needs-repro. Bugs with the needs-repro tag will not be addressed until they are reproduced.
  • If the team is able to reproduce the issue, it will be marked needs-fix, as well as possibly other tags (such as critical), and the issue will be left to be implemented by someone.

Suggesting Enhancements

This section guides you through submitting an enhancement suggestion for continuous-eval, including completely new features and minor improvements to existing functionality. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions.

Before Submitting an Enhancement

  • Make sure that you are using the latest version.
  • Read the documentation carefully and find out if the functionality is already covered, maybe by an individual configuration.
  • Perform a search to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one.
  • Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. If you're just targeting a minority of users, consider writing an add-on/plugin library.

How Do I Submit a Good Enhancement Suggestion?

Enhancement suggestions are tracked as GitHub issues.

  • Use a clear and descriptive title for the issue to identify the suggestion.
  • Provide a step-by-step description of the suggested enhancement in as many details as possible.
  • Describe the current behavior and explain which behavior you expected to see instead and why. At this point you can also tell which alternatives do not work for you.
  • Explain why this enhancement would be useful to most continuous-eval users. You may also want to point out the other projects that solved it better and which could serve as inspiration.

Your First Code Contribution

Setting Up Your Environment

Continuous-eval is a Python package. We've tested primarily with Python versions >= 3.8. Here's a quick and dirty guide to setting up your environment for local development.

  1. Fork the repo and clone it locally. (New to GitHub / git? Here's how.)
  2. In a terminal, cd into the directory of your local clone of your forked repo.
  3. Prepare a virtual environment.
    1. Install Poetry. This will help you manage package dependencies, see poetry basic usage for more information.
    2. Execute poetry install --all-extras. This will install all dependencies needed for local development. To see what will be installed, read the pyproject.toml under that directory.
  4. Install pre-commit hooks by running poetry run pre-commit install. These hooks are small house-keeping scripts executed every time you make a git commit, which automates away a lot of chores.

Testing

If you modified or added code logic, you must create test(s) Tests help preventing other maintainers from accidentally break the code.

All tests are stored in the tests. We use the testing framework pytest to run the tests.

poetry run pytest

Creating a pull request

See these instructions to open a pull request against the main continuous-eval repo.

Improving The Documentation

The documentation is inside the docs folder. Follow the instructions in the README to build the documentation locally.

The content of the documentation is in the docs/src/content/docs folder. Every page is a markdown file. To change the menu structure, edit the docs/astro.config.mjs file.

Style-guide

All the code is formatted with black and isort. Please make sure to run these tools before committing your changes.