-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce execution time of tests #429
Conversation
This was generating too many warnings
Welcome to Codecov 🎉Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment. Thanks for integrating Codecov - We've got you covered ☂️ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is awesome!
I'd just wish that pytest-split
could replace xdist
entirely, but there are two issues:
- Crucially: Is pytest-split supported by pycharm?
- We would need some automated way of starting the maximum number of split jobs possible given the amount of CPUs. Or maybe it'd be enough to just have a config variable (env?) setting this quantity.
|
I mean, if I run tox as a run configuration, will I get IDE support for the tests, in particular with links to the right files (as opposed to useless copies in the tox environment)
I mean running the n commands in one go, and computing the number n automatically. But never mind. xdist manages to fully use the machine and split works wonders on CI, so let's leave it like that. |
@AnesBenmerzoug One more thing: I think it would make sense to deactivate coverage reports in local tests. It's pointless, generates tons of temp files and surely slows things down. |
@AnesBenmerzoug See #454 in case you want to look into it for this PR, otherwise we can merge this as-is. |
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the added docs. I think this is great. I think we can merge, and then release a patch version soon
@AnesBenmerzoug There seems to be some problem with "Run base tests"? |
I don't think this is necessary if the situation is as I described in this comment, namely:
In this case, we get max core usage locally, and IDE built-in support to launch and track the parallel tests. |
@mdbenito |
Description
This PR closes #312 #291
Changes
--failed-first
pytest flag to start testing from the last failed tests.pytest-split
to automatically split the tests in 4 groups based on their measured duration..test_durations
file for use in CI.pytest-cases
to separate the test case definition from the tests in the influences tests.pytest.mark.slow
to mark slow tests and skip them by default.--slow-tests
to enable running slow tests.tests
environment.report
tox environment.1.5.1
.pytest-sugar
as a dev dependency.nbmake
to test notebooks instead of using custom script.Checklist
If notebooks were added/changed, added boilerplate cells are tagged with"tags": ["hide"]
or"tags": ["hide-input"]