Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: set fail-fast as False to allow integration tests to continue on nightly Tutor failures #232

Merged

Conversation

magajh
Copy link
Contributor

@magajh magajh commented Nov 21, 2024

Description

This PR updates the Tutor Integration Tests workflow to ensure that failures in the nightly version of Tutor do not cancel the execution of tests for stable versions (<18.0.0 and <19.0.0). fail-fast: false is added to the strategy, allowing the workflow to proceed even if nightly fails.

This change is important because the nightly version of Tutor is not stable and may occasionally fail due to issues unrelated to our tests. By implementing this update, we ensure that failures in nightly do not block critical tests for stable versions.

Note: For more context on why this solution was implemented instead of others, such as continue-on-error, please refer to the conversation in this issue eduNEXT/eox-core#308.

@magajh magajh marked this pull request as ready for review November 21, 2024 19:10
@magajh magajh requested a review from a team as a code owner November 21, 2024 19:10
jignaciopm
jignaciopm previously approved these changes Nov 21, 2024
@magajh
Copy link
Contributor Author

magajh commented Nov 22, 2024

@BryanttV @jignaciopm removed 'nightly' from the tutor_versions matrix

Copy link
Contributor

@BryanttV BryanttV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @magajh! How could we test if the change works? It's possible to test in eox-core? What has the most integration tests at the moment?

@magajh
Copy link
Contributor Author

magajh commented Nov 26, 2024

@BryanttV turns out continue-on-error does not work since it marks the nightly tests as passed even if the tests were not successful eduNEXT/eox-core@45e66d4

@magajh magajh force-pushed the magagio/allow-integration-tests-to-continue-on-nightly branch from 5ecbb5b to 865cce0 Compare December 5, 2024 04:40
@magajh magajh changed the title ci: allow integration tests to continue on nightly Tutor failures ci: set fail-fast as False to allow integration tests to continue on nightly Tutor failures Dec 5, 2024
@magajh magajh requested a review from a team December 5, 2024 04:56
@magajh magajh merged commit 56e4774 into master Dec 6, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants