Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable pinning our charm test dependencies (ex: if kubeflow-dashboard's tests need to deploy kubeflow-profiles) #136

Open
ca-scribner opened this issue Mar 12, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@ca-scribner
Copy link
Contributor

Context

Most MLOps charms have integration tests that have dependencies on other charms (for example, the kubeflow-dashboard charm requires the kubeflow-profiles charm) . The channel for these dependencies is hard-coded in the test files , generally pointing to latest/edge. This causes three issues:

  1. as tests always run against latest/edge, there is no easy way to test a charm against a specific bundle or set of dependencies. This means we cannot test a charm against a candidate bundle release (we always test against latest/edge)
  2. when we release a product we cut new github branches (ex: we create branch 1.8 for the 1.8 release) but the CI dependencies in these branches still points to latest/edge. When we later need to rerun the CI (ex: to ship a bugfix on the previous release), the CI runs against latest/edge which is the next release, not the frozen release
  3. day-to-day development benefits from a stable development experience (eg: dependencies using */stable) but you also want to check for upcoming issues (eg: test against */edge to catch a breaking future change). As-is we have to choose one

Item (2) is a cost on developer time as well as a source for bugs passing through CI into released charms. Items (1) and (3) are a cost on developer time.

What needs to get done

  1. implement a way to specify the charm dependencies a test should use by configuration, not just by code. For example, enable calling the tests by tox -e integration -- --dependency-bundle bundle.yaml where bundle.yaml specifies the dependency versions that should be used
  2. automation that can freeze the test default dependencies used by a charm repo. For example, this might be called when cutting the kubeflow-dashboard 1.8 release branch to the kubeflow-profiles dependency to 1.8/stable instead of latest/edge

Definition of Done

  1. automation is implemented in all repositories to enable pinning test dependencies as we cut release branches
@ca-scribner ca-scribner added the enhancement New feature or request label Mar 12, 2024
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5454.

This message was autogenerated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant