Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workload Pass/fail criteria should not depend on expected run duration by default #136

Open
smalleni opened this issue May 11, 2020 · 2 comments

Comments

@smalleni
Copy link
Contributor

Currently, a workload script can exit with a non-zero error code in two probably scenarios

  1. If the workload itself fails
  2. If the workload run takes more time than an expected duration set as a variable

For new users using the framework, it is not clear what these default values should be or if they work in their cluster. It would take some time and testing to come up with these expected duration values. It would make sense to make it optional for pass/fail criteria to depend on expected duration.

@smalleni
Copy link
Contributor Author

@chaitanyaenr FYI

@chaitanyaenr
Copy link
Member

Agreed that there should be an option to disable the duration comparison functionality with default being false to avoid any confusion like you mentioned. Also the default ( 600 seconds ) is not enough for most of the workloads to complete, so they will fail unless we tweak it to something high. Once we know the gold values, we can enable it in cases like CI to determine pass/fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants