windsock: add --load-cloud-resources-from-disk and --store-cloud-resources-to-disk #1453
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
progress towards #1274
Usage of the new functionality
Why we need this?
Design decisions
I intentionally designed --store-cloud-resources-file to skip running the benches.
The benefit of this is it ensures that every time we actually run the bench it uses the same command.
This makes:
up arrow -> enter
in a shell to rerun a bench run more likely to do what the user expects.What if we combined the two flags though?
We could just have a single flag
--store-and-reload-cloud-resources-file
that always writes the cloud resources to disk and loads if the cloud resources file exists.However there are two problems with that:
As such I think the best solution is two seperate commands to give the developer control over when they create and recreate these resources.
refreshing instances
When we reuse instances part of the instance setup should be reused and part should be recreated.
In this PR the shotover binary+config, and the bencher binary will be reuploaded.
However the rest of the setup will be reused as is.
If the developer changes anything else about the setup, say adds a new external dependency then they will have to recreate instances.
I think this is the sweet spot as it lets us iterate on shotover and the bencher performance wise while keeping retries fast and/or avoiding complexity for detecting when we need to recreate other parts of the setup.