Cache isn't updated when adding new cache paths

[esteluk]

Troubleshooting

• I've searched discuss.bitrise.io for possible solutions.
• Which version of the step is effected? main
• Is the issue reproducible with the latest version? YES
• Does the issue happen sporadically, or every time? EVERY TIME
• Is the issue reproducible locally by following our local debug guide? NOT APPLICABLE

Useful information

• FAQ
• devcenter.bitrise.io

Issue description

I was trying to test the change that @liamnichols provided in #11 to see if it helped resolve some build slowdown that I was seeing.

It did, which was great! But I nearly didn't notice this because this step doesn't update the cache when paths are changed, and it's necessary to manually delete the cache from the Bitrise UI before the saved cache is updated.

I think this is because the cache key is completely independent of the paths included in the cache paths.

My naive expectation would be that when the cache paths get updated, the stored cache would also be updated.

Bitrise info

• Build URL: Please share the build URL with us.
• Bitrise Support enabled: NO
• Log:

Input:
- verbose: false
Cache key: {{ .OS }}-{{ .Arch }}-gradle-cache-{{ checksum "**/*.gradle*" "**/gradle-wrapper.properties" "gradle.properties" "gradle/libs.versions.toml" }}
Cache paths:
~/.gradle/caches/jars-*
~/.gradle/caches/modules-*/files-*
~/.gradle/caches/modules-*/metadata-*
~/.gradle/caches/**/generated-gradle-jars/*.jar
~/.gradle/caches/**/kotlin-dsl
~/.gradle/wrapper
.gradle/configuration-cache
~/.gradle/jdks
Evaluating key template: {{ .OS }}-{{ .Arch }}-gradle-cache-{{ checksum "**/*.gradle*" "**/gradle-wrapper.properties" "gradle.properties" "gradle/libs.versions.toml" }}
Build trigger doesn't have an explicit git commit hash, using the Git Clone Step's output for the .CommitHash template variable (value: 28a930656d402b20a889e96c0c32b33d0f95d5f2)
Cache key: linux-amd64-gradle-cache-babc08d1de4d839747bdf8969d5880296e0217d6a8ca9ee43b641e90eefaddf2
Cache path doesn't exist: .gradle/configuration-cache
Cache save can be skipped, reason: a cache with the same key was restored in the workflow, new cache would have the same content
  

Steps to reproduce

1. Configure a project using [email protected]
2. Run, and ensure Bitrise saves a cache copy
3. Update to save-gradle-cache@main (or, presumably, 1.1.4 once this is released)
4. Run the build again

What I expect to happen

• Saved cache includes data at the new path

What actually happens

• Cache is unchanged.

[ofalvai]

@esteluk this is an intentional optimization to avoid uploading the same cache archives in every build, slowing down builds where the cache (key) doesn't change. Ideally, the cache key and the cached files should be in sync, do you think this is not the case here?

By the way, PR #11 is only merged, but I did not create a new release of it on Friday, so you were testing the version without that PR I think. I'm creating a new release of the change now.

[liamnichols]

Long time no see @esteluk 😄 👋

Yeah this was a bit of a pain while I was testing, because I had to delete the cache manually before re-running. I did think about how to solve this but concluded that it would be quite difficult to make sure that the cache key was in sync across restore/save steps in the event that a workflow used two different versions.

Another idea was that maybe there could be an option for a custom cache-key-prefix but ultimately I guess there aren't many use cases where this would be useful.

I suppose that the easiest way to solve this moving forward would be to just recommend that people clear the cache in the release notes? It'll eventually happen over time as caches invalidate automatically/people modify their gradle.build files etc.

[esteluk]

this is an intentional optimization to avoid uploading the same cache archives in every build, slowing down builds where the cache (key) doesn't change

Yeah that makes a lot of sense, but I could also see how explicitly adding new subfolders to the cache would be something that might (ideally) modify the key a little bit?

Is it the expected behaviour that customers are manually deleting the cache in order to benefit from the 1.1.4 release? – I think the same issue also applied to 1.1.2. I guess that's fine if so, but like @liamnichols says it might be worth calling out in the release notes.

so you were testing the version without that PR I think.

I tested against the main branch, so I think it was fine 👍🏽

Long time no see @esteluk 😄 👋

👋🏽 ;)

quite difficult to make sure that the cache key was in sync across restore/save steps in the event that a workflow used two different versions

Yeah, keeping these in sync seems like enough manual effort that it might be a non-starter. If the steps were mature perhaps it would be worth considering making updates to the path a breaking change (which is probably what I'd need to be looking at the release notes), but I don't think that makes sense at the moment.