Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pex files with different content gets same stamp #106

Open
elthariel opened this issue Apr 4, 2023 · 5 comments · Fixed by #107
Open

pex files with different content gets same stamp #106

elthariel opened this issue Apr 4, 2023 · 5 comments · Fixed by #107
Labels
bug Something isn't working

Comments

@elthariel
Copy link
Contributor

Hi !

I'm building a small CLI tool and I'm using please to produce a .pex file that I want to distribute to my user.

My problem is that if I distribute a newer version of the .pex file, it appears to get the same stamp, and will try to extract to the same cache folder. Since the pex has already been extracted here, it's not extracted again, and the old code runs.

If my understanding is correct, the __PEX_STAMP__ value, which is used to select the path where the pex file is extracted comes from $RULE_HASH which doesn't change when the built content changes, but only when the rules themselves changes.

I'm not sure I really understand why the rule hash is used instead of the content of the built files themselves. Since I'm not building for a container use case, I need to be able to update the app of my user's boxes.

Would you accept a PR allowing to use the hash of the files instead of the rule hash ? Would that be a fix ?

@elthariel elthariel changed the title .pex with different content gets same stamp pex files with different content gets same stamp Apr 4, 2023
@elthariel
Copy link
Contributor Author

Also, it's worth stating that my pex is big enough that using PEX_NOCACHE is not an option (and it would be a terrible UX for my users)

@elthariel
Copy link
Contributor Author

What I think I'd like to do, is to use that content hash of the "_%s#lib" sub-target

@elthariel
Copy link
Contributor Author

Hum... This is sadly not a fixed problem. Something is wrong with my implementation, and your input, dear @samwestmoreland would be welcomed.

So this works when the content of the sources of the python_binary get modified, but not when the files of dependent python_libraries are changed.

@elthariel
Copy link
Contributor Author

Do you think it could work to split the last rule which gathers everything in two ?

  • One that would gather all the .pex.zip, .wheel and resources
  • One that would add the preamble and produce the final pex

We would then be able to put the rule which inject the stamp in the pex in the middle and it would have all the resources available to compute a proper hash.

Do you think it could work ?

@Tatskaari Tatskaari reopened this Apr 20, 2023
@Tatskaari Tatskaari added the bug Something isn't working label Apr 20, 2023
@samwestmoreland
Copy link
Contributor

Hey, right I see. Yeap, I think that sounds very reasonable. You essentially want to figure out everything you need that you want to be in the hash, i.e. the deps and the lib_rule, and build them beforehand in order to generate the stamp. Are you happy to create the PR? I can help out if you get stuck!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants