[bazel] Make pip wheels install reproducible #20872
Conversation
When installing python packages using pip, we use a custom script that invokes python directly. By default, python will timestamp bytecode files hence making the builds non-reproducible which is bad for CI. Python also uses a random seed for hash. This commit sets two environment variables when running pip that force python to use a deterministic seed and to use a fixed timestamp instead of the current time. Signed-off-by: Amaury Pouly <[email protected]>
There was a problem hiding this comment.
Thanks for making this change, I agree that we should think about merging our usage of rules_python in the future.
You've shown these vars make the python wheels reproducible in your experimental PR so I'm happy that these work as intended.
|
Kokoro has failed twice here and I'm reluctant to waive it because I think it might use this Python setup. @pamaury do you know the procedure for debugging Kokoro errors? The message is about the results file being missing:
|
|
I don't know how kokoro works unfortunately, and we have no visibility in it. I doubt this change would affect it though because we it does not change what is built. Maybe let's wait for @timothytrippel to look into it and confirm that it is unrelated. |
|
I see Kokoro has failed on master too, so I'm sure it's fine |
|
Sorry for the late reply. This LGTM. Not sure how this would impact kokoro. But we can keep an eye on it. |
jwnrt
added
the
CherryPick:earlgrey_es_sival
nbdd0121
added
CherryPick:earlgrey_es_sival
|
Successfully created backport PR for |
When installing python packages using pip, we use a custom script that invokes python directly. By default, python will timestamp bytecode files hence making the builds non-reproducible which is bad for CI. Python also uses a random seed for hash. This commit sets two environment variables when running pip that force python to use a deterministic seed and to use a fixed timestamp instead of the current time.
Note: we should probably get rid of our custom fork of rules python and our custom pip install script. This would be a bigger change that can be done at a later stage.