Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image: include files from git reproducibly #3555

Merged
merged 1 commit into from
Dec 19, 2024
Merged

Conversation

burgerdev
Copy link
Contributor

@burgerdev burgerdev commented Dec 18, 2024

Context

Our image build includes source files from git. Since git respects the users umask, the files in the image might have different permissions depending on the system executing the build.

Proposed change(s)

  • Add a new Bazel rule reproducible_tar that normalizes git subtrees for image inclusion.

Additional info

The added rule depends on gnutar and is unlikely to work on vanilla darwin (cc @elchead). This trades off portability for correctness of the rootfs. Some alternatives:

  • Use pkg_tar with a mode 0555. Downside: our config files are now executable.
  • Split sources for inclusion into two subtrees by desired permission bits. Downside: more rules, unintuitive layout.
  • Explicitly list the files that should be executable in pkg_tar(modes={}). Downside: need to maintain names in two places.

Checklist

@burgerdev burgerdev added the no changelog Change won't be listed in release changelog label Dec 18, 2024
@burgerdev burgerdev added this to the v2.21.0 milestone Dec 18, 2024
@burgerdev burgerdev requested a review from msanft as a code owner December 18, 2024 07:45
@burgerdev burgerdev requested a review from elchead December 18, 2024 07:45
Copy link

netlify bot commented Dec 18, 2024

Deploy Preview for constellation-docs canceled.

Name Link
🔨 Latest commit 0747305
🔍 Latest deploy log https://app.netlify.com/sites/constellation-docs/deploys/67640780c0db8800085c2660

@burgerdev burgerdev force-pushed the burgerdev/image-repro branch from b3d4cc6 to a8fdd6b Compare December 18, 2024 07:51
@msanft
Copy link
Contributor

msanft commented Dec 18, 2024

The added rule depends on gnutar and is unlikely to work on vanilla darwin (cc @elchead). This trades off portability for correctness of the rootfs.

Why aren't you able to use GNU tar on Darwin? Do they require exotic syscalls that Darwin doesn't support?

Copy link
Contributor

@msanft msanft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for spotting this! Changes LGTM

# from pkg_tar, where the mode is a fixed octal or needs to be passed for each file individually.
#
# NOTE: this rule dereferences symbolic links and hard links to weed out sources of nondeterminism.
def reproducible_tar(name, srcs = [], visibility = []):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put a notice here that this requires GNU tar? Maybe explicitly target that notice to MacOS users, so they know that they have to install package X to run this.

@burgerdev burgerdev force-pushed the burgerdev/image-repro branch from a8fdd6b to 66eaf5a Compare December 18, 2024 09:22
Copy link
Contributor

@elchead elchead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine by me because building images needs to be done in the CI anyways

@burgerdev
Copy link
Contributor Author

Unfortunately, I was ignorant of the heavy lifting that rules_pkgs does here when it comes to figuring out the correct paths. Fixing it requires reimplementing a lot of things, so I'd rather opt for the much simpler alternative (a), making everything executable.

@burgerdev burgerdev force-pushed the burgerdev/image-repro branch from 66eaf5a to 0747305 Compare December 19, 2024 11:46
@burgerdev burgerdev merged commit 50ab77e into main Dec 19, 2024
54 checks passed
@burgerdev burgerdev deleted the burgerdev/image-repro branch December 19, 2024 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no changelog Change won't be listed in release changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants