[RFE] Add support for --link in COPY/ADD

[lucacome]

Description

Buildkit added support for the --link flag. Quoting from the docs:

Enabling this flag in COPY or ADD commands allows you to copy files with enhanced semantics where your files remain independent on their own layer and don't get invalidated when commands on previous layers are changed.

When --link is used your source files are copied into an empty destination directory. That directory is turned into a layer that is linked on top of your previous state.

This is a feature request to add this feature in buildah 🙂

[CypherpunkSamurai]

Hello @lucacome and @TomSweeneyRedHat,

I want to contribute to this issue but am new to the project, How can I get started? :)

[rhatdan]

Well first you would need to add support for --link in
https://github.com/openshift/imagebuilder I believe.
@flouthoc Could you help @CypherpunkSamurai contribute.

[flouthoc]

Hi,

Yes implementation must start from imagebuilder and introduce a new attribute link bool to Copy struct and must be set true when COPY or ADD contains --link and on buildah side if Copy.link == truethen layer must be created on a scratch base, in theory COPY --link must be replaced with a functionality similar to stage where scratch image is base and file is just copied to target path.

COPY --link /src /target

must create a intermediate image equivalent to

FROM scatch
COPY /src /target

[lucacome]

Hi @CypherpunkSamurai

I was wondering if you've had a chance to start working on this? Or do you need any help? 🙂

[CypherpunkSamurai]

Really sorry about this, I had gotten a little busy with college work.

I'll start work asap by this Saturday.

[rhatdan]

No problem college work should come first.

[CypherpunkSamurai]

Hello everyone, really sorry again for stalling this issue.

Ok Here's what I understand from the documentation of the --link tag for COPY and ADD build commands:

• It creates a cached layer that can be reused
• This cached layer is a layer from scratch. (i.e. a simple rootfs with busybox ? doubts here)

Here's how I understand how I need to implement it:

• introduce a new attribute link bool to Copy struct and must be set true when COPY or ADD contains --link
• on buildah side if Copy.link == truethen layer must be created on a scratch base, in theory COPY --link must be replaced with a functionality similar to stage where scratch image is base and file is just copied to target path.

Please check if this is correct

[flouthoc]

Yes SGTM. Pretty much what's written here #4325 (comment) but I'd suggest if you have time to take a look and play with buildkit a bit before implementing this. My explanation in above comment is more conceptual and in past we have seen that buildkit's feature is not completely described in the documentation so its worth verifying edge cases before implementing this.

Note: Buildah mostly tries to match parity with buildkit as much as possible.

[CypherpunkSamurai]

Sure. I need to try both the tools first i guess. Would be interesting. I'll try my best to help this issue :)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[CypherpunkSamurai]

Alright I researched quite a bit about rootfs and build root for the few weeks. I feel comfortable with linux rootfs and how it works.

Um, if you don't mind guys (@flouthoc and @rhatdan) can you please help me point out how buildah handles creating scratch rootfs? (in addition line number would be very cool)

[flouthoc]

@CypherpunkSamurai Its hard to pin the exact line since I'll have to skim the code but I can tell that builder creates a container for FROM scratch instruction you can actually find the part where builder actually creates container.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Since we have not heard any more feedback, closing. Reopen if you want to continue looking into this one.

[lucacome]

@rhatdan I'm still interested in this and I'd like to keep it open. Maybe somebody else can take over from @CypherpunkSamurai ?

[flouthoc]

@CypherpunkSamurai If you are running short on time, I'd request to assign this issue to me :)

[CypherpunkSamurai]

Sure. I'm having my exams right now, probably not a great time to work on this. I better not keep this issue stalling :)

…

On 24 January 2023 10:18:08 AM flouthoc ***@***.***> wrote: @CypherpunkSamurai If you are running short on time, I'd request to assign this issue to me :) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[danishprakash]

@flouthoc If you haven't started, I can take a shot at this.

[rhatdan]

@danishprakash go for it. @flouthoc has gone back to school and has much less time to work on this.

[edwbuck]

Any updates on this one?

[apollo13]

I'd love to see this as well. But I am kinda wondering if it makes sense to reimplement what buildkit does instead of using/embedding buildkit in podman? FWIW, aside from being daemonless (though you kind of can run buildkit like that as well: https://github.com/moby/buildkit?tab=readme-ov-file#daemonless ) podman build imo no longer has any features over docker build(x). The performance gains with buildkits are so big that I am currently reworking a few of our internal pipelines to use buildkit.

[flouthoc]

But I am kinda wondering if it makes sense to reimplement what buildkit does instead of using/embedding buildkit in podman?

@apollo13 Could you please elaborate on this, podman does not uses or embed buildkit as-is most of the features are implemented in buildah as per buildah's design and podman uses buildah not buildkit.

[apollo13]

@flouthoc I was mainly wondering if it makes sense (or were possible) for podman to directly use buildkit instead of buildah. This would allow keeping feature parity more easily and also pick up all performance improvements. Especially with multi stage builds podman is really really much slower since buildkit is able to parallelize many things (unless I missed a flag for podman build)

[flouthoc]

Podman multi-stage build is parallel but you have to use --jobs flag. See https://docs.podman.io/en/latest/markdown/podman-build.1.html#jobs-number

[apollo13]

Oh thank you, how did I miss that (is it new?). That does indeed help for multi-stage builds. Can you recommend any other knobs to tune?

…

On Sun, Aug 18, 2024, at 22:17, flouthoc wrote: Podman multi-stage build is parallel but you have to use `--jobs` flag. See https://docs.podman.io/en/latest/markdown/podman-build.1.html#jobs-number — Reply to this email directly, view it on GitHub <#4325 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAT5C7QBYVLOL3FLASV2D3ZSD6OBAVCNFSM6AAAAAAQ72EQZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJVGM3TQMJWGM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[rgov]

Would it be sensible to silently ignore --link until full support is added? This would make Podman / Buildah compatible with Dockerfiles that use the flag, whereas currently the builds just fail.

From the flag description, it sounds like it is an optimization for cache reuse, but missing a cache reuse opportunity is not a big deal.

On the other hand, if there is a requirement that the layer checksums are exactly the same between Buildah and BuildKit, I don't know for certain if ignoring --link would result in the same checksums?

[danishprakash]

From the flag description, it sounds like it is an optimization for cache reuse, but missing a cache reuse opportunity is not a big deal.

It's not but it's also not good design when we support an option for the sake of docker compat, and it might be misleading to users and might impact workflow.

That being said, I'll start working on this in a week or two, but in the meantime, @rhatdan thoughts on adding dummy flags? A similar sentiment was shared regarding --follow-link in containers/podman#16585 ftr.

[nathanweeks]

If COPY/ADD --link will be initially supported only syntactically, could a warning be emitted to stderr notifying users that --link is currently unsupported and is being ignored?

[rhatdan]

I would be fine with adding a warning, but it would be best if someone would step up and add the functionality.

[roman-dzhadan]

By default, all JVM-based applications built with Micronaut Framework offer 2 gradle's tasks: ./gradlew dockerBuild & ./gradlew dockerBuildNative and both these tasks are failing due to this unsupported --link flag

I've reproduced that on my Fedora Workstation 41 (podman version 5.3.1)
What I'm trying to say, is that Micronaut's community is large enough and keeps growing, but unfortunately, the intended SDLC is blocked in environments, where podman was chosen as a docker substitution

Currently, I haven't found any workarounds on how to successfully execute these gradle tasks against the Fedora/CentOS/RHEL/Podman setup

In the case of ./gradlew dockerBuild the following Dockerfile was generated:

FROM eclipse-temurin:21-jre
WORKDIR /home/app
COPY --link layers/libs /home/app/libs
COPY --link layers/app /home/app/
COPY --link layers/resources /home/app/resources
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/home/app/application.jar"]

and respectively, the job failed with the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dockerBuild'.
> Could not build image: building at STEP "COPY --link layers/libs /home/app/libs": COPY --link is not supported

Analogously, in the case of ./gradlew dockerBuildNative the following Dockerfile was generated:

FROM ghcr.io/graalvm/native-image-community:23-ol9 AS graalvm
WORKDIR /home/app
COPY --link layers/libs /home/app/libs
COPY --link layers/app /home/app/
COPY --link layers/resources /home/app/resources
RUN mkdir /home/app/config-dirs
RUN mkdir -p /home/app/config-dirs/generateResourcesConfigFile
RUN mkdir -p /home/app/config-dirs/io.netty/netty-common/4.1.80.Final
RUN mkdir -p /home/app/config-dirs/ch.qos.logback/logback-classic/1.4.9
COPY --link config-dirs/generateResourcesConfigFile /home/app/config-dirs/generateResourcesConfigFile
COPY --link config-dirs/io.netty/netty-common/4.1.80.Final /home/app/config-dirs/io.netty/netty-common/4.1.80.Final
COPY --link config-dirs/ch.qos.logback/logback-classic/1.4.9 /home/app/config-dirs/ch.qos.logback/logback-classic/1.4.9
RUN native-image --exclude-config .*/libs/netty-codec-http-4.1.115.Final.jar ^/META-INF/native-image/.* --exclude-config .*/libs/netty-codec-http2-4.1.115.Final.jar ^/META-INF/native-image/.* --exclude-config .*/libs/netty-buffer-4.1.115.Final.jar ^/META-INF/native-image/.* --exclude-config .*/libs/netty-handler-4.1.115.Final.jar ^/META-INF/native-image/.* --exclude-config .*/libs/netty-transport-4.1.115.Final.jar ^/META-INF/native-image/.* --exclude-config .*/libs/netty-common-4.1.115.Final.jar ^/META-INF/native-image/.* -cp /home/app/libs/*.jar:/home/app/resources:/home/app/application.jar --no-fallback -o application -H:ConfigurationFileDirectories=/home/app/config-dirs/generateResourcesConfigFile,/home/app/config-dirs/io.netty/netty-buffer/4.1.80.Final,/home/app/config-dirs/io.netty/netty-common/4.1.80.Final,/home/app/config-dirs/io.netty/netty-transport/4.1.80.Final,/home/app/config-dirs/io.netty/netty-codec-http/4.1.80.Final,/home/app/config-dirs/io.netty/netty-handler/4.1.80.Final,/home/app/config-dirs/io.netty/netty-codec-http2/4.1.80.Final,/home/app/config-dirs/ch.qos.logback/logback-classic/1.4.9 villa.estate.automation.VillaEstateAutomation
FROM cgr.dev/chainguard/wolfi-base:latest
EXPOSE 8080
COPY --link --from=graalvm /home/app/application /app/application
ENTRYPOINT ["/app/application"]

and that job was failed as well with the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dockerBuildNative'.
> Could not build image: building at STEP "COPY --link layers/libs /home/app/libs": COPY --link is not supported