Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-50424][INFRA] Extract the common content of Dockerfile from Docs, Linter, and SparkR test images #48967

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

panbingkun
Copy link
Contributor

@panbingkun panbingkun commented Nov 26, 2024

What changes were proposed in this pull request?

The pr aims to extract the common content of Dockerfile from Docs, Linter, and SparkR test images.

Why are the changes needed?

In the following files,

  • dev/spark-test-image/docs/Dockerfile

    FROM ubuntu:jammy-20240911.1
    LABEL org.opencontainers.image.authors="Apache Spark project <[email protected]>"
    LABEL org.opencontainers.image.licenses="Apache-2.0"
    LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for Documentation"
    # Overwrite this label to avoid exposing the underlying Ubuntu OS version label
    LABEL org.opencontainers.image.version=""
    ENV FULL_REFRESH_DATE 20241029
    ENV DEBIAN_FRONTEND noninteractive
    ENV DEBCONF_NONINTERACTIVE_SEEN true
    RUN apt-get update && apt-get install -y \

  • dev/spark-test-image/lint/Dockerfile

    FROM ubuntu:jammy-20240911.1
    LABEL org.opencontainers.image.authors="Apache Spark project <[email protected]>"
    LABEL org.opencontainers.image.licenses="Apache-2.0"
    LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for Linter"
    # Overwrite this label to avoid exposing the underlying Ubuntu OS version label
    LABEL org.opencontainers.image.version=""
    ENV FULL_REFRESH_DATE 20241112
    ENV DEBIAN_FRONTEND noninteractive
    ENV DEBCONF_NONINTERACTIVE_SEEN true
    RUN apt-get update && apt-get install -y \

  • dev/spark-test-image/sparkr/Dockerfile

    FROM ubuntu:jammy-20240911.1
    LABEL org.opencontainers.image.authors="Apache Spark project <[email protected]>"
    LABEL org.opencontainers.image.licenses="Apache-2.0"
    LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for SparkR"
    # Overwrite this label to avoid exposing the underlying Ubuntu OS version label
    LABEL org.opencontainers.image.version=""
    ENV FULL_REFRESH_DATE 20241114
    ENV DEBIAN_FRONTEND noninteractive
    ENV DEBCONF_NONINTERACTIVE_SEEN true
    RUN apt-get update && apt-get install -y \

there are some common contents that we should extract for future maintenance and updates.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass GA.

Was this patch authored or co-authored using generative AI tooling?

No.

@panbingkun panbingkun changed the title [Test Only][INFRA] extract common dockerfile [SPARK-50424][INFRA] Extract the common content of Dockerfile from Docs, Linter, and SparkR images Nov 26, 2024
@panbingkun panbingkun changed the title [SPARK-50424][INFRA] Extract the common content of Dockerfile from Docs, Linter, and SparkR images [SPARK-50424][INFRA] Extract the common content of Dockerfile from Docs, Linter, and SparkR test images Nov 26, 2024
@panbingkun panbingkun marked this pull request as ready for review November 26, 2024 11:15
@panbingkun
Copy link
Contributor Author

panbingkun commented Nov 26, 2024

@zhengruifeng
Copy link
Contributor

IMHO, we don't need .r and .py.
For R, we just let it alone, will finally remove it in the future;
For Python, I plan to add more images with different python versions

@panbingkun
Copy link
Contributor Author

IMHO, we don't need .r and .py. For R, we just let it alone, will finally remove it in the future; For Python, I plan to add more images with different python versions

Okay, allow me to update it.

@panbingkun
Copy link
Contributor Author

IMHO, we don't need .r and .py. For R, we just let it alone, will finally remove it in the future; For Python, I plan to add more images with different python versions

Updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants