Multi-stage Docker builds

[camertron]

This discussion pertains to the article on the Evil Martians blog entitled "Kubing Rails: stressless Kubernetes deployments with Kuby". It is part of a series of discussions pertaining to the article that will hopefully help improve Kuby and its ecosystem.

From the article:

When we manually craft Rails dockerfiles, we usually use multi-stage builds to produce all kinds of assets. We only include the necessary files in the final one. Kuby doesn’t do that. At least, not yet 😉

I would love to add multi-stage builds to Kuby, but doing so for a Ruby + Rails app isn't as straightforward as the golang example in the Docker documentation. The necessary app and dependency files are easy enough to copy over, but what about executables like ruby and node, and shared objects like openssl, etc? I feel like any assumptions Kuby makes here will inevitably be frequently the cause of subtle bugs, especially if developers choose to customize, for example, the base image.

/cc @palkan

[palkan]

Here is an example Dockerfile (I implied in the article):

ARG RUBY_VERSION

# === Base image =======================================================================================================

FROM ruby:${RUBY_VERSION}-slim-buster as base

ARG NODE_MAJOR
ARG POSTGRES_VERSION
ARG BUNDLER_VERSION
ARG YARN_VERSION

# Common dependencies
RUN apt-get update -qq \
  && apt-get dist-upgrade -y \
  && DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
    build-essential \
    gnupg2 \
    pkg-config \
    ca-certificates \
    curl \
    wget \
    less \
    git \
    shared-mime-info \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log

# Node
RUN curl -sSL https://deb.nodesource.com/setup_${NODE_MAJOR}.x | bash -

# Postgres client
RUN curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
  && echo 'deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main' ${POSTGRES_VERSION} > /etc/apt/sources.list.d/pgdg.list

# Yarn
# https://github.com/docker/for-mac/issues/5864
RUN wget https://dl.yarnpkg.com/debian/pubkey.gpg \
  && cat pubkey.gpg | apt-key add - \
  && rm pubkey.gpg \
  && echo 'deb http://dl.yarnpkg.com/debian/ stable main' > /etc/apt/sources.list.d/yarn.list

# App's dependencies
RUN apt-get update -qq \
  && DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
    postgresql-client-${POSTGRES_VERSION} \
    libpq-dev \
    nodejs \
    yarn=${YARN_VERSION}-1 \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log

RUN mkdir /app

# Bundler
ENV LANG=C.UTF-8 \
  BUNDLE_JOBS=4 \
  BUNDLE_RETRY=3
RUN gem update --system \
  && gem install --default bundler:${BUNDLER_VERSION}

EXPOSE 3000

# === Image that installs libraries and compiles assets for production image ===========================================

FROM base as production-builder

# Configure bundler paths after installing bundler itself to avoid problems with creating
ENV RAILS_ENV=production \
  LANG=C.UTF-8 \
  BUNDLE_JOBS=4 \
  BUNDLE_RETRY=3 \
  BUNDLE_APP_CONFIG=/home/project/bundle \
  BUNDLE_PATH=/home/project/bundle \
  GEM_HOME=/home/project/bundle

# Container user
RUN groupadd --gid 1005 project \
  && useradd --uid 1005 --gid project --shell /bin/bash --create-home project

USER project

RUN mkdir /home/project/app

WORKDIR /home/project/app

# JS packages
COPY --chown=project:project package.json yarn.lock ./
RUN yarn install --check-files

# Ruby gems
COPY --chown=project:project Gemfile Gemfile.lock .ruby-version .rails-version ./
# other gemfiles
COPY --chown=project:project engines/core/*.gemspec ./engines/core/


RUN mkdir $BUNDLE_PATH \
  && bundle config --local deployment 'true' \
  && bundle config --local path "${BUNDLE_PATH}" \
  && bundle config --local without 'development test' \
  && bundle config --local clean 'true' \
  && bundle config --local no-cache 'true' \
  && bundle install --jobs=${BUNDLE_JOBS} \
  && rm -rf $BUNDLE_PATH/ruby/2.7.0/cache/* \
  && rm -rf /home/project/.bundle/cache/*

# Code
COPY --chown=project:project . .

RUN bundle exec rails assets:precompile


# === Production image =================================================================================================

FROM ruby:${RUBY_VERSION}-slim-buster AS production

ARG POSTGRES_VERSION

# Common dependencies
RUN apt-get update -qq \
  && apt-get dist-upgrade -y \
  && DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
    curl \
    gnupg2 \
    less \
    tzdata \
    time \
    locales \
    shared-mime-info \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log \
  && update-locale LANG=C.UTF-8 LC_ALL=C.UTF-8

# Postgres client for rails dbconsole -p
RUN curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
  && echo 'deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main' ${POSTGRES_VERSION} > /etc/apt/sources.list.d/pgdg.list \
  && apt-get update -qq \
  && DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
    postgresql-client-${POSTGRES_VERSION} \
    libpq5 \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log

RUN gem update --system

ENV RAILS_ENV=production \
  MALLOC_ARENA_MAX=2 \
  BUNDLE_APP_CONFIG=/home/project/bundle \
  BUNDLE_PATH=/home/project/bundle \
  GEM_HOME=/home/project/bundle \
  PATH="/home/project/app/bin:${PATH}" \
  LANG=C.UTF-8 \
  LC_ALL=C.UTF-8

# Container user
RUN groupadd --gid 1005 project \
  && useradd --uid 1005 --gid project --shell /bin/bash --create-home project

RUN mkdir /home/project/app

WORKDIR /home/project/app

USER project

EXPOSE 3000

RUN gem install --default bundler -v "${BUNDLER_VERSION}"

# Code
COPY --chown=project:project . .

# Artifacts
COPY --from=production-builder $BUNDLE_PATH $BUNDLE_PATH
# App-specific build artifacts
COPY --from=production-builder /home/project/app/public/packs /home/project/app/public/packs
COPY --chown=project:project --from=production-builder /home/project/app/tmp/cache/bootsnap* /home/project/app/tmp/cache/

CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]

[camertron]

Fascinating, thank you 👍