Skip to content
This repository has been archived by the owner on Apr 19, 2023. It is now read-only.

Publish dockerfile #30

Draft
wants to merge 12 commits into
base: develop
Choose a base branch
from
Draft

Publish dockerfile #30

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
3ffab11
Add clang-format along with the llvm installation and set the correct…
Mar 5, 2022
5603311
Add cmake_format python package to the docker container
Mar 5, 2022
b3764f3
Fix CMake installation process per https://apt.kitware.com/
ddalcino Apr 3, 2022
fc99d63
Move Dockerfile to dedicated folder
ddalcino Apr 3, 2022
1f9538d
Add basic tests for installed programs.
ddalcino Apr 3, 2022
12532a6
Cleanup anything that should be added by users
ddalcino Apr 3, 2022
10cc4d2
Use dependent container in .devcontainer
ddalcino Apr 3, 2022
c8e855c
add workflow to publish container
ddalcino Apr 3, 2022
9d67086
Fix name of CI action
ddalcino Apr 3, 2022
dc14794
fix CI errors
ddalcino Apr 3, 2022
b0a362e
Apply suggestions from code review
ddalcino Apr 4, 2022
5db36c9
Apply suggestions from code review
ddalcino Apr 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 3 additions & 119 deletions .devcontainer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,120 +1,4 @@
# [Choice] bionic (18.04), focal (20.04)
ARG VARIANT="focal"
FROM ubuntu:${VARIANT}
# FROM ghcr.io/cpp_best_practices/cpp:0.1.0 # TODO: activate
FROM ghcr.io/ddalcino/cpp:0.1.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How the variables like ${VARIANT}, ${LLVM_VER} etc. are passed from the devcontainer.json file to the actual dev container that VSCode use with setup?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add a matrix to .github/workflows/publish-dockerfile.yml that builds separate containers with all the different variables, and names them appropriately. It should not be too difficult to do.


# Restate the variant to use it later on in the llvm and cmake installations
ARG VARIANT

# Install necessary packages available from standard repos
RUN apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
software-properties-common wget apt-utils file zip \
openssh-client gpg-agent socat rsync \
make ninja-build git \
python3 python3-pip

# Install conan
RUN python3 -m pip install --upgrade pip setuptools && \
python3 -m pip install conan && \
conan --version

# By default, anything you run in Docker is done as superuser.
# Conan runs some install commands as superuser, and will prepend `sudo` to
# these commands, unless `CONAN_SYSREQUIRES_SUDO=0` is in your env variables.
ENV CONAN_SYSREQUIRES_SUDO 0
# Some packages request that Conan use the system package manager to install
# a few dependencies. This flag allows Conan to proceed with these installations;
# leaving this flag undefined can cause some installation failures.
ENV CONAN_SYSREQUIRES_MODE enabled

# User-settable versions:
# This Dockerfile should support gcc-[7, 8, 9, 10, 11] and clang-[10, 11, 12, 13]
# Earlier versions of clang will require significant modifications to the IWYU section
ARG GCC_VER="11"
# Add gcc-${GCC_VER}
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test && \
apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
gcc-${GCC_VER} g++-${GCC_VER} gdb

# Set gcc-${GCC_VER} as default gcc
RUN update-alternatives --install /usr/bin/gcc gcc $(which gcc-${GCC_VER}) 100
RUN update-alternatives --install /usr/bin/g++ g++ $(which g++-${GCC_VER}) 100

ARG LLVM_VER="13"
# Add clang-${LLVM_VER}
ARG LLVM_URL="http://apt.llvm.org/${VARIANT}/"
ARG LLVM_PKG="llvm-toolchain-${VARIANT}-${LLVM_VER}"
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - 2>/dev/null && \
add-apt-repository -y "deb ${LLVM_URL} ${LLVM_PKG} main" && \
apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
clang-${LLVM_VER} lldb-${LLVM_VER} lld-${LLVM_VER} clangd-${LLVM_VER} \
llvm-${LLVM_VER}-dev libclang-${LLVM_VER}-dev clang-tidy-${LLVM_VER}

# Set the default clang-tidy, so CMake can find it
RUN update-alternatives --install /usr/bin/clang-tidy clang-tidy $(which clang-tidy-${LLVM_VER}) 1

# Set clang-${LLVM_VER} as default clang
RUN update-alternatives --install /usr/bin/clang clang $(which clang-${LLVM_VER}) 100
RUN update-alternatives --install /usr/bin/clang++ clang++ $(which clang++-${LLVM_VER}) 100

# Add current cmake/ccmake, from Kitware
ARG CMAKE_URL="https://apt.kitware.com/ubuntu/"
ARG CMAKE_PKG=${VARIANT}
RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null \
| gpg --dearmor - | tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null && \
apt-add-repository -y "deb ${CMAKE_URL} ${CMAKE_PKG} main" && \
apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends cmake cmake-curses-gui

# Install editors
RUN apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
neovim emacs nano

# Install optional dependecies
RUN apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
doxygen graphviz ccache cppcheck

# Install include-what-you-use
ENV IWYU /home/iwyu
ENV IWYU_BUILD ${IWYU}/build
ENV IWYU_SRC ${IWYU}/include-what-you-use
RUN mkdir -p ${IWYU_BUILD} && \
git clone --branch clang_${LLVM_VER} \
https://github.com/include-what-you-use/include-what-you-use.git \
${IWYU_SRC}
RUN CC=clang-${LLVM_VER} CXX=clang++-${LLVM_VER} cmake -S ${IWYU_SRC} \
-B ${IWYU_BUILD} \
-G "Unix Makefiles" -DCMAKE_PREFIX_PATH=/usr/lib/llvm-${LLVM_VER} && \
cmake --build ${IWYU_BUILD} -j && \
cmake --install ${IWYU_BUILD}

# Per https://github.com/include-what-you-use/include-what-you-use#how-to-install:
# `You need to copy the Clang include directory to the expected location before
# running (similarly, use include-what-you-use -print-resource-dir to learn
# exactly where IWYU wants the headers).`
RUN mkdir -p $(include-what-you-use -print-resource-dir 2>/dev/null)
RUN ln -s $(readlink -f /usr/lib/clang/${LLVM_VER}/include) \
$(include-what-you-use -print-resource-dir 2>/dev/null)/include

## Cleanup cached apt data we don't need anymore
RUN apt-get autoremove -y && apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Allow the user to set compiler defaults
ARG USE_CLANG
# if --build-arg USE_CLANG=1, set CC to 'clang' or set to null otherwise.
ENV CC=${USE_CLANG:+"clang"}
ENV CXX=${USE_CLANG:+"clang++"}
# if CC is null, set it to 'gcc' (or leave as is otherwise).
ENV CC=${CC:-"gcc"}
ENV CXX=${CXX:-"g++"}

# Include project
#ADD . /workspaces/cpp_starter_project
#WORKDIR /workspaces/cpp_starter_project

CMD ["/bin/bash"]
# Add your own modifications to the Docker environment here
Copy link
Contributor

@Jason5480 Jason5480 Apr 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

editors were removed. I agree with this change since the most common use cases don't need them. However, you can add the commands commented out as an example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I removed the editors and the commented-out code that you wanted to remove in #8, and I asked you not to. I was trying to make the old Dockerfile do too many things at once, and that was never going to work out. I was wrong. Thank you for disagreeing with me, and being a good sport about the disagreement.

43 changes: 43 additions & 0 deletions .github/workflows/publish-dockerfile.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Build/Publish Dockerfile
on:
push:
paths:
- .github/workflows/publish-dockerfile.yml
- docker
pull_request:
paths:
- .github/workflows/publish-dockerfile.yml
- docker
workflow_dispatch:


jobs:
build:
name: Build and publish container
runs-on: ubuntu-latest
env:
IMAGE_NAME: cpp
VERSION: 0.1.0
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Login to GitHub Container Registry
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.PAT }}

- name: Build and push
uses: docker/build-push-action@v2
with:
file: ./docker/Dockerfile
context: ./docker
push: ${{ github.ref == 'refs/heads/main' }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is insufficient. What we need here is to check that:

  1. we are on the main branch
  2. there isn't already an existing container in our package repository that has the same tag, that a push would mistakenly overwrite. We don't want to overwrite an existing cpp-gcc11-llvm-13:0.1.1 when we have forgotten to bump the version to 0.1.2; there may not be a way to get the original back.

If both these conditions are true, it should be safe to push a new version of the container.

I'm not so experienced with GH workflow syntax, and I'm not clear on how to make this check work. Honestly, I'm not even sure if ${{ github.ref == 'refs/heads/main' }} does what it looks like it does.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm not familiar with GH syntax either. However, this "problem" reminds me why conan introduced revisions for recipes. Docker images (like recipes) have different development life cycles thus you need a way to reference back every change both in code and container to have a reproducible environment. I found this post https://stackoverflow.com/questions/56212495/properly-versioning-docker-images that give some options we have about versioning that might be helpful.

tags: |
ghcr.io/${{ github.repository_owner }}/${{ env.IMAGE_NAME }}:latest
ghcr.io/${{ github.repository_owner }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
120 changes: 120 additions & 0 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# [Choice] bionic (18.04), focal (20.04)
ARG VARIANT="focal"
FROM ubuntu:${VARIANT}

# Restate the variant to use it later on in the llvm and cmake installations
ARG VARIANT

# Install necessary packages available from standard repos
RUN apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also have a non-root user implementation https://github.com/Jason5480/cpp_boilerplate_project/blob/aab9da20a2a29578dd119b543785a5cdf36a6349/.devcontainer/Dockerfile#L11 that works but I would prefer to add it in a separate PR after this merge.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! I'm sure that will be very useful

apt-get install -y --no-install-recommends \
software-properties-common wget apt-utils file zip \
openssh-client gpg-agent socat rsync \
make ninja-build git \
python3 python3-pip

# Install conan and cmake format
RUN python3 -m pip install --upgrade pip setuptools && \
python3 -m pip install conan cmake_format

# By default, anything you run in Docker is done as superuser.
# Conan runs some install commands as superuser, and will prepend `sudo` to
# these commands, unless `CONAN_SYSREQUIRES_SUDO=0` is in your env variables.
ENV CONAN_SYSREQUIRES_SUDO 0
# Some packages request that Conan use the system package manager to install
# a few dependencies. This flag allows Conan to proceed with these installations;
# leaving this flag undefined can cause some installation failures.
ENV CONAN_SYSREQUIRES_MODE enabled
Copy link
Contributor

@Jason5480 Jason5480 Apr 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ADD CONAN_REVISIONS_ENABLED=1. This should also be considered as the default

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am actually considering removing all of the CONAN_* variables; I think they might fall under the category of "things that aren't needed by all users, and don't belong here". They can easily be added to .devcontainer/Dockerfile if necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep here only the bare minimum things that are needed to compile this project. However. I think that conan is needed here. Also, I would like to have consistent behavior between devcontainer and the CI. SO if you remove the conan how the CI will compile this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You raise a good point: We should not release a Dockerfile that cannot build the project. That means we need an extra CI step that tests the Dockerfile, to prove that it can build the project: if it fails this step, the workflow should not be allowed to push the container to GHCI.

The idea of 'removing Conan' should really be, 'move Conan out of docker/Dockerfile and into .devcontainer/Dockerfile'. The point of this change would be to allow users the choice of a different package manager, for instance vcpkg (see our sister project https://github.com/aminya/cpp_vcpkg_project). These users would need to extend docker/Dockerfile to make it work with vcpkg, but they would be able to do so without polluting their environment variables with Conan-specific variables they don't need, and wasting extra disk space on Conan programs they aren't going to use.

Now that I think about it, I think this is outside the scope of this PR, and that moving Conan-specific code should wait for another PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package manager selection could be just another variable in the container. To use the selected package manager, we should just pass the correct CMAKE_TOOLCHAIN_FILE generated by the (CMakeToolchain, CMakeDeps) conan and vcpkg. This is the least intrusive way to set up dependencies since I believe that CMakeLists.txt files should be package manager agnostic. Then the -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake can be passed using a 'conan' spesific preset. Well this is outside of the scope of this PR though.


# User-settable versions:
# This Dockerfile should support gcc-[7, 8, 9, 10, 11] and clang-[10, 11, 12, 13]
# Earlier versions of clang will require significant modifications to the IWYU section
ARG GCC_VER="11"
# Add gcc-${GCC_VER}
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test && \
apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
gcc-${GCC_VER} g++-${GCC_VER} gdb

# Set gcc-${GCC_VER} as default gcc
RUN update-alternatives --install /usr/bin/gcc gcc $(which gcc-${GCC_VER}) 100
RUN update-alternatives --install /usr/bin/g++ g++ $(which g++-${GCC_VER}) 100

ARG LLVM_VER="13"
ddalcino marked this conversation as resolved.
Show resolved Hide resolved
# Add clang-${LLVM_VER}
ARG LLVM_URL="http://apt.llvm.org/${VARIANT}/"
ARG LLVM_PKG="llvm-toolchain-${VARIANT}-${LLVM_VER}"
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - 2>/dev/null && \
add-apt-repository -y "deb ${LLVM_URL} ${LLVM_PKG} main" && \
apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
clang-${LLVM_VER} lldb-${LLVM_VER} lld-${LLVM_VER} clangd-${LLVM_VER} \
llvm-${LLVM_VER}-dev libclang-${LLVM_VER}-dev clang-tidy-${LLVM_VER} clang-format-${LLVM_VER}

# Set the default clang-tidy, so CMake can find it
RUN update-alternatives --install /usr/bin/clang-tidy clang-tidy $(which clang-tidy-${LLVM_VER}) 1

# Set the default clang-format
RUN update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-${LLVM_VER}) 1

# Set clang-${LLVM_VER} as default clang
RUN update-alternatives --install /usr/bin/clang clang $(which clang-${LLVM_VER}) 100
RUN update-alternatives --install /usr/bin/clang++ clang++ $(which clang++-${LLVM_VER}) 100

# Add current cmake/ccmake, from Kitware
ENV KEYRING_FILE="/usr/share/keyrings/kitware-archive-keyring.gpg"
ddalcino marked this conversation as resolved.
Show resolved Hide resolved
RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null \
| gpg --dearmor - | tee "${KEYRING_FILE}" >/dev/null && \
echo "deb [signed-by=${KEYRING_FILE}] https://apt.kitware.com/ubuntu/ ${VARIANT} main" \
| tee /etc/apt/sources.list.d/kitware.list >/dev/null && \
apt-get update && \
rm "${KEYRING_FILE}" && \
apt-get install kitware-archive-keyring && \
export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends cmake cmake-curses-gui

# Install optional dependecies
RUN apt-get update -qq && export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends \
doxygen graphviz ccache cppcheck

# Install include-what-you-use
ENV IWYU /home/iwyu
ENV IWYU_BUILD ${IWYU}/build
ENV IWYU_SRC ${IWYU}/include-what-you-use
RUN mkdir -p ${IWYU_BUILD} && \
git clone --branch clang_${LLVM_VER} \
https://github.com/include-what-you-use/include-what-you-use.git \
${IWYU_SRC}
RUN CC=clang-${LLVM_VER} CXX=clang++-${LLVM_VER} cmake -S ${IWYU_SRC} \
-B ${IWYU_BUILD} \
-G "Unix Makefiles" -DCMAKE_PREFIX_PATH=/usr/lib/llvm-${LLVM_VER} && \
cmake --build ${IWYU_BUILD} -j && \
cmake --install ${IWYU_BUILD}

# Per https://github.com/include-what-you-use/include-what-you-use#how-to-install:
# `You need to copy the Clang include directory to the expected location before
# running (similarly, use include-what-you-use -print-resource-dir to learn
# exactly where IWYU wants the headers).`
RUN mkdir -p $(include-what-you-use -print-resource-dir 2>/dev/null)
RUN ln -s $(readlink -f /usr/lib/clang/${LLVM_VER}/include) \
$(include-what-you-use -print-resource-dir 2>/dev/null)/include

## Cleanup cached apt data we don't need anymore
RUN apt-get autoremove -y && apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Allow the user to set compiler defaults
ARG USE_CLANG
# if --build-arg USE_CLANG=1, set CC to 'clang' or set to null otherwise.
ENV CC=${USE_CLANG:+"clang"}
ENV CXX=${USE_CLANG:+"clang++"}
# if CC is null, set it to 'gcc' (or leave as is otherwise).
ENV CC=${CC:-"gcc"}
ENV CXX=${CXX:-"g++"}

# Check that all required programs have been installed. Fail if anything is missing.
COPY print_versions.sh .
RUN /bin/bash ./print_versions.sh

CMD ["/bin/bash"]
19 changes: 19 additions & 0 deletions docker/print_versions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env bash

set -ex
# Prints versions of all relevant installed tools. Fails if those tools do not exist.

cat /etc/lsb-release
g++ --version
clang++ --version
clang-format --version
python3 --version
pip --version
conan --version
cmake --version
ccmake --version
ddalcino marked this conversation as resolved.
Show resolved Hide resolved
cppcheck --version
doxygen --version
include-what-you-use --version
cmake-format --version