Lab 07: Building Containers with Docker

Building Containers with Docker

So far we've been pulling existing signed images using Singularity. We've learned how to run these containers using shell and exec, and we've even seen that docker images usually seem to run fine via Singularity.

Although we could use a pre-existing container for the splice-aware alignment software STAR, we're going to create and host our own image using Docker. Why use Docker and not Singularity? Aside from gaining a well-rounded skillset, Docker seems to play well on both MacOS and Windows, while Singularity isn't yet optimized for MacOS.

Installing Docker Desktop and Creating DockerHub account

The following steps are completely optional, but if you foresee yourself creating custom images these practices should be beneficial.

First, you'll need to install the desktop app and CLI tools for Docker Desktop.

Then, you'll need to create a free account at DockerHub.

Creating an Image for STAR

Per the "COMPILING FROM SOURCE" documentation for STAR, we see:

git clone https://github.com/alexdobin/STAR.git

cd STAR/source
make STAR

This tells us that if we want to create an image from scratch, at bare minimum we will require git and make. If you've ever compiled software from source before, you know that the make command is dependent on your system's compiler and maybe a handful of other common development packages. Typically, software maintainers will provide a list of prerequisite packages to successfully compile from source. In reality, there will probably be some trial and error to discover the minimum set of packages required to get a package installed successfully. For example, STAR requires binutils, xxd, and _____ in addition to the typical packages for compilation.

Parts of a Dockerfile

The structure of a Dockerfile will depend on the requirements and intended application of the final image.

See best practices for more information.

In general, the potential parts of a Dockerfile are the following types of instructions:

• FROM - this will pull a base image, see some official base images
• COPY - this copies files from your working directory into the image at build time
• RUN - run actual commands from the base image (this is where most of the action happens)
• CMD - these are directions to execute at container run time
• ENTRYPOINT - to call container as executable from a specified location, if not specified defaults to /bin/sh

Writing a Dockerfile

For simplicity's sake, we'll be focusing on building a completely self-contained image that requires no local files to be copied. We'll need the FROM and RUN instructions.

FROM debian:12.2

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      git-all \
      ca-certificates \
      make \
      gcc \
      g++ \
      zlib1g-dev \
      libbz2-dev \
      libboost-all-dev \
      binutils \
      xxd \
      procps

RUN cd ~ && \
  git clone https://github.com/alexdobin/STAR.git && \
  cd STAR/source && \
  make STAR && \
  mv STAR /usr/local/bin

We begin this Dockerfile with the FROM instruction, which tells us which base layer to use. This image will be built on Debian 12.2. Although the STAR documentation suggested we would require git and make to compile the software, there are many other developer tools that are assumed to be present. Docker base images are typically "bare-bones" and won't come pre-loaded with these tools, and so we'll need to acquire many of them using 'apt install` before we can compile STAR.

⚠️ Images require the package "procps" in order to run successfully in Nextflow.

Notice we use RUN multiple times. The first instruction downloads a list of packages, and the second performs the download for STAR source code, compiles it, then moves the compiled STAR program to /usr/local/bin so it will be available when using the container.

Building an image from a Dockerfile

Once our directory has a file called 'Dockerfile' with all of the appropriate instructions we want, we can build it!

While in the directory, run the following docker command:

sudo docker build --platform linux/amd64 -t star .

Let's break that down.

• Building typically will involve root privilege, so we require sudo.
• The specific Docker command docker build tells Docker that we are constructing an image.
• The --platform linux/amd64 variable specifies the type of host machine we want to use our image on. This is important because by default Docker may be using your host hardware to assume the host hardware (e.g., building on Apple M2 may create an image that can't be used elsewhere).
• The t is our name for this image, which we have named "star". This can also contain a "tag" such as "star:1.0", but if we provide no tag it will default to "latest".
• The . means to look for the Dockerfile in the current working directory.

If the image successfully builds, we should see it when we run:

docker images

Pushing our new image to Docker Hub

If you have an existing Docker Hub account, you'll need to first log in to connect your image to that repo.

docker login

Now that we have a named image, we can push it to our personal Docker Hub repository for use by the community!

The first step will be to make a new tag for our image that reflects our Docker Hub username.

docker tag star <username>/star

The final step is to push our image.

sudo docker push <username>/star

Your image is now available for use!

For more inspiration and examples of how to produce a Dockerfile, check out any Dockerfiles in the BioContainers recipes repository.