Skip to content

Commit

Permalink
add docker release to release pipeline
Browse files Browse the repository at this point in the history
  • Loading branch information
mikealfare committed Apr 5, 2024
1 parent 336d592 commit 3048bfc
Show file tree
Hide file tree
Showing 4 changed files with 146 additions and 0 deletions.
21 changes: 21 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ jobs:
url: ${{ vars.PYPI_PROJECT_URL }}
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
outputs:
version: ${{ steps.release-inputs.outputs.version }}
is-docker-release: ${{ steps.semver.outputs.is-pre-release == 0 }}

steps:
- name: Check out repository
Expand All @@ -43,6 +46,13 @@ jobs:
version=$(hatch version)
archive_name=dbt-postgres-$version-${{ inputs.deploy-to }}
echo "archive-name=$archive_name" >> $GITHUB_OUTPUT
echo "version=version" >> $GITHUB_OUTPUT
- name: Audit version to determine if it is a pre-release
id: semver
uses: dbt-labs/actions/[email protected]
with:
version: ${{ steps.release-inputs.outputs.version }}

- name: Build `dbt-postgres`
uses: dbt-labs/dbt-adapters/.github/actions/build-hatch@main
Expand All @@ -54,3 +64,14 @@ jobs:
with:
pypi-repository-url: ${{ vars.PYPI_REPOSITORY_URL }}
archive-name: ${{ steps.release-inputs.outputs.archive-name }}

docker-release:
name: "Docker Release"
needs: [release]
if: ${{ needs.release.outputs.is-docker-release }}
permissions:
packages: write
uses: dbt-labs/dbt-release/.github/workflows/release-docker.yml@main
with:
package: "dbt-postgres"
version_number: ${{ needs.release.outputs.version }}
33 changes: 33 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
ARG build_for=linux/amd64

FROM --platform=$build_for python:3.10.7-slim-bullseye as base

Check warning on line 3 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Using Platform Flag with FROM Command

Rule ID: c5e5995a-7d8e-4fbb-8dce-880a79438927 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.{{FROM --platform=$build_for python:3.10.7-slim-bullseye as base}} Don't use '--platform' flag with FROM
Raw output
Expected: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.{{FROM --platform=$build_for python:3.10.7-slim-bullseye as base}} shouldn't use the flag '--platform'
Found: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.{{FROM --platform=$build_for python:3.10.7-slim-bullseye as base}} uses the flag '--platform'

# ref is updated automatically every final release via bumpversion
ARG [email protected]

RUN apt-get update \

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'make' has version defined
Found: Package 'make' does not have version defined

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'ssh-client' has version defined
Found: Package 'ssh-client' does not have version defined

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'build-essential' has version defined
Found: Package 'build-essential' does not have version defined

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'software-properties-common' has version defined
Found: Package 'software-properties-common' does not have version defined

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'ca-certificates' has version defined
Found: Package 'ca-certificates' does not have version defined

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'git' has version defined
Found: Package 'git' does not have version defined

Check warning on line 8 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Apt Get Install Pin Version Not Defined

Rule ID: 8dabde7b-ee7e-440a-8b59-73636b0cfda5 Severity: Medium Resource: FROM={{--platform=$build_for python:3.10.7-slim-bullseye as base}}.RUN={{apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends git ssh-client software-properties-common make build-essential ca-certificates libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} When installing a package, its pin version should be defined
Raw output
Expected: Package 'libpq-dev' has version defined
Found: Package 'libpq-dev' does not have version defined
&& apt-get dist-upgrade -y \
&& apt-get install -y --no-install-recommends \
git \
ssh-client \
software-properties-common \
make \
build-essential \
ca-certificates \
libpq-dev \
&& apt-get clean \
&& rm -rf \
/var/lib/apt/lists/* \
/tmp/* \
/var/tmp/*

ENV PYTHONIOENCODING=utf-8
ENV LANG=C.UTF-8

RUN python -m pip install --upgrade pip setuptools wheel --no-cache-dir

WORKDIR /usr/app/dbt/
ENTRYPOINT ["dbt"]

FROM base as dbt-postgres

Check failure on line 32 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Missing User Instruction

Rule ID: e54afcf9-dc71-484a-8967-d930e3044062 Severity: High Resource: FROM={{base as dbt-postgres}} A user should be specified in the dockerfile, otherwise the image will run as root
Raw output
Expected: The 'Dockerfile' should contain the 'USER' instruction
Found: The 'Dockerfile' does not contain any 'USER' instruction

Check notice on line 32 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Healthcheck Instruction Missing

Rule ID: db295f99-0fff-4e7b-9906-ec2a057f384b Severity: Low Resource: FROM={{base as dbt-postgres}} Ensure that HEALTHCHECK is being used. The HEALTHCHECK instruction tells Docker how to test a container to check that it is still working
Raw output
Expected: Dockerfile should contain instruction 'HEALTHCHECK'
Found: Dockerfile doesn't contain instruction 'HEALTHCHECK'
RUN python -m pip install --no-cache-dir "dbt-postgres @ git+https://github.com/dbt-labs/${dbt_postgres_ref}"

Check warning on line 33 in docker/Dockerfile

View check run for this annotation

Wiz Inc. (266a8a9c32) / Wiz IaC Scanner

Unpinned Package Version in Pip Install

Rule ID: 1f0d05d7-8caf-4f04-bc60-332d472de5a9 Severity: Medium Resource: FROM={{base as dbt-postgres}}.{{RUN python -m pip install --no-cache-dir "dbt-postgres @ git+https://github.com/dbt-labs/${dbt_postgres_ref}"}} Package version pinning reduces the range of versions that can be installed, reducing the chances of failure due to unanticipated changes
Raw output
Expected: RUN instruction with 'pip/pip3 install <package>' should use package pinning form 'pip/pip3 install <package>=<version>'
Found: RUN instruction python -m pip install --no-cache-dir "dbt-postgres @ git+https://github.com/dbt-labs/[email protected]" does not use package pinning form
70 changes: 70 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Docker for dbt
This docker file is suitable for building dbt Docker images locally or using with CI/CD to automate populating a container registry.


## Building an image:
This Dockerfile can create images for the following target: `dbt-postgres`

In order to build a new image, run the following docker command.
```
docker build --tag <your_image_name> --target dbt-postgres <path/to/dockerfile>
```
---
> **Note:** Docker must be configured to use [BuildKit](https://docs.docker.com/develop/develop-images/build_enhancements/) in order for images to build properly!
---

By default the images will be populated with the most recent release of `dbt-postgres`. If you need to use a different version you can specify it by git ref using the `--build-arg` flag:
```
docker build --tag <your_image_name> \
--target dbt-postgres \
--build-arg dbt_postgres_ref=<git_ref> \
<path/to/dockerfile>
```

### Examples:
To build an image named "my-dbt" that supports Snowflake using the latest releases:
```
cd dbt-core/docker
docker build --tag my-dbt --target dbt-postgres .
```

To build an image named "my-other-dbt" that supports Snowflake using the adapter version 1.0.0b1:
```
cd dbt-core/docker
docker build \
--tag my-other-dbt \
--target dbt-postgres \
--build-arg [email protected] \
.
```

## Special cases
There are a few special cases worth noting:

* If you need to build against another architecture (linux/arm64 in this example) you can override the `build_for` build arg:
```
docker build --tag my_dbt \
--target dbt-postgres \
--build-arg build_for=linux/arm64 \
<path/to/dockerfile>
```

Supported architectures can be found in the python docker [dockerhub page](https://hub.docker.com/_/python).

## Running an image in a container:
The `ENTRYPOINT` for this Dockerfile is the command `dbt` so you can bind-mount your project to `/usr/app` and use dbt as normal:
```
docker run \
--network=host \
--mount type=bind,source=path/to/project,target=/usr/app \
--mount type=bind,source=path/to/profiles.yml,target=/root/.dbt/profiles.yml \
my-dbt \
ls
```
---
**Notes:**
* Bind-mount sources _must_ be an absolute path
* You may need to make adjustments to the docker networking setting depending on the specifics of your data warehouse/database host.

---
22 changes: 22 additions & 0 deletions docker/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# - VERY rudimentary test script to run latest + specific branch image builds and test them all by running `--version`
# TODO: create a real test suite

clear \
&& echo "\n\n"\
"########################################\n"\
"##### Testing dbt-postgres latest #####\n"\
"########################################\n"\
&& docker build --tag dbt-postgres \
--target dbt-postgres \
docker \
&& docker run dbt-postgres --version \
\
&& echo "\n\n"\
"#########################################\n"\
"##### Testing dbt-postgres-1.0.0b1 #####\n"\
"#########################################\n"\
&& docker build --tag dbt-postgres-1.0.0b1 \
--target dbt-postgres \
--build-arg [email protected] \
docker \
&& docker run dbt-postgres-1.0.0b1 --version

0 comments on commit 3048bfc

Please sign in to comment.