CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

sheurich · 2024-02-26T22:26:11Z

As a follow-on to #439 this change request implements the first step of building a new CI system for Pebble. As this is a testbed environment for Boulder, some of these changes may prove to be useful in the latter context.

Per #434 (comment), GitHub Actions is a reasonable choice for a new CI implementation to supersede Travis. Much, but not all, of the existing functionality is present in this initial set of changes.

Items currently implemented include:

Binary production of pebble and challtestsrv for platforms:
- AMD64
  - darwin
  - linux
  - windows
- ARM64
  - darwin
  - linux
  - windows
Multiplatform docker images published on the GitHub Container Registry for platforms:
- linux/amd64
- linux/arm64
- windows/amd64
Golang linting.
Golang coverage.
Local test scripts for Go and Docker:
- ./build.sh
- ./docker.sh
- ./test.sh

This PR also adds a -version flag to Pebble, which is set during release build to the Git commit ref.

This change builds on earlier work done in #379 and #442.

ldez · 2024-02-26T22:47:55Z

I think we can merge our 2 PRs #442
My PR is the basics, and your PR is the release part.

.github/workflows/ci.yml

sheurich · 2024-02-26T22:51:44Z

I think we can merge our 2 PRs #442

@ldez Yes let's do that. I can take a look tomorrow.

ldez · 2024-02-26T22:53:20Z

I mean I think my PR can be merged and your PR will just need to be "cleaned" to remove the duplication with mine.
By "merge" I meant "complementary", I know it was not clear.

sheurich · 2024-02-26T22:55:27Z

@ldez glad you found #440! I spent some time troubleshooting at the CI level but hadn't checked Pebble itself yet.

.github/workflows/ci.yml

.github/workflows/build.yml

.github/workflows/checks.yml

sheurich · 2024-02-28T23:44:16Z

@ldez I think all of the changes have been made and would appreciate another review. The PR description hasn't yet been updated to match the actual changes.

.github/workflows/build.yml

sheurich · 2024-02-29T16:44:12Z

This will be ready for review soon, pending a bug fix.

sheurich · 2024-02-29T23:53:03Z

I think this should be ready for review @mcpherrinm

mcpherrinm · 2024-03-01T16:19:00Z

It doesn’t have to be in this PR, but we are generally standardizing on either distroless or debian-slim as the base for docker images instead of alpine. We can make that change later, or it would be helpful if you want to make it here while we’re changing the docker files.

I’ll review more fully later today

sheurich · 2024-03-01T16:34:55Z

It doesn’t have to be in this PR, but we are generally standardizing on either distroless or debian-slim as the base for docker images instead of alpine.

I had assumed that Alpine was chosen over Debian as a derivative of the strategy outlined in the Pebble README:

In places where the ACME specification allows customization/CA choice Pebble aims to make choices different from Boulder.

It's no problem to change to Distroless for release. I will add that update to this PR.

sheurich · 2024-03-01T18:40:09Z

Since cgo is not needed by pebble or pebble-challtestsrv, the release image can be FROM scratch. This WFM but are there other requirements which need glibc or other items from distroless?

mcpherrinm · 2024-03-01T18:44:41Z

Scratch is fine by me

sheurich · 2024-03-02T14:58:37Z

@mcpherrinm the scratch change is implemented and a few other bugs have been squashed.

…and docker.sh

fix: run release workflow on release publish fix: run eggsampler/acme tests with path needed for TLS certs fix: start pebble-challtestsrv fix: use separate jobs for each test type

mcpherrinm · 2024-03-12T22:48:10Z

Merged and tagged as v2.5.0

…ose.yaml (#465) #444 removed exposing the API and management ports directly in the `Dockerfile` in favor of the `docker-compose.yml`. Not everyone will be using `docker compose` so let's document how it should be done. Fixes #452

sheurich changed the title ~~Ci overhaul~~ CI Overhaul Feb 26, 2024

ldez reviewed Feb 26, 2024

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

sheurich force-pushed the ci-overhaul branch 2 times, most recently from 60b576e to 5812b5f Compare February 27, 2024 16:26

ldez reviewed Feb 27, 2024

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

ldez reviewed Feb 27, 2024

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

sheurich force-pushed the ci-overhaul branch from 0aaad56 to f3d4797 Compare February 27, 2024 20:41

ldez reviewed Feb 27, 2024

View reviewed changes

.github/workflows/build.yml Outdated Show resolved Hide resolved

.github/workflows/checks.yml Outdated Show resolved Hide resolved

.github/workflows/checks.yml Show resolved Hide resolved

mcpherrinm mentioned this pull request Feb 28, 2024

Request for a new release #443

Closed

wgreenberg mentioned this pull request Feb 28, 2024

Look into replacing Boulder tests w/ Pebble tests (or removing it entirely) certbot/certbot#9898

Closed

sheurich force-pushed the ci-overhaul branch 3 times, most recently from 2605a05 to 6f2f2c5 Compare February 28, 2024 23:16

ldez reviewed Feb 28, 2024

View reviewed changes

.github/workflows/build.yml Outdated Show resolved Hide resolved

sheurich force-pushed the ci-overhaul branch 2 times, most recently from 0f8556c to 8de4178 Compare February 29, 2024 16:21

sheurich changed the title ~~CI Overhaul~~ CI: Travis -> GitHub Actions; Create Release Binaries and Container Images Mar 2, 2024

sheurich mentioned this pull request Mar 3, 2024

Add "dns-account-01" support from draft-ietf-acme-scoped-dns-challenges #435

Merged

sheurich added 20 commits March 8, 2024 19:09

add local build script

359ddb1

add local docker build/deploy/test script

cb9faa0

add pebble -version flag; smoketest in release; add local build.sh …

d9f9aaf

…and docker.sh

fix: set version ldflags from github.sha in release workflow

cd03dec

release fixes

c6b572a

use ghcr.io registry in compose

d157204

fix base docker tag

8154447

fix: add test configuration files to docker image

60c58d4

cleanup unused dockerfiles

8fc5cbf

add eggsampler/acme-based integration test

68307f6

cleanup docker.sh GOCACHE

2bf1264

simplify local docker testing

57e58e0

use scratch for linux container base image

2af6cb1

update release version string

556b878

fix: remove binary from compose command

3008fdd

fix: run release workflow on release publish fix: run eggsampler/acme tests with path needed for TLS certs fix: start pebble-challtestsrv fix: use separate jobs for each test type

display pebble version in container for release

5d4a227

display pebble version in container for release

7f95d56

create a release and upload release assets

86aaee7

fix: remove excess permissions for release workflow

8ead44c

remove unused shell scripts

09d6943

sheurich force-pushed the ci-overhaul branch from be63839 to 09d6943 Compare March 9, 2024 00:10

aarongable approved these changes Mar 12, 2024

View reviewed changes

aarongable requested review from a team and pgporada and removed request for a team March 12, 2024 22:23

mcpherrinm approved these changes Mar 12, 2024

View reviewed changes

mcpherrinm merged commit 235473c into letsencrypt:main Mar 12, 2024
14 checks passed

pgporada mentioned this pull request May 24, 2024

Document exposing API and management ports when not using docker-compose.yaml #465

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

sheurich commented Feb 26, 2024 •

edited

Loading

ldez commented Feb 26, 2024 •

edited

Loading

sheurich commented Feb 26, 2024

ldez commented Feb 26, 2024 •

edited

Loading

sheurich commented Feb 26, 2024

sheurich commented Feb 28, 2024

sheurich commented Feb 29, 2024

sheurich commented Feb 29, 2024

mcpherrinm commented Mar 1, 2024

sheurich commented Mar 1, 2024 •

edited

Loading

sheurich commented Mar 1, 2024

mcpherrinm commented Mar 1, 2024

sheurich commented Mar 2, 2024

mcpherrinm commented Mar 12, 2024

CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

Conversation

sheurich commented Feb 26, 2024 • edited Loading

ldez commented Feb 26, 2024 • edited Loading

sheurich commented Feb 26, 2024

ldez commented Feb 26, 2024 • edited Loading

sheurich commented Feb 26, 2024

sheurich commented Feb 28, 2024

sheurich commented Feb 29, 2024

sheurich commented Feb 29, 2024

mcpherrinm commented Mar 1, 2024

sheurich commented Mar 1, 2024 • edited Loading

sheurich commented Mar 1, 2024

mcpherrinm commented Mar 1, 2024

sheurich commented Mar 2, 2024

mcpherrinm commented Mar 12, 2024

sheurich commented Feb 26, 2024 •

edited

Loading

ldez commented Feb 26, 2024 •

edited

Loading

ldez commented Feb 26, 2024 •

edited

Loading

sheurich commented Mar 1, 2024 •

edited

Loading