Skip to content

Commit

Permalink
Containerized builds through GitHub Actions. (#21)
Browse files Browse the repository at this point in the history
Added containerized builds for:
  * Alpine 3.12
  * CentOS 5.11
  * CentOS 8.2 (which has OpenSSL 1.1.1c for backward compatibility with 8.0-8.2).

**Drive-by changes**:
  * Forward ported the latest improvements from `python-package`.
  * Specific support for Amazon Linux 2 and RHEL 7 was removed.
  * Trimmed down Alpine deps, to have it work on minimal containers.
  * Updated embedded OpenSSL libs to 1.1.1k.
  * Updated `cryptography` to 3.4.7.
  * Updated `pip` to latest version.
  • Loading branch information
dumol authored May 20, 2021
1 parent 2480fe3 commit 31955c7
Show file tree
Hide file tree
Showing 15 changed files with 264 additions and 136 deletions.
76 changes: 46 additions & 30 deletions .github/workflows/main.yaml → .github/workflows/bare.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,24 @@
# Don't use `-latest` for runners, pin specific OS versions instead. More at
# https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners.

name: GitHub-CI
name: Bare

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

concurrency:
group: bare-${{ github.ref }}
cancel-in-progress: true

# Set to 'yes' to open a tunnel to GitHub's VMs through ngrok/tmate on failures.
# Also increase timeout-minutes for the relevant OS when debugging remotely.
# Version 3 of mxschmitt/action-tmate should also work on Windows.
env:
CHEVAH_DEBUG: 'no'
TMATE_DEBUG: 'no'
NGROK_DEBUG: 'no'


jobs:
Expand All @@ -27,22 +33,20 @@ jobs:
fail-fast: false
matrix:
runs-on: [ windows-2019, windows-2016 ]

timeout-minutes: 20

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Make sure there are no concurrent jobs
- uses: chevah/auto-cancel-redundant-job@v1
# Checks-out the repository under $GITHUB_WORKSPACE, so the job can access it
- uses: actions/checkout@v2
with:
fetch-depth: 2

# Explicitly run our scripts with Bash, not PowerShell (GitHub's default).
- name: Build Python
- name: Build Pythia
run: bash ./pythia build

- name: Test Python
- name: Test Pythia
run: bash ./pythia test

# To use an RSA key with SFTPPlus, install upstream OpenSSH package,
Expand All @@ -58,14 +62,24 @@ jobs:
echo "${{ secrets.SFTPPLUS_BIN_HOST_KEY }}" > ~/.ssh/known_hosts
choco install --yes --no-progress openssh
# Uploads using a (per-OS selected) sftp command, then shows final links.
# Upload using a (per-OS selected) sftp command, then show final links.
# Remove key in same step to avoid leaving it on disk if publishing fails.
- name: Upload testing package
run: bash ./publish_dist.sh
run: bash -c './publish_dist.sh ; rm priv_key'

# Per https://github.com/nelsonjchen/reverse-rdp-windows-github-actions,
# but using the Ngrok token as password for the runnneradmin user.
# Command line debugging through Tmate. v3 works on Windows too.
- name: Tmate debug on failure
if: failure() && env.TMATE_DEBUG == 'yes'
uses: mxschmitt/action-tmate@v3
with:
limit-access-to-actor: true

# To access the Windows desktop for advanced debugging, as per
# https://github.com/nelsonjchen/reverse-rdp-windows-github-actions,
# but using the ngrok token as password for the runnneradmin user.
# Basically use the ngrok token and the ngrok URL (from ngrok's dashboard).
- name: Ngrok debugging on failure
if: failure() && env.CHEVAH_DEBUG == 'yes'
if: failure() && env.NGROK_DEBUG == 'yes'
env:
NGROK_TOKEN: ${{ secrets.NGROK_AUTH_TOKEN }}
run: |
Expand All @@ -86,19 +100,18 @@ jobs:
runs-on: [ ubuntu-20.04, ubuntu-18.04 ]
timeout-minutes: 40
steps:
- uses: chevah/auto-cancel-redundant-job@v1
- uses: actions/checkout@v2
with:
fetch-depth: 2

- name: Install required packages
run: sudo apt-get install libncurses5-dev

- name: Build Python
run: bash ./pythia build
- name: Build Pythia
run: ./pythia build

- name: Test Python
run: bash ./pythia test
- name: Test Pythia
run: ./pythia test

- name: Upload testing package
run: |
Expand All @@ -107,19 +120,20 @@ jobs:
chmod 600 priv_key
echo "${{ secrets.SFTPPLUS_BIN_PRIV_KEY }}" > priv_key
echo "${{ secrets.SFTPPLUS_BIN_HOST_KEY }}" > ~/.ssh/known_hosts
bash ./publish_dist.sh
./publish_dist.sh ; rm priv_key
# If one of the above steps fails, fire up tmate for remote debugging.
- name: Tmate debug on failure
if: failure() && env.CHEVAH_DEBUG == 'yes'
uses: mxschmitt/action-tmate@v2

if: failure() && env.TMATE_DEBUG == 'yes'
uses: mxschmitt/action-tmate@v3
with:
limit-access-to-actor: true

macos:
runs-on: macos-10.15
needs: [ windows, linux ]
timeout-minutes: 60
steps:
- uses: chevah/auto-cancel-redundant-job@v1
- uses: actions/checkout@v2
with:
fetch-depth: 2
Expand All @@ -132,11 +146,11 @@ jobs:
sudo chmod -v a-r /usr/local/include/libintl.h
sudo chmod -v a-r /usr/local/opt/gettext/lib/libintl.*
- name: Build Python
run: bash ./pythia build
- name: Build Pythia
run: ./pythia build

- name: Test Python
run: bash ./pythia test
- name: Test Pythia
run: ./pythia test

- name: Upload testing package
run: |
Expand All @@ -145,16 +159,18 @@ jobs:
chmod 600 priv_key
echo "${{ secrets.SFTPPLUS_BIN_PRIV_KEY }}" > priv_key
echo "${{ secrets.SFTPPLUS_BIN_HOST_KEY }}" > ~/.ssh/known_hosts
bash ./publish_dist.sh
./publish_dist.sh ; rm priv_key
# Fix back Homebrew, to make everything functional for tmate debugging.
- name: Unhack Homebrew
if: failure() && env.CHEVAH_DEBUG == 'yes'
if: failure() && env.TMATE_DEBUG == 'yes'
run: |
sudo chmod -v a+r /usr/local/opt/gettext/lib/libintl*
sudo chmod -v a+r /usr/local/include/libintl.h
sudo mv -v /usr/local/bin/git{.saved,}
- name: Tmate debug on failure
if: failure() && env.CHEVAH_DEBUG == 'yes'
uses: mxschmitt/action-tmate@v2
if: failure() && env.TMATE_DEBUG == 'yes'
uses: mxschmitt/action-tmate@v3
with:
limit-access-to-actor: true
108 changes: 108 additions & 0 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#
# GitHub actions for building and testing.
#
# For best support, use `-latest` for runners spinning up containers. More at
# https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners.

name: Docker

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

concurrency:
group: docker-${{ github.ref }}
cancel-in-progress: true

# Set to 'yes' to open a tunnel to GitHub's VMs through tmate on failures.
# Also increase timeout-minutes for the relevant OS when debugging remotely.
env:
TMATE_DEBUG: 'no'

# Using a job name that doesn't contain the OS name, to minimize the risk of
# confusion with the OS names of the containers, which are the relevant ones.
jobs:
latest:
runs-on: ubuntu-latest
container: ${{ matrix.container }}
strategy:
fail-fast: false
matrix:
container: [ 'alpine:3.12', 'centos:8.2.2004', 'centos:5.11' ]
timeout-minutes: 30
steps:

# OpenSSL gets updated by apk, but that is the Alpine way, so it's fine.
- name: Alpine 3.12 setup
if: matrix.container == 'alpine:3.12'
run: |
apk update
apk upgrade
apk add git curl bash openssh-client
curl -o /usr/local/bin/paxctl https://binary.chevah.com/third-party-stuff/alpine/paxctl-3.12
chmod +x /usr/local/bin/paxctl
# Stick to CentOS 8.2 as OpenSSL got updated in 8.3 from 1.1.1c to 1.1.1g.
- name: CentOS 8.2 setup
if: matrix.container == 'centos:8.2.2004'
run: |
sed -i s/^mirrorlist=/#mirrorlist=/ /etc/yum.repos.d/*.repo
sed -i s@^#baseurl=http://mirror.centos.org/\$contentdir/\$releasever/@baseurl=https://vault.centos.org/8.2.2004/@ /etc/yum.repos.d/*.repo
yum -y upgrade
yum -y install git curl openssh-clients
# Final CentOS 5 version is used to build the generic Linux package.
- name: CentOS 5.11 setup
if: matrix.container == 'centos:5.11'
run: |
sed -i s/^mirrorlist=/#mirrorlist=/ /etc/yum.repos.d/*.repo
sed -i s@^#baseurl=http://mirror.centos.org/centos/\$releasever/@baseurl=http://vault.centos.org/5.11/@ /etc/yum.repos.d/*.repo
yum -y upgrade
# Use http://binary.chevah.com/third-party-stuff/centos5/tuxad/
# when tuxad.de dissapears, it has the minimum required stuff.
rpm -i http://www.tuxad.de/rpms/tuxad-release-5-1.noarch.rpm
yum -y install curl openssh-clients gcc44 make m4 patch unzip wget
ln -s /usr/bin/gcc44 /usr/local/bin/gcc
wget --mirror --no-parent https://binary.chevah.com/third-party-stuff/centos5/endpoint/
cd binary.chevah.com/third-party-stuff/centos5/endpoint/
rpm -i local-perl-*.rpm
rpm -i --nodeps git{-core,}-2.5.0-1.ep.x86_64.rpm
- name: Clone repo independently
run: |
git clone https://github.com/chevah/pythia.git
cd pythia
git checkout ${GITHUB_HEAD_REF}
- name: Build Pythia
run: |
cd pythia
./pythia build
- name: Test Pythia
run: |
cd pythia
./pythia test
# Using `~/` is problematic under Docker, use `/root/`.
# Remove key in same step to avoid leaving it on disk if publishing fails.
- name: Upload testing package
run: |
mkdir -pv /root/.ssh/
cd pythia
touch priv_key
chmod 600 priv_key
echo "${{ secrets.SFTPPLUS_BIN_PRIV_KEY }}" > priv_key
echo "${{ secrets.SFTPPLUS_BIN_HOST_KEY }}" > /root/.ssh/known_hosts
./publish_dist.sh ; rm priv_key
# If one of the above steps fails, fire up tmate for remote debugging.
# Not working on Alpine (not supported) and CentOS 5 (glibc too old).
- name: Tmate debug on failure
if: failure() && env.TMATE_DEBUG == 'yes'
uses: mxschmitt/action-tmate@v3
with:
sudo: false
limit-access-to-actor: true
40 changes: 0 additions & 40 deletions .travis.yml

This file was deleted.

16 changes: 8 additions & 8 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Pythia - a Python Portable Package
Pythia - a portable Python package
==================================

Build system for a portable Python distribution.
Build system for a portable Python package.
A derivative of https://github.com/chevah/python-package/.

Building:
Expand All @@ -22,12 +22,15 @@ Supported platforms
-------------------

* Windows Server 2012 R2 and newer (x86 and x64)
* Red Hat Linux Enterprise 7 and 8 (including derivatives such as CentOS)
* macOS 10.13 and newer.
* all glibc-based Linux distributions (glibc 2.5+ for x64, 2.23+ for arm64)

Platforms on which the system OpenSSL is used:

* Red Hat Linux Enterprise 8 and newer (including derivatives such as CentOS)
* Amazon Linux 2
* Ubuntu Server 18.04 and 20.04
* all glibc-based Linux distributions (glibc 2.5+ for x64, 2.23+ for arm64)
* Alpine Linux 3.12
* macOS 10.13 and newer.

Platforms that should work, but are not regularly tested:

Expand Down Expand Up @@ -70,8 +73,5 @@ right, then save the diff to the sources patched with the preceding patch.
.. image:: https://github.com/chevah/pythia/workflows/GitHub-CI/badge.svg
:target: https://github.com/chevah/pythia/actions

.. image:: https://travis-ci.com/chevah/pythia.svg?branch=main
:target: https://travis-ci.com/github/chevah/pythia

.. image:: https://img.shields.io/github/issues/chevah/pythia.svg
:target: https://github.com/chevah/pythia/issues
Loading

0 comments on commit 31955c7

Please sign in to comment.