Skip to content

Commit

Permalink
Get the build log back in play and auto inject the GITHUB_TOKEN as a …
Browse files Browse the repository at this point in the history
…kubernetes secret, this is safe as long as we use private clusters when the yaml is deployed to the CI system
  • Loading branch information
karlmutch committed Feb 6, 2019
1 parent 078b3aa commit 41c10ac
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 41 deletions.
1 change: 0 additions & 1 deletion Dockerfile_standalone
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ RUN mkdir $GOPATH/bin && \
git config --global url."https://".insteadOf git:// && \
go get github.com/karlmutch/enumer && \
go get github.com/karlmutch/petname && \
go get github.com/karlmutch/petname && \
go install github.com/karlmutch/petname/cmd/petname && \
wget -q -O $GOPATH/bin/semver https://github.com/karlmutch/duat/releases/download/0.9.3/semver-linux-amd64 && \
wget -q -O $GOPATH/bin/stencil https://github.com/karlmutch/duat/releases/download/0.9.3/stencil-linux-amd64 && \
Expand Down
6 changes: 5 additions & 1 deletion ci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,17 @@ working_file=$$.studio-go-runner-working
rm -f $working_file
trap Tidyup 1 2 3 15

export GIT_BRANCH=`echo '{{.duat.gitBranch}}' | stencil - | tr '_' '-' | tr '\/' '-'`
export RUNNER_BUILD_LOG=build-$GIT_BRANCH.log

exit_code=0

# Determine if we are running under a keel based CI build and if so ...
export

travis_fold start "build.image"
travis_time_start
set -o pipefail ; (go run build.go -r -dirs=internal && go run build.go -r -dirs=cmd && echo "Success" || echo "Failure") 2>&1
set -o pipefail ; (go run build.go -r -dirs=internal && go run build.go -r -dirs=cmd && echo "Success" || echo "Failure") 2>&1 | tee $RUNNER_BUILD_LOG
exit_code=$?
if [ $exit_code -ne 0 ]; then
exit $exit_code
Expand All @@ -108,6 +111,7 @@ echo "Starting the namespace injections etc" $K8S_POD_NAME
kubectl label deployment build keel.sh/policy=force --namespace=$K8S_NAMESPACE
kubectl scale --namespace $K8S_NAMESPACE --replicas=0 rc/rabbitmq-controller
kubectl scale --namespace $K8S_NAMESPACE --replicas=0 deployment/minio-deployment

for (( ; ; ))
do
sleep 600
Expand Down
2 changes: 1 addition & 1 deletion ci_keel.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ metadata:
namespace: {{ .Namespace }}
type: Opaque
data:
github_token: ''
github_token: '{{ expandenv "$GITHUB_TOKEN" | b64enc }}'
---
apiVersion: v1
kind: Service
Expand Down
40 changes: 2 additions & 38 deletions docs/quay_io_ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,42 +66,6 @@ stencil -input ci_keel.yaml -values Namespace=ci-go-runner | kubectl apply -f -

This will deploy a stack capable of builds and testing. As a build finishes the stack will scale down the dependencies it uses for queuing and storage and will keep the build container alive so that logs can be examined. The build activities will disable container upgrades while the build is running and will then open for upgrades once the build steps have completed to prevent premature termination. When the build, and test has completed and pushed commits have been seen for the code base then the pod will be shutdown for the latest build and a new pod created.

When deploying an integration stack it is possible to specify a GithubToken for performing releases. If the token is present as a Kubernetes secret then upon successful build and test cycles the running container will attempt to create and deploy a release using the github release pages.
If the env variable GITHUB_TOKEN is present when deploying an integration stack it will be placed as a Kubernetes secret into the integration stack. If the secret is present then upon successful build and test cycles the running container will attempt to create and deploy a release using the github release pages.

OPtional GITHUB_TOKEN secrets are added to the cluster

Annotations updated via stencil with gitHash etc and also with desired regular expression or keel semver policy
namespace is generated and used for the bootstrapped build
stencil -input ci_keel.yaml | kubectl apply -f -
git commit and push to start things rolling
Keel repo polling triggers build

built container in build pod removes itself from keel using Kubernetes preStartHook by renaming annotations
```
Using downward API
metadata.annotations['myannotation']
```

build pod starts
new namespace generated for next listener
```
github.com/docker/docker/pkg/namesgenerator
Loop creating namespace with uuid annotation and then validating we owned it
```

container used the included ci_keel and injects the annotations from itself to create the next listening deployment
```
stencil with variables in a file for all annotations now renamed for their real keys
```

new namspace with deployment using ci_keel.yaml is dispatched
build starts in our now liberated namespace

build finishes
set ReplicationControllers and deployment .spec.replicas to 0
```
kubectl scale --namespace build-test-k8s-local --replicas=0 deployment/minio-deployment
kubectl scale --namespace build-test-k8s-local --replicas=0 rc/rabbitmq-controller
```

and the build then sits until such time as we decide on a policy for self destruction like push results back to github, at which point we dispose of the unique namespace used for the build
When the build completes the pods that are present that are only useful during the actual build and test steps will be scaled back to 0 instances.

0 comments on commit 41c10ac

Please sign in to comment.