Skip to content

Commit

Permalink
Add Skaffold configuration and readme (pangeo-data#207)
Browse files Browse the repository at this point in the history
* Add Skaffold configuration and readme

* Move skaffold-skaffolding to skaffold subfolder

Co-authored-by: Alex <[email protected]>

[ci skip]
  • Loading branch information
gforsyth authored Feb 18, 2020
1 parent c3f6db1 commit 2f02d73
Show file tree
Hide file tree
Showing 4 changed files with 129 additions and 0 deletions.
30 changes: 30 additions & 0 deletions skaffold.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
apiVersion: skaffold/v2alpha3
kind: Config

build:
local:
push: false
useBuildkit: true
tagPolicy:
envTemplate:
template: "dask-gateway-server:latest"
artifacts:
- image: dask-gateway-server
docker:
dockerfile: skaffold/Dockerfile

deploy:
helm:
releases:
- name: dask-gateway
chartPath: resources/helm/dask-gateway/
namespace: default
valuesFiles:
- resources/helm/dask-gateway/values.yaml
setValues:
gateway.image.name: dask-gateway-server
gateway.image.pullPolicy: IfNotPresent
gateway.cookieSecret: <key>
gateway.proxyToken: <key>
local: true
recreatePods: true
28 changes: 28 additions & 0 deletions skaffold/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM daskgateway/dask-gateway-server:latest

COPY --chown=dask:dask skaffold/condarc /opt/conda/condarc

RUN conda install -c conda-forge \
go=1.12.5 \
cryptography \
# Leaving tornado for backwards compatibility.
tornado \
traitlets \
sqlalchemy \
python-kubernetes \
aiohttp \
colorlog \
-y

COPY --chown=dask:dask . /dask
WORKDIR /dask/dask-gateway-server/
# Perform an editable install to avoid packaging a wheel.
RUN pip install -e .
# TODO: Add kubernetes extras once the kubernetes backend is ready.
# RUN pip install -e .[kubernetes]
WORKDIR /srv/dask-gateway
USER root
RUN chown dask:dask /srv/dask-gateway
USER 1000:1000

CMD ["dask-gateway-server", "--config", "/etc/dask-gateway/dask_gateway_config.py"]``
63 changes: 63 additions & 0 deletions skaffold/SKAFFOLD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Skaffold

[Skaffold](https://skaffold.dev) is a CLI developed by Google that streamlines
the local Kubernetes development cycle. We're going to use it to streamline
development of Dask Gateway Server.

Note, this workflow is compatible with release version 0.6.1. The current
gateway server build succeeds but the container fails because the Kubernetes
backend has not been implemented yet. Nevertheless, the Skaffold development
cycle can be leveraged as work on the backend proceeds.

## Installation

Install [Skaffold](https://skaffold.dev/docs/install/).
```bash
brew install skaffold
```

[Activate](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#activating-an-environment)
your conda environment.

Enable [Docker
Buildkit](https://docs.docker.com/develop/develop-images/build_enhancements/).
```bash
export DOCKER_BUILDKIT=1
```

## Configuration

Generate a randomized key for the `cookieSecret` and `proxyToken` in
[skaffold.yaml](skaffold.yaml).
```bash
sed -i "s|<key>|$(openssl rand -hex 32)|" skaffold.yaml
```

## Usage

Run Skaffold in [dev mode](https://skaffold.dev/docs/). This ensures that any
modifications to the source result in a rebuild and redeployment (via Helm) of
the source code. The `--no-prune=false` and `--cache-artifacts=false` automate
the image [clean-up](https://skaffold.dev/docs/pipeline-stages/cleanup/)
process.
```bash
skaffold dev --no-prune=false --cache-artifacts=false
```

You may want to create an alias for the above command.
```bash
# bash
echo 'alias skd="skaffold dev --no-prune=false --cache-artifacts=false"' >> ~/.bash_profile
# zsh
echo 'alias skd="skaffold dev --no-prune=false --cache-artifacts=false"' >> ~/.zshrc
```

Update one of the source files in `skaffold/dask-gateway/dask-gateway-server`
and rejoice.

The very first build will take about 90 seconds. After that, the Docker build
process won't need to re-run `conda install`, so subsequent builds should take
around 5.4s!

If you're using an internal artifact store, open [condarc](condarc) and follow
the instructions on the first two lines.
8 changes: 8 additions & 0 deletions skaffold/condarc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Note: If you're using an internal artifact store, uncomment lines 3 and 4
# and populate the channel_alias field with a link to the store.
# channel_alias:
# allow_other_channels: false

channels:
- conda-forge
- defaults

0 comments on commit 2f02d73

Please sign in to comment.