forked from pangeo-data/pangeo-cloud-federation
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add Skaffold configuration and readme (pangeo-data#207)
* Add Skaffold configuration and readme * Move skaffold-skaffolding to skaffold subfolder Co-authored-by: Alex <[email protected]> [ci skip]
- Loading branch information
Showing
4 changed files
with
129 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
apiVersion: skaffold/v2alpha3 | ||
kind: Config | ||
|
||
build: | ||
local: | ||
push: false | ||
useBuildkit: true | ||
tagPolicy: | ||
envTemplate: | ||
template: "dask-gateway-server:latest" | ||
artifacts: | ||
- image: dask-gateway-server | ||
docker: | ||
dockerfile: skaffold/Dockerfile | ||
|
||
deploy: | ||
helm: | ||
releases: | ||
- name: dask-gateway | ||
chartPath: resources/helm/dask-gateway/ | ||
namespace: default | ||
valuesFiles: | ||
- resources/helm/dask-gateway/values.yaml | ||
setValues: | ||
gateway.image.name: dask-gateway-server | ||
gateway.image.pullPolicy: IfNotPresent | ||
gateway.cookieSecret: <key> | ||
gateway.proxyToken: <key> | ||
local: true | ||
recreatePods: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
FROM daskgateway/dask-gateway-server:latest | ||
|
||
COPY --chown=dask:dask skaffold/condarc /opt/conda/condarc | ||
|
||
RUN conda install -c conda-forge \ | ||
go=1.12.5 \ | ||
cryptography \ | ||
# Leaving tornado for backwards compatibility. | ||
tornado \ | ||
traitlets \ | ||
sqlalchemy \ | ||
python-kubernetes \ | ||
aiohttp \ | ||
colorlog \ | ||
-y | ||
|
||
COPY --chown=dask:dask . /dask | ||
WORKDIR /dask/dask-gateway-server/ | ||
# Perform an editable install to avoid packaging a wheel. | ||
RUN pip install -e . | ||
# TODO: Add kubernetes extras once the kubernetes backend is ready. | ||
# RUN pip install -e .[kubernetes] | ||
WORKDIR /srv/dask-gateway | ||
USER root | ||
RUN chown dask:dask /srv/dask-gateway | ||
USER 1000:1000 | ||
|
||
CMD ["dask-gateway-server", "--config", "/etc/dask-gateway/dask_gateway_config.py"]`` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Skaffold | ||
|
||
[Skaffold](https://skaffold.dev) is a CLI developed by Google that streamlines | ||
the local Kubernetes development cycle. We're going to use it to streamline | ||
development of Dask Gateway Server. | ||
|
||
Note, this workflow is compatible with release version 0.6.1. The current | ||
gateway server build succeeds but the container fails because the Kubernetes | ||
backend has not been implemented yet. Nevertheless, the Skaffold development | ||
cycle can be leveraged as work on the backend proceeds. | ||
|
||
## Installation | ||
|
||
Install [Skaffold](https://skaffold.dev/docs/install/). | ||
```bash | ||
brew install skaffold | ||
``` | ||
|
||
[Activate](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#activating-an-environment) | ||
your conda environment. | ||
|
||
Enable [Docker | ||
Buildkit](https://docs.docker.com/develop/develop-images/build_enhancements/). | ||
```bash | ||
export DOCKER_BUILDKIT=1 | ||
``` | ||
|
||
## Configuration | ||
|
||
Generate a randomized key for the `cookieSecret` and `proxyToken` in | ||
[skaffold.yaml](skaffold.yaml). | ||
```bash | ||
sed -i "s|<key>|$(openssl rand -hex 32)|" skaffold.yaml | ||
``` | ||
|
||
## Usage | ||
|
||
Run Skaffold in [dev mode](https://skaffold.dev/docs/). This ensures that any | ||
modifications to the source result in a rebuild and redeployment (via Helm) of | ||
the source code. The `--no-prune=false` and `--cache-artifacts=false` automate | ||
the image [clean-up](https://skaffold.dev/docs/pipeline-stages/cleanup/) | ||
process. | ||
```bash | ||
skaffold dev --no-prune=false --cache-artifacts=false | ||
``` | ||
|
||
You may want to create an alias for the above command. | ||
```bash | ||
# bash | ||
echo 'alias skd="skaffold dev --no-prune=false --cache-artifacts=false"' >> ~/.bash_profile | ||
# zsh | ||
echo 'alias skd="skaffold dev --no-prune=false --cache-artifacts=false"' >> ~/.zshrc | ||
``` | ||
|
||
Update one of the source files in `skaffold/dask-gateway/dask-gateway-server` | ||
and rejoice. | ||
|
||
The very first build will take about 90 seconds. After that, the Docker build | ||
process won't need to re-run `conda install`, so subsequent builds should take | ||
around 5.4s! | ||
|
||
If you're using an internal artifact store, open [condarc](condarc) and follow | ||
the instructions on the first two lines. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
# Note: If you're using an internal artifact store, uncomment lines 3 and 4 | ||
# and populate the channel_alias field with a link to the store. | ||
# channel_alias: | ||
# allow_other_channels: false | ||
|
||
channels: | ||
- conda-forge | ||
- defaults |