Skip to content

Commit

Permalink
feat: use k8s init and setup jobs and remove cockroach chart dependen…
Browse files Browse the repository at this point in the history
…cy (#48)

* feat: use pre-update hook for setup

* improve timeouts

* remove init job

* remove support for k8s v1.16 and v1.17

* enable extraContainers for setup job

* make sa a hook

* remove pre-upgrade from sa

* remove helm dependency cockroach

* describe upgrade path

* describe crdb chart 8.1.8

* catch up with tests

* add crdb repo to tests

* omit cleanup on github

* retry accessibility checks

* enable custom job annotations

* enable custom pod annotations for jobs

* enable configuring job container lifecyles

* remove job container lifecycle

* Update README.md

* Update README.md

* describe uninstalling the chart

* Update README.md

* default to zitadel v2.15.0

* await eventual consistency
  • Loading branch information
eliobischof authored Dec 14, 2022
1 parent 8c3c89e commit 28ba175
Show file tree
Hide file tree
Showing 19 changed files with 553 additions and 97 deletions.
2 changes: 0 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ jobs:
# digests should be appended in image versions according to the KIND release notes
# https://github.com/kubernetes-sigs/kind/releases/tag/v0.14.0
kindest-image-tag:
- 'v1.16.15'
- 'v1.17.17'
- 'v1.18.20@sha256:738cdc23ed4be6cc0b7ea277a2ebcc454c8373d7d8fb991a7fcdbd126188e6d7'
- 'v1.19.16@sha256:a146f9819fece706b337d34125bbd5cb8ae4d25558427bf2fa3ee8ad231236f2'
- 'v1.20.15@sha256:45d0194a8069c46483a0e509088ab9249302af561ebee76a1281a1f08ecb4ed3'
Expand Down
53 changes: 50 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,61 @@ Learn more about ZITADEL by checking out the [source repository on GitHub](https

## What's in the Chart

By default, this chart installs:
- A highly available ZITADEL deployment.
- A highly available and secure CockroachDB statefulset.
By default, this chart installs a highly available ZITADEL deployment.

## Install the Chart

Follow the [guide for deploying ZITADEL on Kubernetes](https://docs.zitadel.com/docs/guides/deploy/kubernetes).

## Upgrading from v3

:::Note
Apart from breaking changes in this chart, v4 also update the
default ZITADEL version to v2.14.2. For upgrading
ZITADEL, please refer to the
[ZITADEL release notes](https://github.com/zitadel/zitadel/releases/tag/v2.14.0).

This section is only relevant for existing releases with the
values property cockroachdb.enabled not set to false.

In v4, the cockroachdb chart dependency is removed.
We decided to go this way because:
- Maintaining two separate releases is easier, especially in production.
- We can use Helm hooks specific to ZITADEL.
- ZITADEL doesn't only support CockroachDB.

If you have cockroachdb.enabled=true in your values.yaml,
you need to make sure, that the cockroachdb chart is not
managed by the zitadel release anymore. The following
example for doing so uninstalls your entire zitadel
release, reinstalls cockroach using a dedicated release,
and then installs the new zitadel chart version.
The new cockroach release will take over the PersistentVolumeClaims
from the uninstalled chart, so no data migration is needed.
Nevertheless, we highly recommend making and testing a backup before upgrading.
Also note, that you will have downtime when
following the example while zitadel is uninstalled.

```bash
helm repo add cockroachdb https://charts.cockroachdb.com/
helm repo update cockroachdb zitadel
helm uninstall my-zitadel
helm install crdb cockroachdb/cockroachdb --version 8.1.8 --set fullnameOverride=crdb
helm install my-zitadel zitadel/zitadel --values ./my-zitadel-values.yaml
```

## Uninstalling the Chart

The ZITADEL chart uses Helm hooks,
[which are not garbage collected by helm uninstall, yet](https://helm.sh/docs/topics/charts_hooks/#hook-resources-are-not-managed-with-corresponding-releases).
Therefore, to also remove hooks installed by the ZITADEL Helm chart,
delete them manually:

```bash
helm uninstall my-zitadel
kubectl delete job --selector app.kubernetes.io/name=zitadel,app.kubernetes.io/managed-by=Helm
```

## Contributors

<a href="https://github.com/zitadel/zitadel-charts/graphs/contributors">
Expand Down
6 changes: 0 additions & 6 deletions charts/zitadel/Chart.lock

This file was deleted.

17 changes: 6 additions & 11 deletions charts/zitadel/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,11 @@ apiVersion: v2
name: zitadel
description: A Helm chart for ZITADEL v2
type: application
appVersion: "v2.13.1"
version: 3.5.0
kubeVersion: '>= 1.16.15-0'
appVersion: "v2.15.0"
version: 4.0.0
kubeVersion: ">= 1.18.20-0"
icon: https://zitadel.zitadel.cloud/ui/login/resources/themes/zitadel/logo-dark.svg
dependencies:
- name: cockroachdb
version: ~8.1.0
repository: https://charts.cockroachdb.com/
condition: cockroachdb.enabled
maintainers:
- name: zitadel
email: [email protected]
url: https://zitadel.com
- name: zitadel
email: [email protected]
url: https://zitadel.com
4 changes: 4 additions & 0 deletions charts/zitadel/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ apiVersion: v1
kind: ConfigMap
metadata:
name: zitadel-config-yaml
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
data:
zitadel-config-yaml: |-
{{ .Values.zitadel.configmapConfig | toYaml | nindent 4 }}
6 changes: 3 additions & 3 deletions charts/zitadel/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ metadata:
name: {{ include "zitadel.fullname" . }}
labels:
{{- include "zitadel.labels" . | nindent 4 }}
app.kubernetes.io/component: start
spec:
replicas: {{ .Values.replicaCount }}
selector:
Expand All @@ -19,6 +20,7 @@ spec:
checksum/secret-db-ssl-root-crt: {{ include (print $.Template.BasePath "/secret_db-ssl-root-crt.yaml") . | sha256sum }}
checksum/secret-zitadel-secrets: {{ include (print $.Template.BasePath "/secret_zitadel-secrets.yaml") . | sha256sum }}
labels:
app.kubernetes.io/component: start
{{- include "zitadel.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
Expand All @@ -36,7 +38,7 @@ spec:
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
args:
- start-from-init
- start
- --config
- /config/zitadel-config-yaml
{{- if .Values.zitadel.secretConfig }}
Expand All @@ -47,8 +49,6 @@ spec:
- --config
- /.secrets/config-yaml
{{- end }}
- --steps
- /config/zitadel-config-yaml
- --masterkeyFile
- /.secrets/masterkey
env:
Expand Down
166 changes: 166 additions & 0 deletions charts/zitadel/templates/initjob.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
{{- if .Values.initJob.enabled }}
apiVersion: batch/v1
kind: Job
metadata:
name: "{{ include "zitadel.fullname" . }}-init"
labels:
{{- include "zitadel.labels" . | nindent 4 }}
app.kubernetes.io/component: init
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "1"
spec:
backoffLimit: 5
activeDeadlineSeconds: {{ .Values.initJob.activeDeadlineSeconds }}
template:
metadata:
labels:
{{- include "zitadel.selectorLabels" . | nindent 8 }}
app.kubernetes.io/component: init
{{- with .Values.initJob.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "zitadel.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
enableServiceLinks: false
restartPolicy: OnFailure
containers:
- name: "{{ .Chart.Name }}-init"
securityContext:
{{- toYaml .Values.securityContext | nindent 14 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
args:
- init
- --config
- /config/zitadel-config-yaml
{{- if .Values.zitadel.secretConfig }}
- --config
- /.secrets/zitadel-secrets-yaml
{{- end }}
{{- if .Values.zitadel.configSecretName }}
- --config
- /.secrets/config-yaml
{{- end }}
env:
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
{{- if (or .Values.zitadel.dbSslRootCrt .Values.zitadel.dbSslRootCrtSecret) }}
- name: ZITADEL_DATABASE_COCKROACH_USER_SSL_ROOTCERT
value: /.secrets/ca.crt
- name: ZITADEL_DATABASE_COCKROACH_ADMIN_SSL_ROOTCERT
value: /.secrets/ca.crt
{{- end}}
{{- if .Values.zitadel.dbSslClientCrtSecret }}
- name: ZITADEL_DATABASE_COCKROACH_ADMIN_SSL_CERT
value: /.secrets/tls.crt
- name: ZITADEL_DATABASE_COCKROACH_ADMIN_SSL_KEY
value: /.secrets/tls.key
{{- end}}
{{- with .Values.env }}
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
- name: zitadel-config-yaml
mountPath: /config
- name: chowned-secrets
mountPath: /.secrets
resources:
{{- toYaml .Values.initJob.resources | nindent 14 }}
{{- if .Values.initJob.extraContainers }}
{{- toYaml .Values.initJob.extraContainers | nindent 8 }}
{{- end }}
initContainers:
- args:
- "{{ include "zitadel.joincpcommands" (dict "commands" (list
(include "zitadel.makecpcommand" (dict "value" .Values.zitadel.secretConfig "path" "/zitadel-secrets-yaml/*" ))
(include "zitadel.makecpcommand" (dict "value" (or .Values.zitadel.dbSslRootCrt .Values.zitadel.dbSslRootCrtSecret) "path" "/db-ssl-root-crt/*" ))
(include "zitadel.makecpcommand" (dict "value" .Values.zitadel.dbSslClientCrtSecret "path" "/db-ssl-client-crt/*" ))
(include "zitadel.makecpcommand" (dict "value" .Values.zitadel.configSecretName "path" "/zitadel-config-yaml/*" ))
)) }} chown -R 1000:1000 /chowned-secrets/* && chmod 400 /chowned-secrets/*"
command:
- sh
- -c
image: "{{ .Values.chownImage.repository }}:{{ .Values.chownImage.tag }}"
imagePullPolicy: {{ .Values.chownImage.pullPolicy }}
name: chown
volumeMounts:
- name: chowned-secrets
mountPath: /chowned-secrets
{{- if .Values.zitadel.secretConfig }}
- name: zitadel-secrets-yaml
mountPath: /zitadel-secrets-yaml
{{- end }}
{{- if .Values.zitadel.configSecretName }}
- name: zitadel-secret-config-yaml
mountPath: /zitadel-config-yaml
{{- end }}
{{- if (or .Values.zitadel.dbSslRootCrt .Values.zitadel.dbSslRootCrtSecret) }}
- name: db-ssl-root-crt
mountPath: /db-ssl-root-crt
{{- end }}
{{- if .Values.zitadel.dbSslClientCrtSecret }}
- name: db-ssl-client-crt
mountPath: /db-ssl-client-crt
{{- end }}
securityContext:
runAsNonRoot: false
runAsUser: 0
volumes:
- name: zitadel-config-yaml
configMap:
name: zitadel-config-yaml
{{- if .Values.zitadel.secretConfig }}
- name: zitadel-secrets-yaml
secret:
secretName: zitadel-secrets-yaml
{{- end }}
{{- if .Values.zitadel.configSecretName }}
- name: zitadel-secret-config-yaml
secret:
secretName: {{ .Values.zitadel.configSecretName }}
{{- end }}
{{- if .Values.zitadel.dbSslRootCrt }}
- name: db-ssl-root-crt
secret:
secretName: db-ssl-root-crt
{{- end }}
{{- if .Values.zitadel.dbSslRootCrtSecret }}
- name: db-ssl-root-crt
secret:
secretName: {{ .Values.zitadel.dbSslRootCrtSecret }}
{{- end }}
{{- if .Values.zitadel.dbSslClientCrtSecret }}
- name: db-ssl-client-crt
secret:
secretName: {{ .Values.zitadel.dbSslClientCrtSecret }}
{{- end }}
{{- if (or (and .Values.zitadel.masterkey .Values.zitadel.masterkeySecretName) (and (not .Values.zitadel.masterkey) (not .Values.zitadel.masterkeySecretName)) ) }}
{{- fail "Eighter set .Values.zitadel.masterkey or .Values.zitadel.masterkeySecretName exclusively" }}
{{- end }}
- name: chowned-secrets
emptyDir: {}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{ end }}
4 changes: 4 additions & 0 deletions charts/zitadel/templates/secret_db-ssl-root-crt.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ kind: Secret
type: Opaque
metadata:
name: db-ssl-root-crt
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
stringData:
ca.crt: |-
{{ .Values.zitadel.dbSslRootCrt | default "" | nindent 6 }}
Expand Down
4 changes: 4 additions & 0 deletions charts/zitadel/templates/secret_zitadel-masterkey.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ kind: Secret
type: Opaque
metadata:
name: zitadel-masterkey
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
stringData:
masterkey: {{ .Values.zitadel.masterkey }}
{{- end -}}
7 changes: 4 additions & 3 deletions charts/zitadel/templates/secret_zitadel-secrets.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
{{- if (and .Values.cockroachdb.enabled .Values.cockroachdb.tls.enabled (not ((((.Values.zitadel.secretConfig).Database).cockroach).User).Password)) }}
{{- fail ".Values.zitadel.secretConfig.Database.cockroach.User.Password is mandatory for tls enabled cockroach" }}
{{- end }}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: zitadel-secrets-yaml
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
stringData:
zitadel-secrets-yaml: |-
{{ .Values.zitadel.secretConfig | toYaml | nindent 4 }}
9 changes: 6 additions & 3 deletions charts/zitadel/templates/serviceaccount.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,11 @@ metadata:
name: {{ include "zitadel.serviceAccountName" . }}
labels:
{{- include "zitadel.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
{{- with .Values.serviceAccount.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
Loading

0 comments on commit 28ba175

Please sign in to comment.