Refactor helm chart, splitting operator deps into separate chart

.github/workflows/test.yaml

@@ -17,10 +17,11 @@ jobs:
       - uses: actions/checkout@v3
         with:
           persist-credentials: false
-      - name: Helm dep update
+      - name: Helm dep update for galaxy
         run: cd galaxy/ && helm dep update && cd ..
-      - name: Helm linting
+      - name: Helm linting for galaxy
         run: helm lint galaxy/
+
   test:
     runs-on: ubuntu-latest
     steps:
@@ -29,7 +30,7 @@ jobs:
       - uses: actions/checkout@v3
         with:
           persist-credentials: false
-      - name: Helm dep update
+      - name: Helm dep update for galaxy
         run: cd galaxy/ && helm dep update && cd ..
       - name: Start k8s locally
         uses: jupyterhub/action-k3s-helm@v3
@@ -47,8 +48,10 @@ jobs:
           helm list
       - name: Helm repo add galaxy
         run: helm repo add galaxy https://github.com/CloudVE/helm-charts/raw/master
+      - name: Helm install galaxy-helm-deps
+        run: time bash -c 'helm install --create-namespace --wait -n "galaxy-deps" "galaxy-deps" galaxy/galaxy-deps --set cvmfs.cvmfscsi.cache.alien.enabled=false --wait --timeout=1200s'
       - name: Helm install Galaxy
-        run: time bash -c 'helm install --create-namespace -n galaxy galaxy ./galaxy --set persistence.accessMode="ReadWriteOnce" --set resources.requests.memory=0Mi,resources.requests.cpu=0m,cvmfs.cvmfscsi.cache.alien.enabled=false --wait --timeout=1200s'
+        run: time bash -c 'helm install --create-namespace -n galaxy galaxy ./galaxy --set persistence.accessMode="ReadWriteOnce" --set resources.requests.memory=0Mi,resources.requests.cpu=0m --wait --timeout=1200s'
       - name: Get events
         run: kubectl get events -n galaxy; kubectl get events -n csi-drivers;
         if: always()

README.md

@@ -1,4 +1,4 @@
-# Galaxy Helm Chart (v5)
+# Galaxy Helm Chart (v6)
 
 [Galaxy](https://galaxyproject.org/) is a data analysis platform focusing on
 accessibility, reproducibility, and transparency of primarily bioinformatics
@@ -43,8 +43,10 @@ This chart relies on the features of other charts for common functionality:
 
 In a production setting, especially if the intention is to run multiple Galaxies
 in a single cluster, we recommend installing the dependency charts separately
-once per cluster, and installing Galaxy with `--set postgresql.deploy=false
---set s3csi.deploy=false --set cvmfs.deploy=false --set rabbitmq.deploy=false`.
+once per cluster. For convenience, we provide a `galaxy-deps` helm chart that will
+install all of these general dependencies (often installable cluster-wide) for you.
+Simply install using
+`helm install --create-namespace -n galaxy-deps galaxy-deps galaxyproject/galaxy-deps`.
 
 ---
 
@@ -61,28 +63,42 @@ helm repo add cloudve https://raw.githubusercontent.com/CloudVE/helm-charts/mast
 helm repo update
 ```
 
-2. Install the chart with the release name `my-galaxy`. It is not advisable to
+2. Install global dependencies such as the postgres operator.
+
+```console
+helm install --create-namespace -n galaxy-deps galaxy-deps galaxyproject/galaxy-deps
+```
+
+3. Install the chart with the release name `my-galaxy`. It is not advisable to
    install Galaxy in the `default` namespace.
 
 ```console
-helm install my-galaxy-release cloudve/galaxy
+helm install -n my-namespace my-galaxy-release cloudve/galaxy
 ```
 
 ### Using the chart from GitHub repo
 
-1. Clone this repository and add required dependency charts:
+1. Clone this repository:
 
 ```console
 git clone https://github.com/galaxyproject/galaxy-helm.git
-cd galaxy-helm/galaxy
+```
+
+2. Setup cluster-wide operators and dependencies:
+
+```console
+cd galaxy-helm/galaxy-deps
 helm dependency update
+helm install --create-namespace -n galaxy-deps galaxy-deps .
 ```
 
-2. To install the chart with the release name `my-galaxy`. See [Data
+3. To install the chart with the release name `my-galaxy`. See [Data
    persistence](#data-persistence) section below about the use of persistence
    flag that is suitable for your Kubernetes environment.
 
 ```console
+cd ../galaxy
+helm dependency update
 helm install --create-namespace -n galaxy my-galaxy . --set persistence.accessMode="ReadWriteOnce"
 ```
 
@@ -98,42 +114,12 @@ To uninstall/delete the `my-galaxy` deployment, run:
 helm delete my-galaxy
 ```
 
-if you see that some RabbitMQ and Postgres elements remain after some 10 minutes or more, you should be able to issue:
+If you no longer require cluster-wide operators, you can optionally uninstall them, although,
+in general, we recommend installing them once and leaving them as is.
 
+```console
+helm delete -n galaxy-deps galaxy-deps
 ```
-kubectl delete RabbitmqCluster/my-galaxy-rabbitmq-server
-kubectl delete statefulset/galaxy-my-galaxy-postgres
-```
-
-it might be needed to remove the finalizer from the RabbitmqCluster if the above doesn't seem to get rid of RabbitmqCluster, through a
-
-```
-kubectl edit RabbitmqCluster/my-galaxy-rabbitmq-server
-```
-
-remove the finalizer in:
-
-```
-apiVersion: rabbitmq.com/v1beta1
-kind: RabbitmqCluster
-metadata:
-  annotations:
-    meta.helm.sh/release-name: my-galaxy
-    meta.helm.sh/release-namespace: default
-  creationTimestamp: "2022-12-19T16:54:33Z"
-  deletionGracePeriodSeconds: 0
-  deletionTimestamp: "2022-12-19T17:41:40Z"
-  finalizers:
-  - deletion.finalizers.rabbitmqclusters.rabbitmq.com
-```
-
-and remove the postgres secret:
-
-```
-kubectl delete secrets/standby.galaxy-my-galaxy-postgres.credentials.postgresql.acid.zalan.do
-```
-
-Consider as well that if you set persistence to be enabled, Postgres and Galaxy will leave their PVCs behind, which you might want to delete or not depending on your use case.
 
 ## Configuration
 
@@ -163,11 +149,8 @@ current default values can be found in `values.yaml` file.
 | `jobs.priorityClass.existingClass`         | Use an existing [priorityClass](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass) to assign if `jobs.priorityClass.enabled=true`                                   |
 | `refdata.enabled`                          | Whether or not to mount cloud-hosted Galaxy reference data and tools.                                           |
 | `refdata.type`                             | `s3csi` or `cvmfs`, determines the CSI to use for mounting reference data. `cvmfs` is the default type for reference data. |
-| `s3csi.deploy`                             | Deploy the CSI-S3 Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource.          |
-| `cvmfs.deploy`                             | Deploy the Galaxy-CVMFS-CSI Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource                                           |
 | `cvmfs.enabled`                            | Enable use of CVMFS in configs, and deployment of CVMFS Persistent Volume Claims for Galaxy                                                                                                                |
 | `cvmfs.pvc.{}`                             | Persistent Volume Claim to deploy for CVMFS repositories. See <a href="galaxy/values.yaml">`values.yaml`</a> for examples.                                                                                                                 |
-| `cvmfs.deployPostInstallFix`                             |Deploy the fix for Galaxy-CVMFS-CSI Helm Chart. |
 | `setupJob.ttlSecondsAfterFinished`          | Sets `ttlSecondsAfterFinished` for the initialization jobs. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/#ttl-controller) for more details.       |
 | `setupJob.downloadToolConfs.enabled`        | Download configuration files and the `tools` directory from an archive via a job at startup                                                                                                                |
 | `setupJob.downloadToolConfs.archives.startup` | A URL to a `tar.gz` publicly accessible archive containing AT LEAST conf files and XML tool wrappers. Meant to be enough for Galaxy handlers to startup.                                                   |
@@ -524,3 +507,17 @@ The following fields can be specified for each file.
 See the `example` cron job included in the `values.yaml` file for a full example.
 
 
+## Upgrading
+
+## From v5 to v6
+
+* v6 splits all global dependencies such as the postgres and rabbitbq operators into a separate `galaxy-deps` chart, in contrast to v5, which
+  had all dependencies bundled in for convenience. This bundling caused problems during uninstallation in particular, because for example,
+  the postgres operator could be uninstalled before postgres itself was uninstalled, leaving various artefacts behind. This made reinstallation
+  particularly tricky, as all such left-over resources had to be cleaned up manually. Therefore, our production installation notes specified
+  installing these dependencies separately anyway. v6 makes this separation explicit by specifically debundling these dependencies into a separate
+  chart.
+
+  If upgrading in production scenarios, you may simply omit installing the `galaxy-deps` chart and continue as usual. If upgrading in development
+  scenarios, there is no straightforward upgrade path. The galaxy chart will have to be uninstalled, the `galaxy-deps` chart installed, and subsequently,
+  galaxy can be reinstalled.

VALUES.md

@@ -41,7 +41,6 @@
 | refdata.type | `s3fs` or `cvmfs`, determines the CSI to use for mounting reference data. `cvmfs` is the default and recommended for the time being. |
 | cvmfs | Configuration block if `cvmfs` is used as `refdata.type` |
 | cvmfs.deploy | Deploy the Galaxy-CVMFS-CSI Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource |
-| cvmfs.deployPostInstallFix| Deploy the fix for Galaxy-CVMFS-CSI Helm Chart. |
 | s3csi | Configuration block if `s3csi` is used as the `refdata.type` |
 | s3csi.deploy | Deploy the CSI-S3 Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource. |
 | useSecretConfigs | When this flag is set to true, all configs will be set in secrets, when it is set to false, all configs will be set in configmaps |

galaxy/Chart.yaml

@@ -5,32 +5,8 @@ version: 5.19.0
 appVersion: "24.1.1"
 description: Chart for Galaxy, an open, web-based platform for accessible, reproducible, and transparent computational biomedical research.
 icon: https://galaxyproject.org/images/galaxy-logos/galaxy_project_logo_square.png
-dependencies:
-  - name: postgres-operator
-    repository: https://raw.githubusercontent.com/zalando/postgres-operator/master/charts/postgres-operator/
-    version: 1.13.0
-    condition: postgresql.deploy
-    alias: postgresql
-    tags:
-      - deploy-postgres
-  - name: csi-s3
-    repository: https://raw.githubusercontent.com/cloudve/helm-charts/master/
-    version: 0.35.3
-    condition: s3csi.deploy
-    alias: s3csi
-    tags:
-      - deploy-s3csi
-  - name: galaxy-cvmfs-csi
-    repository: https://raw.githubusercontent.com/cloudve/helm-charts/master/
-    version: 2.5.1
-    condition: cvmfs.deploy
-    alias: cvmfs
-    tags:
-      - deploy-cvmfs
-  - name: rabbitmq-cluster-operator
-    repository: https://charts.bitnami.com/bitnami
-    version: 4.3.25
-    condition: rabbitmq.deploy
-    alias: rabbitmq
-    tags:
-      - deploy-rabbitmq
+
+# This helm chart has no direct dependencies. However, it expects external environmental
+# dependencies, such as cluster-wide operators for postgres, rabbitmq and cvmfs for example, to be preinstalled.
+# This can be done more conveniently by installing the galaxy-deps chart.
+dependencies: []

galaxy/values.yaml

@@ -392,9 +392,6 @@ affinity: {}
 
 postgresql:
   enabled: true
-  #- Whether to deploy the postgresl operator.
-  #- In general, we recommend installing the operator globally in production.
-  deploy: true
   #- hostname and port of an existing database to use.
   existingDatabase:
   galaxyDatabaseUser: galaxydbuser
@@ -429,21 +426,10 @@ refdata:
 cvmfs:
   #- Deploy the Galaxy-CVMFS-CSI Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource
   deploy: true
-  #- Deploy the CVMF post install fix,
-  deployPostInstallFix: true
-  storageClassName: "{{ $.Release.Name }}-cvmfs"
-  cvmfscsi:
-    cache:
-      alien:
-        pvc:
-          name: cvmfs-alien-cache
+  storageClassName: "galaxy-cvmfs"
 
 #- Configuration block if `s3csi` is used as the `refdata.type`
 s3csi:
-  #- Deploy the CSI-S3 Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource.
-  deploy: false
-  images:
-    csi: cloudve/csi-s3:0.31.3
   storageClass:
     name: refdata-gxy-data
     mounter: s3fs
@@ -455,8 +441,6 @@ s3csi:
     # mountOptions: "--memory-limit 4000 --dir-mode 0777 --file-mode 0666 --cache /tmp/geesecache --debug --debug_fuse --stat-cache-ttl 9m0s --cache-to-disk-hits 1"
     # --no-dir-object --no-implicit-dir --stat-cache-ttl 120m0s --max-disk-cache-fd 4096
   secret:
-    endpoint: https://s3.ap-southeast-2.amazonaws.com
-    usePrefix: true
     prefix: /galaxy/v1/data.galaxyproject.org
 
 #- When this flag is set to true, all configs will be set in secrets,
@@ -807,7 +791,6 @@ tusd:
 
 rabbitmq:
   enabled: true
-  deploy: true
   existingCluster:
   existingSecret: '{{ include "galaxy-rabbitmq.fullname" . }}-default-user'
   protocol: amqp