Update/Add synapse.podSecurityContext vs synapse.securityContext …

…and fix how we install extra pip packages (#494) * attempt to fix pip issue * change to pip install --user for synapse * add an ls -hal so we can examine permissions * add more parameters for securityContexts and distinguish the podSecurityContext for synapse * set the exact location we want the pip package * use all available synapse container securityContext * try installing pip packages in an init container with a special volume just for that * remove ls from init container * attempt to add additional paths to existing pythong path * checkw hat the python path is * adding explicit pathing for PYTHONPATH * no special entry point for synapse * change chart bump to minor from patch version * default the pip extra packages to 100mi * add more notes for pvc and only deploy it if s3 is enabled * only mount the extra packages volume if we're installing s3 package
small-hack · Nov 9, 2023 · cbd5c9a · cbd5c9a
1 parent 7bbd9f8
commit cbd5c9a
Show file tree

Hide file tree

Showing 5 changed files with 120 additions and 25 deletions.
diff --git a/charts/matrix/Chart.yaml b/charts/matrix/Chart.yaml
@@ -8,7 +8,7 @@ sources:
 
 type: application
 
-version: 6.1.0
+version: 6.2.0
 
 # renovate: image=matrixdotorg/synapse
 appVersion: v1.95.1

diff --git a/charts/matrix/README.md b/charts/matrix/README.md
@@ -1,6 +1,6 @@
 # matrix
 
-![Version: 6.1.0](https://img.shields.io/badge/Version-6.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v1.95.1](https://img.shields.io/badge/AppVersion-v1.95.1-informational?style=flat-square)
+![Version: 6.2.0](https://img.shields.io/badge/Version-6.2.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v1.95.1](https://img.shields.io/badge/AppVersion-v1.95.1-informational?style=flat-square)
 
 A Helm chart to deploy a Matrix homeserver stack on Kubernetes
 
@@ -347,6 +347,12 @@ A Helm chart to deploy a Matrix homeserver stack on Kubernetes
 | synapse.metrics.annotations | bool | `true` |  |
 | synapse.metrics.enabled | bool | `true` | Whether Synapse should capture metrics on an additional endpoint |
 | synapse.metrics.port | int | `9092` | Port to listen on for metrics scraping |
+| synapse.podSecurityContext | object | `{"env":false,"fsGroup":1000,"runAsGroup":1000,"runAsNonRoot":true,"runAsUser":1000}` | securityContext for the entire synapse pod, including the all containers Does not work by default in all cloud providers, disable by default |
+| synapse.podSecurityContext.env | bool | `false` | Enable if your k8s environment allows containers to chuser/setuid https://github.com/matrix-org/synapse/blob/96cf81e312407f0caba1b45ba9899906b1dcc098/docker/start.py#L196 |
+| synapse.podSecurityContext.fsGroup | int | `1000` | A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod: 1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw---- If unset, the Kubelet will not modify the ownership and permissions of any volume. Note that this field cannot be set when spec.os.name is windows. |
+| synapse.podSecurityContext.runAsGroup | int | `1000` | group ID to run the synapse POD as |
+| synapse.podSecurityContext.runAsNonRoot | bool | `true` | Indicates that the pod's containers must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. |
+| synapse.podSecurityContext.runAsUser | int | `1000` | user ID to run the synapse POD as |
 | synapse.probes.liveness.periodSeconds | int | `10` | liveness probe seconds trying again |
 | synapse.probes.liveness.timeoutSeconds | int | `5` | liveness probe seconds before timing out |
 | synapse.probes.readiness.periodSeconds | int | `10` | readiness probe seconds trying again |
@@ -356,15 +362,19 @@ A Helm chart to deploy a Matrix homeserver stack on Kubernetes
 | synapse.probes.startup.timeoutSeconds | int | `5` | startup probe seconds before timing out |
 | synapse.replicaCount | int | `1` |  |
 | synapse.resources | object | `{}` |  |
-| synapse.securityContext.env | bool | `false` | Enable if your k8s environment allows containers to chuser/setuid https://github.com/matrix-org/synapse/blob/96cf81e312407f0caba1b45ba9899906b1dcc098/docker/start.py#L196 |
-| synapse.securityContext.fsGroup | int | `1000` |  |
-| synapse.securityContext.runAsGroup | int | `1000` | group to run the synapse container as |
-| synapse.securityContext.runAsNonRoot | bool | `true` |  |
-| synapse.securityContext.runAsUser | int | `1000` | user to run the synapse container as |
+| synapse.securityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":false,"runAsGroup":1000,"runAsNonRoot":true,"runAsUser":1000}` | securityContext for the synapse CONTAINER ONLY Does not work by default in all cloud providers, disable by default |
+| synapse.securityContext.allowPrivilegeEscalation | bool | `false` | AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN Note that this field cannot be set when spec.os.name is windows. |
+| synapse.securityContext.readOnlyRootFilesystem | bool | `false` | Whether this container has a read-only root filesystem. Default is false. Note that this field cannot be set when spec.os.name is windows. |
+| synapse.securityContext.runAsGroup | int | `1000` | group ID to run the synapse container as |
+| synapse.securityContext.runAsNonRoot | bool | `true` | Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. |
+| synapse.securityContext.runAsUser | int | `1000` | user ID to run the synapse container as |
 | synapse.service.federation.port | int | `80` |  |
 | synapse.service.federation.type | string | `"ClusterIP"` |  |
 | synapse.service.port | int | `80` | service port for synapse |
 | synapse.service.type | string | `"ClusterIP"` | service type for synpase |
+| volumes.extraPipPackages.capacity | string | `"100Mi"` | Capacity of the extra pip packages PVC. Note: 1Mi is more than enough, but some cloud providers set a min PVC size of 1Mi or 1Gi, adjust as necessary |
+| volumes.extraPipPackages.existingClaim | string | `""` | name of an existing persistent volume claim for the extra pip packages |
+| volumes.extraPipPackages.storageClass | string | `""` | Storage class (optional) |
 | volumes.media.capacity | string | `"10Gi"` | Capacity of the media PVC - ignored if using exsitingClaim |
 | volumes.media.existingClaim | string | `""` | name of an existing PVC to use for uploaded attachments and multimedia |
 | volumes.media.storageClass | string | `""` | Storage class of the media PVC - ignored if using exsitingClaim |

diff --git a/charts/matrix/templates/synapse/deployment.yaml b/charts/matrix/templates/synapse/deployment.yaml
@@ -24,17 +24,17 @@ spec:
         {{- toYaml . | nindent 8 }}
       {{- end }}
       securityContext:
-        {{- if .Values.synapse.securityContext.runAsUser }}
-        runAsUser: {{ .Values.synapse.securityContext.runAsUser }}
+        {{- if .Values.synapse.podSecurityContext.runAsUser }}
+        runAsUser: {{ .Values.synapse.podSecurityContext.runAsUser }}
         {{- end }}
-        {{- if .Values.synapse.securityContext.runAsGroup }}
-        runAsGroup: {{ .Values.synapse.securityContext.runAsGroup }}
+        {{- if .Values.synapse.podSecurityContext.runAsGroup }}
+        runAsGroup: {{ .Values.synapse.podSecurityContext.runAsGroup }}
         {{- end }}
-        {{- if .Values.synapse.securityContext.fsGroup }}
-        fsGroup: {{ .Values.synapse.securityContext.fsGroup }}
+        {{- if .Values.synapse.podSecurityContext.fsGroup }}
+        fsGroup: {{ .Values.synapse.podSecurityContext.fsGroup }}
         {{- end}}
-        {{- if .Values.synapse.securityContext.runAsNonRoot }}
-        runAsNonRoot: {{ .Values.synapse.securityContext.runAsNonRoot }}
+        {{- if .Values.synapse.podSecurityContext.runAsNonRoot }}
+        runAsNonRoot: {{ .Values.synapse.podSecurityContext.runAsNonRoot }}
         {{- end }}
       initContainers:
         {{- if .Values.postgresql.enabled }}
@@ -250,16 +250,37 @@ spec:
               mountPath: /data
             - name: signing-key
               mountPath: /data/keys
-      {{- end }} {{/* end if .Release.IsInstall */}}
-      containers:
-        - name: "synapse"
+        {{- if .Values.s3.enabled }}
+        - name: pip-install-extra-packages
           image: {{ include "matrix.image" . }}
           imagePullPolicy: {{ .Values.synapse.image.pullPolicy }}
-          {{- if .Values.s3.enabled }}
+          env:
+            - name: SYNAPSE_SERVER_NAME
+              value: {{ .Values.matrix.serverName }}
+            - name: SYNAPSE_REPORT_STATS
+              value: {{ .Values.matrix.telemetry | ternary "yes" "no" | quote }}
+            - name: PYTHONUSERBASE
+              value: /data/extra_pip_packages 
+            - name: PIP_CACHE_DIR
+              value: /tmp
           command:
+            - /bin/sh
+            - -ec
             - |
-              pip install synapse-s3-storage-provider
+              pip install --user synapse-s3-storage-provider
+          volumeMounts:
+            - name: synapse-config
+              mountPath: /data
+            - name: signing-key
+              mountPath: /data/keys
+            - name: extra-packages
+              mountPath: /data/extra_pip_packages
           {{- end }}
+      {{- end }} {{/* end if .Release.IsInstall */}}
+      containers:
+        - name: "synapse"
+          image: {{ include "matrix.image" . }}
+          imagePullPolicy: {{ .Values.synapse.image.pullPolicy }}
           env:
             {{- if .Values.synapse.securityContext.env }}
             {{- if .Values.synapse.securityContext.runAsUser }}
@@ -294,6 +315,10 @@ spec:
             - name: PGSSLROOTCERT
               value: {{ .Values.externalDatabase.sslrootcert }}
             {{- end }}
+            {{- if .Values.s3.enabled }}
+            - name: PYTHONPATH
+              value: "/usr/local/lib/python3.11/site-packages:/data/extra_pip_packages/lib/python3.11/site-packages"
+            {{- end }}
             {{- if .Values.synapse.extraEnv }}
             {{- toYaml .Values.synapse.extraEnv | nindent 12 }}
             {{- end }}
@@ -313,6 +338,10 @@ spec:
               mountPath: /data/keys
             - name: media-store
               mountPath: /data/media_store
+            {{- if .Values.s3.enabled }}
+            - name: extra-packages
+              mountPath: /data/extra_pip_packages
+            {{- end }}
             {{- if or .Values.bridges.whatsapp.enabled .Values.bridges.discord.enabled .Values.bridges.irc.enabled }}
             - name: bridges
               mountPath: /bridges
@@ -350,8 +379,11 @@ spec:
             capabilities:
               drop:
                 - ALL
-            readOnlyRootFilesystem: true
-            allowPrivilegeEscalation: false
+            runAsGroup: {{ .Values.synapse.securityContext.runAsGroup }}
+            runAsUser: {{ .Values.synapse.securityContext.runAsUser }}
+            runAsNonRoot: {{ .Values.synapse.securityContext.runAsNonRoot }}
+            readOnlyRootFilesystem: {{ .Values.synapse.securityContext.readOnlyRootFilesystem }}
+            allowPrivilegeEscalation: {{ .Values.synapse.securityContext.allowPrivilegeEscalation }}
           {{- with .Values.synapse.resources }}
           resources:
             {{- toYaml . | nindent 12 }}
@@ -374,6 +406,15 @@ spec:
             {{ else }}
             claimName: {{ .Values.volumes.signingKey.existingClaim }}
             {{- end }}
+        {{- if .Values.s3.enabled }}
+        - name: extra-packages
+          persistentVolumeClaim:
+            {{- if .Values.volumes.extraPipPackages.existingClaim }}
+            claimName: {{ .Values.volumes.extraPipPackages.existingClaim }}
+            {{ else }}
+            claimName: {{ include "matrix.fullname" . }}-extra-pip-packages
+            {{- end }}
+        {{- end }}
         - name: media-store
           persistentVolumeClaim:
             {{- if not .Values.volumes.media.existingClaim }}

diff --git a/charts/matrix/templates/synapse/extra-pip-packages-pvc.yaml b/charts/matrix/templates/synapse/extra-pip-packages-pvc.yaml
@@ -0,0 +1,18 @@
+{{- if and .Values.s3.enabled (not .Values.volumes.extraPipPackages.existingClaim) }}
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ include "matrix.fullname" . }}-extra-pip-packages
+  labels:
+{{ include "matrix.labels" . | indent 4}}
+{{ include "matrix.synapse.labels" . | indent 4}}
+spec:
+  {{- if .Values.volumes.extraPipPackages.storageClass }}
+  storageClassName: {{ .Values.volumes.extraPipPackages.storageClass }}
+  {{- end }}
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: {{ .Values.volumes.extraPipPackages.capacity }}
+{{- end }}
diff --git a/charts/matrix/values.yaml b/charts/matrix/values.yaml
@@ -371,6 +371,15 @@ volumes:
     storageClass: ""
     # -- name of an existing persistent volume claim for synapse config file
     existingClaim: ""
+  # optional PVC used only when s3.enabled is set to true, to install synapse-s3-storage-provider
+  extraPipPackages:
+    # -- Capacity of the extra pip packages PVC. Note: 1Mi is more than enough, but
+    # some cloud providers set a min PVC size of 1Mi or 1Gi, adjust as necessary
+    capacity: 100Mi
+    # -- Storage class (optional)
+    storageClass: ""
+    # -- name of an existing persistent volume claim for the extra pip packages
+    existingClaim: ""
 
 
 externalDatabase:
@@ -533,15 +542,32 @@ synapse:
       # -- liveness probe seconds trying again
       periodSeconds: 10
 
+
+  # -- securityContext for the synapse CONTAINER ONLY
   # Does not work by default in all cloud providers, disable by default
   securityContext:
-    # -- user to run the synapse container as
+    # -- user ID to run the synapse container as
     runAsUser: 1000
-    # -- group to run the synapse container as
+    # -- group ID to run the synapse container as
     runAsGroup: 1000
-    fsGroup: 1000
+    # -- Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed.
     runAsNonRoot: true
+    # -- Whether this container has a read-only root filesystem. Default is false. Note that this field cannot be set when spec.os.name is windows.
+    readOnlyRootFilesystem: false
+    # -- AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN Note that this field cannot be set when spec.os.name is windows.
+    allowPrivilegeEscalation: false
 
+  # -- securityContext for the entire synapse pod, including the all containers
+  # Does not work by default in all cloud providers, disable by default
+  podSecurityContext:
+    # -- user ID to run the synapse POD as
+    runAsUser: 1000
+    # -- group ID to run the synapse POD as
+    runAsGroup: 1000
+    # -- A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod: 1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw---- If unset, the Kubelet will not modify the ownership and permissions of any volume. Note that this field cannot be set when spec.os.name is windows.
+    fsGroup: 1000
+    # -- Indicates that the pod's containers must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed.
+    runAsNonRoot: true
     # -- Enable if your k8s environment allows containers to chuser/setuid
     # https://github.com/matrix-org/synapse/blob/96cf81e312407f0caba1b45ba9899906b1dcc098/docker/start.py#L196
     env: false