Skip to content

Latest commit

 

History

History
142 lines (128 loc) · 7.93 KB

README.md

File metadata and controls

142 lines (128 loc) · 7.93 KB

CSI WekaFS SELinux Support

General Information

When installing Weka CSI plugin on SELinux-enabled Kubernetes cluster, pods might be denied access to the persistent volumes provisioned on top of Weka filesystem.

The reason behind this is a lack of permissions for containers to access objects stored on Weka cluster.

In this directory you can find a custom policy that provides all the necessary security configuration to optionally enable pod access to WekaFS-based Persistent Volumes, and it should be applied on each Kubernetes worker node that is intended to service WekaFS-based persistent volumes.

The provided policy allows processes with container_t seclabel to access objects having wekafs_t label (which is set for all files and directories of mounted CSI volumes).

The policy comes both as a Type Enforcement file, and as a precompiled policy package. In order to use Weka CSI Plugin with SELinux enforcement, the following steps must be performed:

Custom SELinux Policy Installation

  1. Distribute the SELinux policy package to all Kubernetes nodes, by using either one of those options:

    • Clone Weka CSI Plugin Github repository, by issuing 
      git clone https://github.com/weka/csi-wekafs.git
    • Copy the content of selinux directory directly to Kubernetes nodes

  2. Apply the policy package directly by issuing:

    $ semodule -i csi-wekafs.pp

    Check that the policy was applied correctly:

    $ getsebool -a | grep wekafs
container_use_wekafs --> off

    If the output matches mentioned above, skip to step 4. Otherwise, proceed to step 3 to build the policy from sources.

  3. In certain circumstances (e.g. different Kernel version or Linux distribution), the pre-compiled policy installation could fail. In this case, the policy must be built and installed from source by following the procedure below.

    $ checkmodule -M -m -o csi-wekafs.mod csi-wekafs.te
$ semodule_package -o csi-wekafs.pp -m csi-wekafs.mod
$ make -f /usr/share/selinux/devel/Makefile csi-wekafs.pp
$ semodule -i csi-wekafs.pp

    NOTE: for this purpose, policycoreutils-devel package (or its alternative in case of Linux distribution different from RedHat family) is required

    Check that the policy was applied correctly:

    $ getsebool -a | grep wekafs
container_use_wekafs --> off

  4. The policy provides a boolean setting which allows on-demand enablement of relevant permissions. To enable WekaFS CSI volumes access from pods, perform the command

    $ setsebool container_use_wekafs=on

    To disable access, perform the command

    $ setsebool container_use_wekafs=off

    The configuration changes are applied immediately.

CSI Plugin Installation and Configuration

  1. Weka CSI Plugin must be installed in a SELinux-compatible mode to correctly label volumes.
    This can be done by setting the selinuxSupport value to either "enforced" or "mixed", either via editing values.yaml or by passing the parameter directly in Helm installation command, e.g.

    $ helm install --upgrade csi-wekafsplugin csi-wekafs/csi-wekafsplugin --namespace csi-wekafsplugin --create-namespace --set selinuxSupport=enforced

    NOTE: enforced and mixed modes are supported for CSI SELinux support.

    • When selinuxSupport is set to enforced, only SELinux-enabled CSI plugin node components will be installed
    • When selinuxSupport is set to mixed, both non-SELinux and SELinux-enabled components will be installed
    • When selinuxSupport is set to off, only non-SELinux CSI plugin node components will be installed.

    NOTE: Since SELinux status cannot be known from within CSI plugin pod, a certain way of distinguishing between SELinux-enabled and non-SELinux nodes needs to be established. Binding of relevant CSI node component to node is mutually exclusive and relies on node affinity mechanism by matching host labels.
    Hence, the following label must be set on each SELinux-enabled Kubernetes node to ensure the plugin start in compatibility mode:

    csi.weka.io/selinux_enabled="true"

    NOTE: If another label stating SELinux support is already maintained on nodes, the expected label name may be changed by editing the selinuxNodeLabel parameter by either modifying it in values.yaml or by setting it directly during plugin installation, e.g.

    $ helm install --upgrade csi-wekafsplugin csi-wekafs/csi-wekafsplugin --namespace csi-wekafsplugin --create-namespace --set selinuxSupport=mixed --set selinuxNodeLabel="selinux_enabled"

    NOTE: If node label was modified after Weka CSI Plugin node component has already deployed on that node, terminate the csi-wekafs-node-XXXX component on the affected node, a replacement pod will be scheduled on the node automatically, but with correct SELinux configuration.

Checking Plugin Operation & Troubleshooting

  1. Make sure you have configured a valid CSI API secret,Create a valid Weka CSI Plugin storageClass

    NOTE: If using an example storageClass, make sure to update endpoints and credentials prior to apply

  2. Provision a PersistentVolumeClaim

  3. Provision a DaemonSet, in order to be able access of all pods on all nodes

  4. Monitor the pod logs using a command below, nothing should be printed in log files:

    $ kubectl logs -f -lapp=csi-daemonset-app-on-dir-api

    IF the command returns a repeating message like the one below, it seems that the node on which the relevant pod is running is misconfigured:

    /bin/sh: can't create /data/csi-wekafs-test-api-gldmk.txt: Permission denied

  5. Obtain node name from the pod:

    $ kubectl get pod csi-wekafs-test-api-gldmk -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
csi-wekafs-test-api-gldmk   1/1     Running   0          98m   10.244.15.2   don-kube-8   <none>           <none>

  6. Connect to the relevant node and check if Weka CSI SELinux policy is installed and enabled

    $ getsebool -a | grep wekafs
container_use_wekafs --> on
    • If the output matches example, proceed to next step.
    • If no output, policy is not installed, proceed to Custom SELinux Policy Installation
    • If the policy is off, enable it and check output of the pod again by issuing 
      $ setsebool container_use_wekafs=on

  7. Check if the node is labeled with plugin is operating in SELinux-compatible mode by issuing the following command:

    $ kubectl describe node don-kube-8 | grep csi.weka.io/selinux_enabled
             csi.weka.io/selinux_enabled=true
    • If the output is empty, proceed to CSI Plugin Installation and Configuration

      NOTE: If the label was missing and added by you during troubleshooting, the CSI node server component must be restarted on the node.
      Perform the following command to terminate the relevant pod and another instance will start automatically:

      $ POD=$(kubectl get pod -n csi-wekafs -lcomponent=csi-wekafs-node -o wide | grep -w don-kube-8 | cut -d" " -f1)
$ kubectl delete pod -n csi-wekafs $POD
    • If the output matches example, proceed to next step

  8. Collect CSI node server logs from the matching Kubernetes nodes and contact Weka Customer Success Team:

    $ POD=$(kubectl get pod -n csi-wekafs -lcomponent=csi-wekafs-node -o wide | grep -w don-kube-8 | cut -d" " -f1)
$ kubectl logs -n csi-wekafs -c wekafs $POD > log.txt