Add CIS Benchmarks support to Rancher Distributions RKE/RKE2/K3s (#1523)

* add Support VMware Tanzu(TKGI) Benchmarks v1.2.53 with this change, we are adding 1. latest kubernetes cis benchmarks for VMware Tanzu1.2.53 2. logic to kube-bench so that kube-bench can auto detect vmware platform, will be able to execute the respective vmware tkgi compliance checks. 3. job-tkgi.yaml file to run the benchmark as a job in tkgi cluster Reference Document for checks: https://network.pivotal.io/products/p-compliance-scanner/#/releases/1248397 * add Support VMware Tanzu(TKGI) Benchmarks v1.2.53 with this change, we are adding 1. latest kubernetes cis benchmarks for VMware Tanzu1.2.53 2. logic to kube-bench so that kube-bench can auto detect vmware platform, will be able to execute the respective vmware tkgi compliance checks. 3. job-tkgi.yaml file to run the benchmark as a job in tkgi cluster Reference Document for checks: https://network.pivotal.io/products/p-compliance-scanner/#/releases/1248397 * release: prepare v0.6.15 (#1455) Signed-off-by: chenk <[email protected]> * build(deps): bump golang from 1.19.4 to 1.20.4 (#1436) Bumps golang from 1.19.4 to 1.20.4. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/setup-go from 3 to 4 (#1402) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v3...v4) --- updated-dependencies: - dependency-name: actions/setup-go dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: chenk <[email protected]> * Fix test_items in cis-1.7 - node - 4.2.12 (#1469) Related issue: #1468 * Fix node.yaml - 4.1.7 and 4.1.8 audit by adding uniq (#1472) * chore: add fips compliant images (#1473) For fips complaince we need to generate fips compliant images. As part of this change, we will create new kube-bench image which will be fips compliant. Image name follows this tag pattern <version>-ubi-fips * release: prepare v0.6.16-rc (#1476) * release: prepare v0.6.16-rc Signed-off-by: chenk <[email protected]> * release: prepare v0.6.16-rc Signed-off-by: chenk <[email protected]> --------- Signed-off-by: chenk <[email protected]> * release: prepare v0.6.16 official (#1479) Signed-off-by: chenk <[email protected]> * Update job.yaml (#1477) * Update job.yaml Fix on typo for image version * chore: sync with upstream Signed-off-by: chenk <[email protected]> --------- Signed-off-by: chenk <[email protected]> Co-authored-by: chenk <[email protected]> * release: prepare v0.6.17 (#1480) Signed-off-by: chenk <[email protected]> * Bump docker base images (#1465) During a recent CVE scan we found kube-bench to use `alpine:3.18` as the final image which has a known high CVE. ``` grype aquasec/kube-bench:v0.6.15 ✔ Vulnerability DB [no update available] ✔ Loaded image ✔ Parsed image ✔ Cataloged packages [73 packages] ✔ Scanning image... [4 vulnerabilities] ├── 0 critical, 4 high, 0 medium, 0 low, 0 negligible └── 4 fixed NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY libcrypto3 3.1.0-r4 3.1.1-r0 apk CVE-2023-2650 High libssl3 3.1.0-r4 3.1.1-r0 apk CVE-2023-2650 High openssl 3.1.0-r4 3.1.1-r0 apk CVE-2023-2650 High ``` The CVE in question was addressed in the latest [alpine release](https://www.alpinelinux.org/posts/Alpine-3.15.9-3.16.6-3.17.4-3.18.2-released.html), hence updating the dockerfiles accordingly * build(deps): bump golang from 1.20.4 to 1.20.6 (#1475) Bumps golang from 1.20.4 to 1.20.6. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add CIS Benchmarks support to Rancher Distributions RKE/RKE2/K3s Based on the information furnished in https://ranchermanager.docs.rancher.com/v2.7/pages-for-subheaders/rancher-hardening-guides kube-bench executes CIS-1.23 (Kubernetes v1.23) , CIS-1.24(Kubernetes v1.24),CIS-1.7 (Kubernetes v1.25,v1.26,v1.27) CIS Benchmarks of respective distributions. * RKE/RKE2 CIS Benchmarks Updated the order of checks for RKE and RKE2 Platforms. * fixed vulnerabilities|upgraded package golang.org/x/net to version v0.17.0 * Error handling for RKE Detection Pre-requisites * Based on the information furnished in https://ranchermanager.docs.rancher.com/v2.7/pages-for-subheaders/rancher-hardening-guides#hardening-guides-and-benchmark-versions, kube-bench executes CIS-1.23 (Kubernetes v1.23) , CIS-1.24(Kubernetes v1.24),CIS-1.7 (Kubernetes v1.25,v1.26,v1.27) CIS Benchmarks of respective distributions. updated documentation specific to added rancher platforms * addressed review comments 1.Implemented IsRKE functionality in kube-bench 2. Removed containerd from global level config and accommodated in individual config file 3. Corrected the control id from 1.2.25 to 1.2.23 in master.yaml(k3s-cis-1.23 and k3s-cis-1.24) * Removed unncessary dependency - kubernetes-provider-detector --------- Signed-off-by: chenk <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: chenk <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Andy Pitcher <[email protected]> Co-authored-by: Devendra Turkar <[email protected]> Co-authored-by: Guille Vigil <[email protected]> Co-authored-by: Jonas-Taha El Sesiy <[email protected]>
aquasecurity · Nov 26, 2023 · f8fe5ee · f8fe5ee
1 parent fac90f7
commit f8fe5ee
Show file tree

Hide file tree

Showing 61 changed files with 18,233 additions and 31 deletions.
diff --git a/cfg/config.yaml b/cfg/config.yaml
@@ -36,6 +36,7 @@ master:
       - /var/snap/microk8s/current/args/kube-apiserver
       - /etc/origin/master/master-config.yaml
       - /etc/kubernetes/manifests/talos-kube-apiserver.yaml
+      - /var/lib/rancher/rke2/agent/pod-manifests/kube-apiserver.yaml
     defaultconf: /etc/kubernetes/manifests/kube-apiserver.yaml
 
   scheduler:
@@ -53,6 +54,7 @@ master:
       - /var/snap/microk8s/current/args/kube-scheduler
       - /etc/origin/master/scheduler.json
       - /etc/kubernetes/manifests/talos-kube-scheduler.yaml
+      - /var/lib/rancher/rke2/agent/pod-manifests/kube-scheduler.yaml
     defaultconf: /etc/kubernetes/manifests/kube-scheduler.yaml
     kubeconfig:
       - /etc/kubernetes/scheduler.conf
@@ -77,6 +79,7 @@ master:
       - /var/snap/kube-controller-manager/current/args
       - /var/snap/microk8s/current/args/kube-controller-manager
       - /etc/kubernetes/manifests/talos-kube-controller-manager.yaml
+      - /var/lib/rancher/rke2/agent/pod-manifests/kube-controller-manager.yaml
     defaultconf: /etc/kubernetes/manifests/kube-controller-manager.yaml
     kubeconfig:
       - /etc/kubernetes/controller-manager.conf
@@ -101,6 +104,7 @@ master:
       - /var/snap/etcd/common/etcd.conf.yaml
       - /var/snap/microk8s/current/args/etcd
       - /usr/lib/systemd/system/etcd.service
+      - /var/lib/rancher/rke2/server/db/etcd/config
     defaultconf: /etc/kubernetes/manifests/etcd.yaml
     defaultdatadir: /var/lib/etcd/default.etcd
 
@@ -132,6 +136,9 @@ node:
       - "/etc/kubernetes/certs/ca.crt"
       - "/etc/kubernetes/cert/ca.pem"
       - "/var/snap/microk8s/current/certs/ca.crt"
+      - "/var/lib/rancher/rke2/agent/server.crt"
+      - "/var/lib/rancher/rke2/agent/client-ca.crt"
+      - "/var/lib/rancher/k3s/agent/client-ca.crt"
     svc:
       # These paths must also be included
       #  in the 'confs' property below
@@ -151,8 +158,12 @@ node:
       - "/var/lib/kubelet/kubeconfig"
       - "/etc/kubernetes/kubelet-kubeconfig"
       - "/etc/kubernetes/kubelet/kubeconfig"
+      - "/etc/kubernetes/ssl/kubecfg-kube-node.yaml"
       - "/var/snap/microk8s/current/credentials/kubelet.config"
       - "/etc/kubernetes/kubeconfig-kubelet"
+      - "/var/lib/rancher/rke2/agent/kubelet.kubeconfig"
+      - "/var/lib/rancher/k3s/server/cred/admin.kubeconfig"
+      - "/var/lib/rancher/k3s/agent/kubelet.kubeconfig"
     confs:
       - "/etc/kubernetes/kubelet-config.yaml"
       - "/var/lib/kubelet/config.yaml"
@@ -177,6 +188,8 @@ node:
       - "/etc/systemd/system/snap.kubelet.daemon.service"
       - "/etc/systemd/system/snap.microk8s.daemon-kubelet.service"
       - "/etc/kubernetes/kubelet.yaml"
+      - "/var/lib/rancher/rke2/agent/kubelet.kubeconfig"
+
     defaultconf: "/var/lib/kubelet/config.yaml"
     defaultsvc: "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
     defaultkubeconfig: "/etc/kubernetes/kubelet.conf"
@@ -200,8 +213,11 @@ node:
       - "/etc/kubernetes/kubelet-kubeconfig"
       - "/etc/kubernetes/kubelet-kubeconfig.conf"
       - "/etc/kubernetes/kubelet/config"
+      - "/etc/kubernetes/ssl/kubecfg-kube-proxy.yaml"
       - "/var/lib/kubelet/kubeconfig"
       - "/var/snap/microk8s/current/credentials/proxy.config"
+      - "/var/lib/rancher/rke2/agent/kubeproxy.kubeconfig"
+      - "/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig"
     svc:
       - "/lib/systemd/system/kube-proxy.service"
       - "/etc/systemd/system/snap.microk8s.daemon-proxy.service"
@@ -227,6 +243,8 @@ etcd:
       - /var/snap/etcd/common/etcd.conf.yaml
       - /var/snap/microk8s/current/args/etcd
       - /usr/lib/systemd/system/etcd.service
+      - /var/lib/rancher/rke2/agent/pod-manifests/etcd.yaml
+      - /var/lib/rancher/k3s/server/db/etcd/config
     defaultconf: /etc/kubernetes/manifests/etcd.yaml
     defaultdatadir: /var/lib/etcd/default.etcd
 
@@ -272,6 +290,15 @@ version_mapping:
   "cis-1.6-k3s": "cis-1.6-k3s"
   "cis-1.24-microk8s": "cis-1.24-microk8s"
   "tkgi-1.2.53": "tkgi-1.2.53"
+  "k3s-cis-1.7": "k3s-cis-1.7"
+  "k3s-cis-1.23": "k3s-cis-1.23"
+  "k3s-cis-1.24": "k3s-cis-1.24"
+  "rke-cis-1.7": "rke-cis-1.7"
+  "rke-cis-1.23": "rke-cis-1.23"
+  "rke-cis-1.24": "rke-cis-1.24"
+  "rke2-cis-1.7": "rke2-cis-1.7"
+  "rke2-cis-1.23": "rke2-cis-1.23"
+  "rke2-cis-1.24": "rke2-cis-1.24"
 
 target_mapping:
   "cis-1.5":
@@ -386,3 +413,57 @@ target_mapping:
     - "controlplane"
     - "node"
     - "policies"
+  "k3s-cis-1.7":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "k3s-cis-1.23":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "k3s-cis-1.24":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "rke-cis-1.7":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "rke-cis-1.23":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "rke-cis-1.24":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "rke2-cis-1.7":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "rke2-cis-1.23":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
+  "rke2-cis-1.24":
+    - "master"
+    - "etcd"
+    - "controlplane"
+    - "node"
+    - "policies"
diff --git a/cfg/k3s-cis-1.23/config.yaml b/cfg/k3s-cis-1.23/config.yaml
@@ -0,0 +1,46 @@
+---
+## Version-specific settings that override the values in cfg/config.yaml
+
+master:
+  components:
+    - apiserver
+    - scheduler
+    - controllermanager
+    - etcd
+    - policies
+
+  apiserver:
+    bins:
+      - containerd
+
+  scheduler:
+    bins:
+      - containerd
+
+  controllermanager:
+    bins:
+      - containerd
+
+  etcd:
+    bins:
+      - containerd
+
+  node:
+    components:
+      - kubelet
+      - proxy
+
+    kubelet:
+      bins:
+        - containerd
+      defaultkubeconfig: /var/lib/rancher/k3s/agent/kubelet.kubeconfig
+      defaultcafile: /var/lib/rancher/k3s/agent/client-ca.crt
+
+    proxy:
+      bins:
+        - containerd
+      defaultkubeconfig: /var/lib/rancher/k3s/agent/kubeproxy.kubeconfig
+
+  policies:
+    components:
+      - policies
diff --git a/cfg/k3s-cis-1.23/controlplane.yaml b/cfg/k3s-cis-1.23/controlplane.yaml
@@ -0,0 +1,47 @@
+---
+controls:
+version: "k3s-cis-1.23"
+id: 3
+text: "Control Plane Configuration"
+type: "controlplane"
+groups:
+  - id: 3.1
+    text: "Authentication and Authorization"
+    checks:
+      - id: 3.1.1
+        text: "Client certificate authentication should not be used for users (Manual)"
+        type: "manual"
+        remediation: |
+          Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
+          implemented in place of client certificates.
+        scored: false
+
+  - id: 3.2
+    text: "Logging"
+    checks:
+      - id: 3.2.1
+        text: "Ensure that a minimal audit policy is created (Manual)"
+        audit: "journalctl -D /var/log/journal  -u k3s | grep 'Running kube-apiserver' | tail -n1 | grep 'audit-policy-file'"
+        type: "manual"
+        tests:
+          test_items:
+            - flag: "--audit-policy-file"
+              set: true
+        remediation: |
+          Create an audit policy file for your cluster.
+        scored: false
+
+      - id: 3.2.2
+        text: "Ensure that the audit policy covers key security concerns (Manual)"
+        type: "manual"
+        remediation: |
+          Review the audit policy provided for the cluster and ensure that it covers
+          at least the following areas,
+          - Access to Secrets managed by the cluster. Care should be taken to only
+            log Metadata for requests to Secrets, ConfigMaps, and TokenReviews, in
+            order to avoid risk of logging sensitive data.
+          - Modification of Pod and Deployment objects.
+          - Use of `pods/exec`, `pods/portforward`, `pods/proxy` and `services/proxy`.
+          For most requests, minimally logging at the Metadata level is recommended
+          (the most basic level of logging).
+        scored: false