Merge pull request #139 from msau42/deployment

update deployment page
kubernetes-csi · Apr 9, 2019 · 7813831 · 7813831
2 parents a37e79e + 88a70f3
commit 7813831
Showing 1 changed file with 78 additions and 79 deletions.
diff --git a/book/src/deploying.md b/book/src/deploying.md
@@ -1,101 +1,102 @@
 # Deploying CSI Driver on Kubernetes
 
-> ## *This page is out-of-date and under active development.*
-
 This page describes to CSI driver developers how to deploy their driver onto a Kubernetes cluster.
 
 ## Overview
-There are three components plus the kubelet that enable CSI drivers to provide storage to Kubernetes. These components are sidecar containers which are responsible for communication with both Kubernetes and the CSI driver, making the appropriate CSI calls for their respectful Kubernetes events.
 
-## Sidecar Containers
-[![sidecar-container](images/sidecar-container.png)](https://docs.google.com/a/greatdanedata.com/drawings/d/1JExJ_98dt0NAsJ7iI0_9loeTn2rbLeEcpOMEvKrF-9w/edit?usp=sharing)
+A CSI driver is typically deployed in Kubernetes as two components:
+a controller component and a per-node component.
 
-Sidecar containers manage Kubernetes events and make the appropriate calls to the CSI driver. These are the _external attacher_, _external provisioner_, _external snapshotter_ and the _driver registrar_.
+### Controller Plugin
 
-### External Attacher
-[external-attacher](https://github.com/kubernetes-csi/external-attacher) is a sidecar container that watches Kubernetes _VolumeAttachment_ objects and triggers CSI _ControllerPublish_ and _ControllerUnpublish_ operations against a driver endpoint. As of this writing, the external attacher does not support leader election and therefore there can be only one running per CSI driver.  For more information please read [_Attaching and Detaching_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#attaching-and-detaching).
+The controller component can be deployed as a Deployment or StatefulSet on
+any node in the cluster. It consists of the CSI driver that implements the
+CSI Controller service and one or more
+[sidecar containers](sidecar-containers.md). These controller
+sidecar containers typically interact with Kubernetes objects and make calls
+to the driver's CSI Controller service.
 
-Note, even though this is called the _external attacher_, its function is to call the CSI API calls _ControllerPublish_ and _ControllerUnpublish_. These calls most likely will occur in a node which is _not_ the one that will mount the volume. For this reason, many CSI drivers do not support these calls, instead doing the attach/detach and mount/unmount both in the CSI _NodePublish_ and _NodeUnpublish_ calls done by the kubelet at the node which is supposed to mount.
+It generally does not need direct access to the host and can perform all its
+operations through the Kubernetes API and external control plane services.
+Multiple copies of the controller component can be deployed for HA, however
+it is recommended to use leader election to ensure there is only one active
+controller at a time.
 
-### External Provisioner
-[external-provisioner](https://github.com/kubernetes-csi/external-provisioner) is a Sidecar container that watches Kubernetes _PersistentVolumeClaim_ objects and triggers CSI _CreateVolume_ and _DeleteVolume_ operations against a driver endpoint. For more information please read [_Provisioning and Deleting_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#provisioning-and-deleting).
+Controller sidecars include the external-provisioner, external-attacher,
+external-snapshotter, and external-resizer. Including a sidecar in the
+deployment may be optional.  See each sidecar's page for more details.
 
-### External Snapshotter
-[external-snapshotter](https://github.com/kubernetes-csi/external-snapshotter) is a Sidecar container that watches Kubernetes _VolumeSnapshot_ objects and triggers CSI _CreateSnapshot_ and _DeleteSnapshot_ operations against a driver endpoint. For more information please read [_Snapshot Design Proposal_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-snapshot.md).
+#### Communication with Sidecars
+[![sidecar-container](images/sidecar-container.png)](https://docs.google.com/a/greatdanedata.com/drawings/d/1JExJ_98dt0NAsJ7iI0_9loeTn2rbLeEcpOMEvKrF-9w/edit?usp=sharing)
 
-### Driver Registrar
-[driver-registrar](https://github.com/kubernetes-csi/driver-registrar) is a sidecar container that registers the CSI driver with kubelet, and adds the drivers custom NodeId to a label on the Kubernetes Node API Object. It does this by communicating with the _Identity_ service on the CSI driver and also calling the CSI _GetNodeId_ operation. The driver registrar must have the Kubernetes name for the node set through the environment variable `KUBE_NODE_NAME` as follows:
+Sidecar containers manage Kubernetes events and make the appropriate
+calls to the CSI driver. The calls are made by sharing a UNIX domain socket
+through an emptyDir volume between the sidecars and CSI Driver.
 
-```yaml
-        - name: csi-driver-registrar
-          imagePullPolicy: Always
-          image: quay.io/k8scsi/driver-registrar:v0.2.0
-          args:
-            - "--v=5"
-            - "--csi-address=$(ADDRESS)"
-          env:
-            - name: ADDRESS
-              value: /csi/csi.sock
-            - name: KUBE_NODE_NAME
-              valueFrom:
-                fieldRef:
-                  fieldPath: spec.nodeName
-          volumeMounts:
-            - name: socket-dir
-              mountPath: /csi
-```
+#### RBAC Rules
+
+Most controller sidecars interact with Kubernetes objects and therefore need
+to set RBAC policies. Each sidecar repository contains example RBAC
+configurations.
+
+### Node Plugin
+
+The node component should be deployed on every node in the cluster through a
+DaemonSet. It consists of the CSI driver that implements the CSI Node service and the
+[node-driver-registrar](node-driver-registrar) sidecar container.
+
+#### Communication with Kubelet
 
-### Kubelet
 [![kubelet](images/kubelet.png)](https://docs.google.com/a/greatdanedata.com/drawings/d/1NXaVNDh3mSDhog7Q3Y9eELyEF24F8Z-Kk0ujR3pyOes/edit?usp=sharing)
 
-The Kubernetes kubelet runs on every node and is responsible for making the CSI calls _NodePublish_ and _NodeUnpublish_. These calls mount and unmount the storage volume from the storage system, making it available to the Pod to consume. As shown in the _external-attacher_, most CSI drivers choose to implement both their attach/detach and mount/unmount calls in the _NodePublish_ and _NodeUnpublish_ calls. They do this because the kubelet makes the request on the node which is to consume the volume.
+The Kubernetes kubelet runs on every node and is responsible for making the CSI
+Node service calls. These calls mount and unmount the storage volume from the
+storage system, making it available to the Pod to consume. Kubelet makes calls
+to the CSI driver through a UNIX domain socket shared on the host via a HostPath
+volume. There is also a second UNIX domain socket that the node-driver-registrar
+uses to register the CSI driver to kubelet.
+
+#### Driver Volume Mounts
+The node plugin needs direct access to the host for making block devices and/or
+filesystem mounts available to the Kubernetes kubelet.
 
-### Mount point
-The mount point used by the CSI driver must be set to _Bidirectional_. See the example below:
+The mount point used by the CSI driver must be set to _Bidirectional_ to allow Kubelet
+on the host to see mounts created by the CSI driver container. See the example below:
 
 ```yaml
-          volumeMounts:
-            - name: socket-dir
-              mountPath: /csi
-            - name: mountpoint-dir
-              mountPath: /var/lib/kubelet/pods
-              mountPropagation: "Bidirectional"
-      volumes:
+      containers:
+      - name: my-csi-driver
+        ...
+        volumeMounts:
         - name: socket-dir
-          hostPath:
-            path: /var/lib/kubelet/plugins/csi-hostpath
-            type: DirectoryOrCreate
+          mountPath: /csi
         - name: mountpoint-dir
-          hostPath:
-            path: /var/lib/kubelet/pods
-            type: Directory
+          mountPath: /var/lib/kubelet/pods
+          mountPropagation: "Bidirectional"
+      - name: node-driver-registrar
+        ...
+        volumeMounts:
+        - name: registration-dir
+          mountPath: /registration
+      volumes:
+      # This volume is where the socket for kubelet->driver communication is done
+      - name: socket-dir
+        hostPath:
+          path: /var/lib/kubelet/plugins/<driver-name>
+          type: DirectoryOrCreate
+      # This volume is where the driver mounts volumes
+      - name: mountpoint-dir
+        hostPath:
+          path: /var/lib/kubelet/pods
+          type: Directory
+      # This volume is where the node-driver-registrar registers the plugin
+      # with kubelet
+      - name: registration-dir
+        hostPath:
+          path: /var/lib/kubelet/plugins_registry
+          type: Directory
 ```
 
-### RBAC Rules
-Side car containers need the appropriate permissions to be able to access and manipulate Kubernetes objects. Here are the RBAC rules needed:
-
-```yaml
-kind: ClusterRole
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
-  name: csi-hostpath-role
-rules:
-  - apiGroups: [""]
-    resources: ["persistentvolumes"]
-    verbs: ["create", "delete", "get", "list", "watch", "update"]
-  - apiGroups: [""]
-    resources: ["persistentvolumeclaims"]
-    verbs: ["get", "list", "watch", "update"]
-  - apiGroups: [""]
-    resources: ["nodes"]
-    verbs: ["get", "list", "watch", "update"]
-  - apiGroups: ["storage.k8s.io"]
-    resources: ["storageclasses"]
-    verbs: ["get", "list", "watch"]
-  - apiGroups: ["storage.k8s.io"]
-    resources: ["volumeattachments"]
-    verbs: ["get", "list", "watch", "update"]
-```
 
 ## Deploying
 Deploying a CSI driver onto Kubernetes is highlighted in detail in [_Recommended Mechanism for Deploying CSI Drivers on Kubernetes_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#recommended-mechanism-for-deploying-csi-drivers-on-kubernetes). 
@@ -121,10 +122,8 @@ Another feature that CSI depends on is mount propagation.  It allows the sharing
 
 ### Examples
 
-- Simple deployment example using a single pod for all components: see the [hostpath example](Example.html).
-- Full deployment example using a _DaemonSet_ for the node plugin and _StatefulSet_ for the controller plugin: check the [NFS driver deployment files][nfs-driver]. 
+- Simple deployment example using a single pod for all components: see the [hostpath example](example.html).
+- Full deployment example using a _DaemonSet_ for the node plugin and _StatefulSet_ for the controller plugin: TODO
 
 ## More information
 For more information, please read [_CSI Volume Plugins in Kubernetes Design Doc_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md).
-
-[nfs-driver]: https://github.com/kubernetes-csi/drivers/tree/master/pkg/nfs/deploy/kubernetes