Dell CSM for Replication is part of the CSM (Container Storage Modules) open-source suite of Kubernetes storage enablers for Dell products.
This project aims at extending native Kubernetes functionality to support Disaster Recovery workflows by utilizing storage array based replication.
CSM Replication project includes the following components:
- CSM Replication CRDs - Custom Resource definitions
- CSM Replication sidecar container for CSI drivers
- CSM Replication Controller - Multi-Cluster Controller
- repctl - helper utility for setup and queries
CSM Replication sidecar is installed as part of the CSI driver controller pod and utilizes custom Dell CSI extensions to communicate
with Dell CSI drivers. It includes a set of Kubernetes controllers which keep watch on requests related to Persistent Volumes (PVs)
and Persistent Volume Claims(PVCs). It uses dell-csi-extensions APIs to perform replication related actions.
You can read more about the extensions here.
CSM Replication Controller is a set of Kubernetes controllers which are multi-cluster aware and can communicate with various Kubernetes API servers to reconcile state. These controllers are responsible for the syncing of objects and the associated metadata.
This project has been created using kubebuilder, and the various kubernetes controllers
have been implemented using controller-runtime. Additionally, this project uses kustomize for generating various yaml manifests for Kubernetes objects.
For documentation, please visit Container Storage Modules documentation.
- Code of Conduct
- Maintainer Guide
- Committer Guide
- Contributing Guide
- List of Adopters
- Support
- Security
- About
This project contains multiple go modules, and you can build multiple binaries & images with the provided Makefile.
This project relies on the following tools which have to be installed in order to generate certain manifests.
| Tool | Version |
|---|---|
| controller-gen | v0.15.0 |
| kustomize | v5.4.3 |
The above tools can also be installed by running the command make tools.
Run the command make manifests to build the CSM Replication Custom Resource Definitions (CRDs). This command invokes controller-gen to generate the CRDs.
The API code is annotated with kubebuilder tags which are used by the controller-gen generators. The generated definitions are present in the form
of a kustomize recipe in the config/crd folder.
To build all the binaries associated with CSM Replication, run make build.
To compile individual binaries run:
make build-controller-managerto build the CSM Replication Controller, dell-replication-controller.make build-sidecar-managerto build the CSM Replication sidecar, dell-csi-replicator.make build-sidecar-migratorto build the CSM Replication sidecar, dell-csi-migrator.make build-sidecar-node-rescannerto build the CSM Replication sidecar, dell-csi-node-rescanner.
To build the repctl binary, run cd repctl && make build.
In order to simplify installation for users who don't want to use kustomize, statically generated manifests are stored in the repository.
- deploy/controller.yaml - manifest for installing the CSM Replication Controller.
- deploy/role.yaml - manifest for creating ClusterRole for CSM Replication Controller.
- deploy/replicationcrds.all.yaml - manifest for install CSM Replication CRDs.
In case there are modifications done which update the CRD, run make static-crd and commit changes to the CRD file in the repository.
In case there are modifications done which update any manifests for the installation of the CSM Replication Controller, run
make static-controller and commit changes to the file deploy/controller.yaml in the repository.
There are various Makefile targets available for building and pushing container images
| Target | Description |
|---|---|
| image-sidecar | Build dell-csi-replicator sidecar image |
| image-sidecar-push | Build dell-csi-replicator sidecar image & push to an image repo |
| image-controller | Build dell-replication-controller image |
| image-controller-push | Build dell-replication-controller image & push to an image repo |
| images | Build both dell-csi-replicator & dell-replication-controller images |
| images-push | Push both dell-csi-replicator & dell-replication-controller images |
Note: make images will also tag the built images with the release tags and push those tags to the image repo.
For building the RBAC rules for the sidecar, run make sidecar-rbac.
For building the RBAC rules for the controller, run make controller-rbac.
Please consult the Installation Guide for instructions on installing the replication controller and sidecar.
To build and install your own images using the provided Makefile targets, below information may be useful:
You can run the command make install to install the Custom Resource Definitions in your Kubernetes cluster.
The CSM Replication sidecar container can only be installed as a sidecar within the CSI driver controller pod. The kubernetes controllers which run as part of the sidecar container, communicate with the CSI drivers using gRPC over a Unix Domain Socket (UDS). The address for this socket can be passed as an argument to the sidecar container.
The RBAC permissions required by the kubernetes controllers are present in the file
config/sidecar-rbac/role.yaml and should be included in the CSI driver's RBAC permissions.
You can also run the dell-csi-replicator process directly in your Kubernetes cluster by running the command make run-sidecar.
Make sure that the kubernetes user has the desired RBAC permissions as described above.
You can install the CSM Replication Controller by running the command make deploy-controller. The namespace dell-replication-controller must exist in the cluster before this installation.
You can also run the dell-replication-controller process directly in your Kubernetes cluster by running the command make run-controller.
Make sure that the kubernetes user has the desired RBAC permissions.
If you wish to install CSM Replication Controller with repctl on your openSUSE server, when referring to this section, you need to install glibc-devel-static devel package before running make build.
Click here for details on how to test.