diff --git a/README.md b/README.md index 59cc6e6b..9ecf21b7 100644 --- a/README.md +++ b/README.md @@ -1,55 +1,155 @@ -# Cluster API k3s +# Cluster API Provider k3s -Cluster API bootstrap provider k3s (CABP3) is a component of [Cluster API](https://github.com/kubernetes-sigs/cluster-api/blob/master/README.md) that is responsible for generating a cloud-init script to turn a Machine into a Kubernetes Node; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters. +Cluster API Provider k3s provides the following [Cluster API]( (CAPI) providers: -CABP3 is the bootstrap component of Cluster API for k3s and brings in the following CRDS and controllers: -- k3s bootstrap provider (KThrees, KThreesTemplate) +- **Cluster API Bootstrap Provider k3s (CABP3)** is responsible for generating the instructions (and encoding them as cloud-init) to turn a Machine into a Kubernetes Node; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters. -Cluster API ControlPlane provider k3s (CACP3) is a component of [Cluster API](https://github.com/kubernetes-sigs/cluster-api/blob/master/README.md) that is responsible for managing the lifecycle of control plane machines for k3s; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters. +- **Cluster API ControlPlane Provider k3s (CACP3)** is responsible for managing the lifecycle of control plane machines for k3s; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters. -CACP3 is the controlplane component of Cluster API for k3s and brings in the following CRDS and controllers: -- k3s controlplane provider (KThreesControlPlane) +## Getting Started -Together these two components make up Cluster API k3s... +**Warning**: Project and documentation are in an early stage, there is an assumption that a user of this provider is already familiar with Cluster API. Please consider contributing. -## Testing it out. +### Prerequisites -**Warning**: Project and documentation are in an early stage, there is an assumption that an user of this provider is already familiar with ClusterAPI. +Check out the general [Cluster API Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) page to see the prerequisites for Cluster API. -### Prerequisites +Three main pieces are: -Check out the [ClusterAPI Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) page to see the prerequisites for ClusterAPI. +1. Management cluster. In the `samples/azure/azure-setup.sh` script, [k3d](https://k3d.io/) is used, but feel free to use [kind](https://kind.sigs.k8s.io/) as well . +2. clusterctl. Please check out [Cluster API Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) for instructions. +3. Infrastructure specific prerequisites: + - For more Azure information go to [CAPZ Getting Started](https://capz.sigs.k8s.io/topics/getting-started.html) + - For more AWS information go to [CAPA Getting Started](https://cluster-api-aws.sigs.k8s.io/) + - For more Nutanix information go to [CAPX Getting Started](https://opendocs.nutanix.com/capx/latest/getting_started/) + - For more OpenStack information go to [CAPO Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html) + - For more Vsphere information go to [CAPV Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html) -Three main pieces are +In this getting started guide we'll be using Docker as the infrastructure provider (CAPD). -1. Bootstrap cluster. In the `samples/azure/azure-setup.sh` script, I use [k3d](https://k3d.io/), but feel free to use [kind](https://kind.sigs.k8s.io/) as well. -2. clusterctl. Please check out [ClusterAPI Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) for instructions. -3. Infrastructure Specific Prerequisites: +### Create a management cluster - * For more Azure information go to [CAPZ Getting Started](https://capz.sigs.k8s.io/topics/getting-started.html) - * For more AWS information go to [CAPA Getting Started](https://cluster-api-aws.sigs.k8s.io/) - * For more Nutanix information go to [CAPX Getting Started](https://opendocs.nutanix.com/capx/latest/getting_started/) - * For more OpenStack information go to [CAPO Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html) - * For more Vsphere information go to [CAPV Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html) +1. Ensure kind is installed ([instructions](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)) +2. Create a kind configuration to expose the local docker socket: -Cluster API k3s has been tested on AWS, Azure, AzureStackHCI, Nutanix, OpenStack, and Vsphere environments. +```bash +cat > kind-cluster-with-extramounts.yaml < This configuration tells clusterctl where to look for the provider manifests. + +2. Install the providers: + +```bash +clusterctl init --bootstrap k3s --control-plane k3s --infrastructure docker +``` + +3. Wait for the pods to start + +### Create a workload cluster + +There are a number of different cluster templates in the [samples](./samples/) directory. + +1. Run the following command to generate your cluster definition: + +```bash +clusterctl generate cluster --from samples/docker/quickstart.yaml test1 --kubernetes-version v1.28.6 --worker-machine-count 2 --control-plane-machine-count 1 > cluster.yaml +``` + +> NOTE: the kubernetes version specified with without the k3s suffix, this gets added in the template. + +2. Check the contents of the generated cluster definition in **cluster.yaml** +3. Ensure the definition is valid by doing a dry run: + +```bash +kubectl apply -f cluster.yaml --dry-run=server +``` + +4. When you are happy apply the definition: + +```bash +kubectl apply -f cluster.yaml +``` -* Support for External Databases -* Fix Token Logic -* Clean up Control Plane Provider Code -* Post an issue! +## Check the workload cluster + +- Check the state of the CAPI machines: + +```bash +kubectl get machine +``` + +- Get the kubeconfig for the cluster: + +```bash +clusterctl get kubeconfig test1 > workload-kubeconfig.yaml +``` + +- Connect to the child cluster + +```bash +kubectl --kubeconfig workload-kubeconfig.yaml get pods -A +``` + +### Deleting the workload cluster + +When deleting a cluster created via CAPI you must delete the top level **Cluster** resource. DO NOT delete using the original file. + +For the quick start: + +```bash +kubectl delete cluster test1 +``` + +### Additional Samples + +Cluster API k3s has been tested on AWS, Azure, AzureStackHCI, Nutanix, OpenStack, Docker and Vsphere environments. + +- To try out the Azure flow, fork the repo and look at `samples/azure/azure-setup.sh`. +- To try out the AWS flow, fork the repo and look at `samples/aws/aws-setup.sh`. +- To try out the Nutanix flow, fork the repo and look at `samples/nutanix/nutanix-setup.sh`. +- To try out the OpenStack flow, fork the repo and look at `samples/openstack/setup.sh`. +- To try out the Vsphere flow, fork the repo and look at `samples/vsphere-capv/setup.sh`. + +## Roadmap +- Support for External Databases +- Fix Token Logic +- Clean up Control Plane Provider Code +- Post an issue! diff --git a/samples/docker/quickstart.yaml b/samples/docker/quickstart.yaml new file mode 100644 index 00000000..0d76923e --- /dev/null +++ b/samples/docker/quickstart.yaml @@ -0,0 +1,91 @@ +apiVersion: cluster.x-k8s.io/v1beta1 +kind: Cluster +metadata: + name: ${CLUSTER_NAME} +spec: + clusterNetwork: + pods: + cidrBlocks: + - 10.45.0.0/16 + services: + cidrBlocks: + - 10.46.0.0/16 + serviceDomain: cluster.local + controlPlaneRef: + apiVersion: controlplane.cluster.x-k8s.io/v1beta1 + kind: KThreesControlPlane + name: ${CLUSTER_NAME}-control-plane + infrastructureRef: + apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + kind: DockerCluster + name: ${CLUSTER_NAME} +--- +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: DockerCluster +metadata: + name: ${CLUSTER_NAME} +spec: {} +--- +apiVersion: controlplane.cluster.x-k8s.io/v1beta1 +kind: KThreesControlPlane +metadata: + name: ${CLUSTER_NAME}-control-plane + namespace: default +spec: + infrastructureTemplate: + apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + kind: DockerMachineTemplate + name: ${CLUSTER_NAME}-control-plane + replicas: ${CONTROL_PLANE_MACHINE_COUNT} + version: ${KUBERNETES_VERSION}+k3s2 +--- +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: DockerMachineTemplate +metadata: + name: ${CLUSTER_NAME}-control-plane +spec: + template: + spec: + customImage: kindest/node:${KUBERNETES_VERSION} +--- +apiVersion: cluster.x-k8s.io/v1beta1 +kind: MachineDeployment +metadata: + name: worker-md-0 +spec: + clusterName: ${CLUSTER_NAME} + replicas: ${WORKER_MACHINE_COUNT} + selector: + matchLabels: + cluster.x-k8s.io/cluster-name: ${CLUSTER_NAME} + template: + spec: + version: ${KUBERNETES_VERSION} + clusterName: ${CLUSTER_NAME} + bootstrap: + configRef: + apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 + kind: KThreesConfigTemplate + name: ${CLUSTER_NAME}-md-0 + infrastructureRef: + apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + kind: DockerMachineTemplate + name: ${CLUSTER_NAME}-md-0 +--- +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: DockerMachineTemplate +metadata: + name: ${CLUSTER_NAME}-md-0 +spec: + template: + spec: + customImage: kindest/node:${KUBERNETES_VERSION} +--- +apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 +kind: KThreesConfigTemplate +metadata: + name: ${CLUSTER_NAME}-md-0 +spec: + template: + spec: + version: ${KUBERNETES_VERSION}+k3s2