Orchestrating scalability, networking, and storage with ease.
Table of Contents
The k8s-cluster-proxmox-terraform project orchestrates the deployment of a Kubernetes cluster on Proxmox using Terraform. It automates the setup of Kubernetes master and worker nodes, storage nodes, and a HAProxy load balancer. By streamlining the configuration of network connections, software installations, and resource provisioning, this project simplifies the creation of scalable and resilient Kubernetes environments. With a focus on ease of deployment and management, it offers a valuable solution for users looking to quickly establish robust Kubernetes clusters on Proxmox infrastructure.
| Feature | Description | |
|---|---|---|
| ⚙️ | Architecture | Infrastructure setup for a Kubernetes cluster on Proxmox using Terraform. Key components include master nodes, worker nodes, storage nodes, load balancer, and essential configurations. |
| 🔩 | Code Quality | Codebase exhibits clear structure and organization. Follows best practices for Terraform configurations. |
| 📄 | Documentation | Well-documented with detailed explanations of configuration files, variables, and setup procedures. README provides clear instructions for deployment. |
| 🔌 | Integrations | Relies on Proxmox API for VM management and Kubernetes components for cluster setup. |
| 🧩 | Modularity | Modular design with separate configuration files for different components, promoting code reusability and maintainability. |
| 🧪 | Testing | Testing frameworks not explicitly mentioned in the repository details. |
| ⚡️ | Performance | Efficient resource allocation and network setup for optimal performance. Load balancer enhances traffic distribution and scalability. |
| 🛡️ | Security | Utilizes SSH keys for secure connections, cloud-init for VM initialization, and TLS settings for API interactions. Follows security best practices for infrastructure deployment. |
| 📦 | Dependencies | Dependencies include Terraform for infrastructure management and Python libraries. |
| 🚀 | Scalability | Designed for scalability with provisions for adding more worker nodes, storage nodes, and load balancers to handle increased cluster load. |
└── k8s-cluster-proxmox-terraform/
├── assets
│ └── joinExtractor.py
├── configurations
│ └── prod.tfvars.exemple
├── k8s_loadbalancer.tf
├── k8s_master.tf
├── k8s_master_main.tf
├── k8s_storage.tf
├── k8s_worker.tf
├── main.tf
└── variables.tf.
| File | Summary |
|---|---|
| k8s_master_main.tf | Implements Kubernetes master VM deployment, incorporating necessary configurations and tools setup. Establishes key network connections, deploys essential software components, and initializes the cluster with specialized resources. |
| k8s_worker.tf | Creates Proxmox VMs for Kubernetes workers, configures network settings, and provisions packages. Establishes SSH connection for setup, fetches workerJoin script, and executes it to join master nodes. |
| k8s_master.tf | Deploys Kubernetes master nodes on Proxmox using Terraform. Ensures cloud-init configuration, networking, SSH setup, and software installation via remote-exec provisioner. Implements Docker, containerd, Kubernetes components, and joins master nodes after setup. |
| k8s_storage.tf | Creates storage nodes in the Proxmox VM cluster. Configures NFS server, helm, and storage provisioning. Ensures high availability and scalability for Kubernetes storage. |
| variables.tf | Defines variables for Proxmox API, networking, VM configuration, Kubernetes cluster specifics, load balancer settings, node resources, and IP addressing within the Terraform infrastructure for a Kubernetes cluster on Proxmox. |
| main.tf | Defines Proxmox provider configuration for managing infrastructure.uses API URL, token for authentication, and TLS settings. Crucial for interacting with Proxmox VMs in the Kubernetes cluster deployment through Terraform. |
| k8s_loadbalancer.tf | Defines a Proxmox virtual machine for a HAProxy load balancer, handling cluster traffic & API server requests. Conducts software installations & configurations for Kubernetes cluster management. |
configurations
| File | Summary |
|---|---|
| prod.tfvars.exemple | Specifies essential configuration variables for a Proxmox cluster deployment, including API details, network settings, SSH keys, VM parameters, and node specifics such as CPU, memory, and IP addresses. Crucial for setting up a robust Kubernetes cluster. |
System Requirements:
- Terraform:
1.6.4 - Proxmox:
8.1.3
Prepare a cloud-init template on your Proxmox server. You can use the following repository to create a cloud-init template: Proxmox Cloud-Init Imager
- Clone the k8s-cluster-proxmox-terraform repository:
git clone https://github.com/AlxFrst/k8s-cluster-proxmox-terraform
- Change to the project directory:
cd k8s-cluster-proxmox-terraform
- Install the dependencies and remove the line
backend "s3" {}from themain.tffile.[IMPORTANT] remove the line `backend "s3" {}` from the `main.tf` file. terraform init
- Copy the
secrets.tfvars.exemplefile tosecrets.tfvarsand fill in the necessary configuration details.cp configurations/prod.tfvars.exemple configurations/prod.tfvars
- Configure the configurations/prod.tfvars file with the necessary details.
- Plan the Terraform deployment:
terraform plan -var-file=configurations/prod.tfvars -var-file=configurations/secrets.tfvars
- If the plan looks good, apply the Terraform configuration:
terraform apply -var-file=configurations/prod.tfvars -var-file=configurations/secrets.tfvars
- After the deployment is complete, access the Kubernetes cluster by the Load Balancer IP address and check the nodes:
kubectl get nodes
Installation & Deploy with Terraform using Github Actions (Perfect for CI/CD and continuous deployment)
- Automated Deployment: Github Actions automates the deployment process, ensuring consistency and reliability.
- Version Control: Changes to the Terraform configuration are versioned and tracked in the repository.
- Scalability: Easily scale the Kubernetes cluster by modifying the Terraform configuration and triggering the Github Actions workflow.
- MinIO Server: To store the Terraform state file, you need a MinIO server.
- OpenVPN Server: To access the Proxmox API, you need an OpenVPN server, you can use the script in tools/openvpnInstall.sh to install an OpenVPN server on a Ubuntu server. (Don't forget to open the port 1194 on your firewall)
- Fork the repository to your Github account.
- Create a new repository secret with the name
SECRETS_PRODand the content of yoursecrets.tfvarsin base64.- Setup a MinOI server follwing the official documentation: MinIO Quickstart Guide
- Create a new repository secret with the name
BACKEND_PRODand the content of yourbackend.tfvarsin base64 (replace the MinIO credentials with your own).- Push a new commit to the repository to trigger the Github Actions workflow.
Contributions are welcome! Here are several ways you can contribute:
- Report Issues: Submit bugs found or log feature requests for the
k8s-cluster-proxmox-terraformproject. - Submit Pull Requests: Review open PRs, and submit your own PRs.
- Join the Discussions: Share your insights, provide feedback, or ask questions.
Contributing Guidelines
- Fork the Repository: Start by forking the project repository to your github account.
- Clone Locally: Clone the forked repository to your local machine using a git client.
git clone https://github.com/AlxFrst/k8s-cluster-proxmox-terraform
- Create a New Branch: Always work on a new branch, giving it a descriptive name.
git checkout -b new-feature-x
- Make Your Changes: Develop and test your changes locally.
- Commit Your Changes: Commit with a clear message describing your updates.
git commit -m 'Implemented new feature x.' - Push to github: Push the changes to your forked repository.
git push origin new-feature-x
- Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.
- Review: Once your PR is reviewed and approved, it will be merged into the main branch. Congratulations on your contribution!
Contributor Graph