Important
As of October 11th, 2024, I am no longer using Kubernetes in my homelab.
Thank you to everyone that has followed, had questions and learnt from my k8s journey.
My new homelab repo is built with Nix + NixOS at can be found at: https://github.com/deedee-ops/nixlab
This will be publically archived for learning purposes but please note that it will be outdated.
This is a repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using tools like OpenTofu, Kubernetes, ArgoCD, Renovate and GitHub Actions.
This semi hyper-converged cluster runs Talos Linux, an immutable and ephemeral Linux distribution built for Kubernetes, deployed on bare-metal Intel NUCs. Rook then provides my workloads with persistent block, and file storage; while a seperate server provides file storage for my media.
- actions-runner-controller: Self-hosted Github runners.
- cilium: Internal Kubernetes networking plugin.
- cert-manager: Creates SSL certificates for services in my Kubernetes cluster.
- external-dns: Automatically manages DNS records from my cluster in a cloud DNS provider.
- ingress-nginx: Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.
- rook: Distributed block storage for peristent storage.
- spegel: Stateless cluster local OCI registry mirror.
- vault: Safe and encrypted storage for all Kubernetes secrets.
- volsync: Backup and recovery of persistent volume claims.
ArgoCD watches the clusters in my kubernetes folder (see Directories below), and makes the changes to my clusters based on the state of my Git repository.
The way ArgoCD works for me here is it will recursively search the kubernetes/clusters/${cluster} folder,
and deploys all application.yaml manifests. I follow "app of apps" pattern, so cluster apps can include other apps,
which can be shared between clusters, and which live under kubernetes/apps directory.
Renovate watches my entire repository looking for dependency updates. When they are found a PR is automatically created. When some PRs are merged ArgoCD applies the changes to my cluster.
This Git repository contains the following directories under Kubernetes.
📁 kubernetes
├── 📁 apps # applications
└── 📁 clusters # clusters
├── 📁 deedee # main cluster
└── 📁 meemee # development cluster, deployed on VMs
📁 opentofu # opentofu scripts for external services (cloudflare)
📁 talos # talhelper scripts to bootstrap TalosWhile most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically need whether my cluster is online or not.
| Service | Use | Cost |
|---|---|---|
| addy.io | Email address protection | $12/yr |
| BorgBase | Backups | $80/yr |
| Cloudflare | Domains and tunnel | Free |
| GitHub | Hosting this repository and continuous integration/deployments | Free |
| Migadu | Email hosting | $19/yr |
| Pushover | Kubernetes Alerts and application notifications | $5 (one time) |
| Total: ~$10/mo |
| Device | Count | OS Disk Size | Data Disk Size | Ram | Operating System | Purpose |
|---|---|---|---|---|---|---|
| Dell Wyse 5070 | 3 | 128GB SSD | - | 8GB | Talos Linux | Kubernetes Masters |
| Intel NUC12WSHi5 | 3 | 128GB SSD | 512GB NVMe & 1TB PLP SSD(rook-ceph) | 64GB | Talos Linux | Kubernetes Workers |
| Synology DS1621+ | 1 | 256GB SSD | 4x4TB HDD (mirrored) | 32GB | Synology DSM | NFS + Backup Server |
| Minisforum MS-01 | 1 | 1TB SSD | - | 48GB | Proxmox PVE | Router + VMs |
| TP-LINK SG3428X-M2 | 1 | - | - | - | - | 2.5Gb Core Switch |
| TP-LINK SG2005P-PD | 1 | - | - | - | - | 1Gb PoE Switch |
Thanks to all the people who donate their time to the Home Operations Discord community. Be sure to check out kubesearch.dev for ideas on how to deploy applications or get ideas on what you may deploy.
See my awful commit history
See LICENSE