Flux Structure Example

This repo contains a full example on how to structure your Flux Repository in a scalable and DRY way. Find a full post explaining this here: https://hackernoon.com/how-to-structure-your-k8s-gitops-repository-at-scale-part-1

Structure

.
├── clusters
│   ├── _profiles # Store all the different profiles
│   │   ├── _base # Base for all cluster profiles (things installed in all variants)
│   │   ├── home
│   │   └── prod
│   ├── home-cluster-raspi # A cluster instance
│   │   ├── flux-system # Generated by flux bootstrap
│   │   └── platform
│   │       ├── kustomization.yaml # Maps to a profile and injects secrets/config in the cluster
│   │       ├── cluster-secrets.yaml
│   │       └── cluster-config.yaml
│   ├── azure-cluster-aks
│   └── ...
└── platform # Contains all the platform services
    ├── grafana-operator
    │   └── _base
    ├── grafana-agent
    ├── cert-manager
    ├── datadog-operator
    ├── datadog-agent
    ├── ingress-nginx
    │   ├── _base    # Base implementation of this service
    │   └── nodeport # Feature to expose nginx in a NodePort instead of in a LoadBalancer
    ├── local-path-provisioner
    └── ...

Usage

Make sure to clone/fork this repo in GitHub before using it.

Run with Terraform/OpenTofu (requires setup of GitHub App)

We will be creating a GitHub App instead of a PAT so it only has access to specific repos that store GitOps and Flux config.

1. Create a new GitHub app here : https://github.com/settings/apps/new
   • You can give it any name and just provide any random HomePage Url
   • Do not provide webhook, callback or any other URL
2. Make sure that the app has the following permissions at repository level
   • Read and Write Administration (To create a deploy key)
   • Read and Write Contents (To create new content in the repos)
3. Once created
   • Note the AppID and create a new `Private Key`` and download the file
   • Open its "Public Page" and give it access to the previously cloned repo
4. Once installed it should show the app settings. Otherwise you can find them in Settings -> Applications -> Configure (your app). In the URL of this page, copy the installation ID, that should be a number. We will use it later.

# Install tooling with rtx (or install terraform and kind binaries yourself)
rtx install 

cd terraform/kind
terraform init

Place the secret key file in ./terraform/kind/secret-key.pem

Update ./terraform/kind/main.tf and update the values of the local variables:

• github_app_id is the ID of the APP (check step 3)
• github_app_installation_id is the ID you got from the URL (check step 4)

terraform apply

# This will ready a cluster locally and change your kubecontext to it!

Run manually

You can also do the flux bootstrap manually with flux CLI and then add the files. Point to a clone of this repo!

You can find the full docs here.

flux bootstrap github \
  --token-auth \
  --owner=my-github-username \
  --repository=my-repository-name \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

And then updating the ./clusters/my-cluster folder with the missing files. Check the templates in ./terraform/modules/flux-cluster/templates to get an idea of the files it is expecting.

Otherwise, you can find more information and examples in the post linked at the top.