GitOps homelab on k3s — Ahmad Babaei

Two k3s clusters at home, fully reconciled by Flux from Git: photos, documents, media automation, home automation, monitoring, offsite backups, and this website.

What it is

Two single-node k3s clusters on the home LAN. Nothing is applied by hand: a make reinstall produces an empty cluster that Flux refills from Git.

Highlights

GitOps end to end — layered Flux Kustomizations (sources → controllers → config → apps) with dependsOn so each layer’s CRDs exist before the next consumes them.
Secrets in Git, encrypted — SOPS with two age recipients, one of them a break-glass key that lives only in a password manager.
Real certificates without open ports — cert-manager answers ACME DNS-01 against Route 53 with an IAM identity scoped to one hosted zone.
Stable ingress address — MetalLB in L2 mode gives Traefik a VIP that moves with the pod, so DNS never has to follow a node.
One pane of glass — a single Prometheus, Loki and Grafana; the second cluster only runs collectors and pushes to them.
Offsite backups — nightly restic CronJobs to S3 with lifecycle rules, budget alarms and documented restore drills.
This website — built by GitHub Actions into a ~10 MB static-web-server image, rolled out by Flux image automation, and served through a dedicated Traefik entrypoint that is the only thing reachable from the internet.

Forcing a reconcile

When a change should land now instead of at the next interval, ask Flux to pull the source and reconcile the layer:

flux reconcile source git flux-system
flux reconcile kustomization apps --with-source
kubectl -n flux-system get kustomizations

Each layer is an ordinary Flux Kustomization that waits for the one before it:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 10m
  path: ./clusters/home/apps
  dependsOn:
    - name: config
  sourceRef:
    kind: GitRepository
    name: flux-system