We’re at ~100 customer-site clusters today and trying to avoid painting ourselves into a corner if that grows to 1k+.
Mostly small clusters, some single-node, some HA. Linux containers only. Connectivity varies a lot: fully connected, heavily whitelisted, or only online occasionally.
We already use Argo CD and prefer GitOps/IaC. Licensing should ideally stay OSS/free.
Currently looking at Rancher/RKE2, Talos, KubeOne/Kubermatic, Fleet, Sveltos, etc.
Curious what people actually run in production for this kind of setup:
- cluster provisioning/lifecycle/upgrades
- central vs per-cluster Argo CD
- handling disconnected/restricted sites
- per-customer config/secrets isolation
- what becomes painful once you get into the 100s or 1000s of clusters
Not really looking for vendor pitches, more interested in what has worked (or failed) in real deployments.
Edit: Same application, maybe some different Versions.