Why One Cluster Isn't Always Enough
Summary
- A cluster is a failure and trust boundary, so the number of clusters you run is a design decision about blast radius, not just capacity.
- Insight: you scale a single cluster for throughput, but you add clusters for isolation - the two axes are independent.
- Key Concepts: blast radius, regional locality, environment isolation, control-plane limits, tenancy model.
- When to Use: reach for multiple clusters when regulatory, latency, availability, or upgrade-risk boundaries do not fit inside one control plane.
- Limitations/Trade-offs: every extra cluster multiplies operational surface - upgrades, add-ons, RBAC, and observability all fan out.
- Related Topics: Cluster API for lifecycle, Fleet GitOps for config delivery, DR standby for regional failover.
Foundations
A Kubernetes cluster is one control plane governing a set of nodes.
Everything inside that boundary shares an API server, an etcd datastore, a set of controllers, and a scheduler.
That shared substrate is exactly what makes a single cluster convenient and also what makes it a single point of correlated failure.
The mental model that matters is the blast radius: when something goes wrong, how much of your platform is affected at once.
A bad admission webhook, a corrupted etcd, a runaway controller, or a botched upgrade can degrade an entire cluster.
If all of production lives in one cluster, that incident is a full outage.
So the first-principles question is not "how big can one cluster get" but "what should be allowed to fail together."
Capacity is rarely the reason to split.
Kubernetes 1.36 can run thousands of nodes and hundreds of thousands of pods per cluster within documented limits, so vertical growth is usually fine.
You split when a boundary is about trust, geography, or risk, not size.
Mechanics & Interactions
Think in three orthogonal boundaries: region, environment, and tenancy.
Regional boundaries exist because a cluster's control plane and etcd quorum live in one region.
Cross-region etcd is not supported for latency and quorum reasons, so a region generally maps to at least one cluster.
Running one cluster per region gives you locality for users and an availability boundary if a region degrades.
Environment boundaries separate dev, staging, and production.
Namespaces give you soft isolation inside a cluster, but they share the same API server, upgrade cadence, and node pool kernel.
A separate production cluster means you can rehearse a 1.35 to 1.36 upgrade in staging without any chance of touching production's control plane.
Tenancy boundaries decide whether teams share a cluster (namespace-as-tenant) or get their own.
Soft multi-tenancy uses namespaces, RBAC, ResourceQuota, NetworkPolicy, and Pod Security Standards.
Hard multi-tenancy - untrusted or strongly regulated tenants - often justifies a cluster per tenant because the API server and node kernel are shared attack surfaces.
These boundaries compose.
A common shape is: per-region production clusters, a separate staging cluster, and a management cluster that provisions and observes the rest.
Advanced Considerations & Applications
Multiple clusters trade a single large blast radius for many operational surfaces.
Every cluster needs its own upgrades, CNI, ingress or Gateway controllers, cert-manager, CSI drivers, autoscaler, and monitoring agents.
Version skew across the fleet is the quiet failure mode: a Gateway API CRD or ingress controller that differs between clusters produces bugs that only appear in one place.
This is why fleet operators standardize add-on versions and deliver them through GitOps rather than by hand.
Networking gets harder too.
Pods in different clusters do not share a flat network by default, so cross-cluster calls go through load balancers, a service mesh, or Gateway API rather than plain ClusterIP.
You either design services to be region-local or you invest in multi-cluster connectivity, and the latter is a real project.
Data is the hardest constraint.
Stateful systems rarely span clusters cleanly, so most designs pin a workload's primary datastore to one region and replicate asynchronously for disaster recovery.
Cost and toil also scale with cluster count, which pushes teams toward automation: declarative provisioning with Cluster API, config delivery with a GitOps controller, and identity federation so operators do not juggle dozens of kubeconfigs.
The goal is a fleet that feels like one system to operate even though it is many isolated failure domains.
Common Misconceptions
"More clusters always means more resilience."
Only if the clusters are truly independent and traffic can shift between them.
Ten clusters sharing one etcd backup pipeline or one identity provider still have shared failure modes.
"Namespaces are enough to isolate production from everything else."
Namespaces isolate names and enforce quotas and policy, but they share the control plane, node kernels, and upgrade timeline.
For upgrade-risk and hard-tenancy isolation you need separate clusters.
"A cluster per team or per service is the modern default."
Cluster sprawl multiplies upgrades, add-on drift, and cost.
Prefer the fewest clusters that satisfy your real boundaries, then use namespaces inside them.
"Multi-cluster gives you a global flat network for free."
It does not.
Cross-cluster networking requires deliberate design with a mesh, Gateway API, or cloud load balancers.
"Docker Engine runs the pods, so I need Docker on every node."
Nodes run containerd via the CRI; dockershim was removed.
Docker is for build and Compose-based local dev, not the in-cluster runtime.
FAQs
How many clusters should a small team start with?
Usually two or three: one production, one non-production, and optionally a management cluster once you automate provisioning.
Is one big cluster per region a valid strategy?
Yes, and it is often the right one - regional clusters with strong namespace isolation cover many organizations.
When is a management or "hub" cluster worth it?
When you provision clusters declaratively (Cluster API) or run fleet GitOps, a dedicated hub keeps that tooling off your workload clusters.
Do multiple clusters require a service mesh?
No, but a mesh or Gateway API makes cross-cluster traffic, mTLS, and failover far more manageable.
What is the biggest hidden cost of many clusters?
Add-on and version drift - keeping every cluster's CNI, controllers, and CRDs identical is the ongoing tax.
Related
Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).