The Operator Pattern Explained
Summary
- An operator is a custom controller that encodes the operational knowledge of running a specific application as software.
- Insight: The operator pattern automates the day-2 tasks a human SRE would otherwise do: backups, failover, upgrades, scaling, and reconfiguration.
- Key Concepts: operator, day-2 operations, custom controller, reconciliation loop, domain knowledge as code, CRD as API.
- When to Use: When an application needs stateful, multi-step lifecycle management that plain manifests or Helm cannot express safely.
- Limitations/Trade-offs: An operator is a long-lived program you build, secure, upgrade, and put on-call for. It only earns its complexity when the automation saves real operational toil.
- Related Topics: Controllers and reconciliation, CRD design, running third-party operators in production.
Foundations
The operator pattern was named by CoreOS to describe a simple insight: the person who knows how to run an application can encode that knowledge into a controller.
A human operator of a database knows how to take backups, promote a replica during failover, and perform rolling version upgrades without data loss.
Those procedures are conditional and stateful. They depend on the current state of the system and often require ordered, careful steps.
Plain Kubernetes objects handle stateless rollouts well but do not know your application's semantics. A Deployment cannot know that a database primary must be drained before the pod is replaced.
An operator closes that gap. It introduces a Custom Resource that represents the application at a high level, like a PostgresCluster, and a controller that reconciles reality toward that spec.
The mental model: the CRD is the API you offer users, and the controller is the tireless SRE that keeps the promise the API makes.
Mechanics & Interactions
At runtime an operator is an ordinary Pod running in the cluster, usually with a ServiceAccount granted RBAC over the resources it manages.
It watches its custom resources and, on every change, runs a reconcile function that compares desired state (the spec) to observed state (the world) and takes corrective action.
Crucially the loop is level-triggered. It does not react to a single event; it reasserts the whole desired state each time, which makes it self-healing after crashes or missed updates.
The controller composes with built-in objects. To run a database it typically creates StatefulSets, Services, Secrets, PersistentVolumeClaims, and PodDisruptionBudgets, owning each so they are garbage-collected with the parent.
It reports progress back on the status subresource, so users and other systems can read .status to see whether the cluster is healthy, upgrading, or degraded.
Operators frequently define multiple custom resources. A database operator might offer PostgresCluster, PostgresBackup, and PostgresRestore, each with its own reconcile behavior.
The mechanics are covered in depth in How Controllers Work (controller-runtime); the point here is that the pattern is just a controller plus a domain-specific API.
Advanced Considerations & Applications
The hard part of an operator is not the plumbing; it is correctly encoding operational judgment for failure cases.
A good database operator must handle split-brain, partial upgrades, failed backups, and disk-full conditions without making things worse. This is where most operator complexity lives.
Operators are usually rated by maturity. The CNCF Operator Capability Levels run from basic install (Level 1) through seamless upgrades, full lifecycle, insights, and finally auto-pilot (Level 5).
Most homegrown operators sensibly target the lower levels: install and configure. Reaching auto-pilot for a stateful system is a major engineering investment.
Blast radius is a first-class concern. An operator often holds broad RBAC and can affect many workloads, so a bug can cascade across a namespace or cluster.
That is why scoping RBAC tightly, adding leader election so only one instance acts, and testing reconcile logic against failure injection all matter more than feature count.
The pattern applies well beyond databases: certificate management, messaging systems, service meshes, and internal platform abstractions are all common operator use cases.
An operator earns its complexity when the alternative is a runbook a human executes under pressure. If a Deployment or a Helm chart already does the job safely, an operator is over-engineering.
Common Misconceptions
"Every app should have an operator." Most stateless apps are served perfectly by Deployments and Helm. Operators pay off for stateful, procedure-heavy systems.
"An operator is a special Kubernetes feature." It is not. It is a naming convention for a custom controller plus one or more CRDs, using the same APIs anyone can use.
"Writing an operator is mostly writing YAML." The CRD is YAML, but the value is in the reconcile logic that handles real-world failure, which is real software with real tests.
"Higher capability level is always better." Auto-pilot automation is expensive and can hide problems. Many teams deliberately stop at reliable install and upgrade.
"An operator removes the need to understand the app." It encodes understanding; it does not replace it. You still own the operator's behavior during incidents.
FAQs
What is the difference between a controller and an operator? All operators are controllers. "Operator" specifically means a controller that manages an application using CRDs and encodes operational knowledge.
Do I need to write my own operator? Usually not. Prefer a proven third-party operator; see Using Operators in Production. Write your own only for domain-specific needs.
What language are operators written in? Most use Go with controller-runtime and kubebuilder, but the Operator SDK also supports Ansible and Helm-based operators for simpler cases.
How does an operator upgrade an application safely? By reconciling in ordered steps it controls: draining, updating one replica, verifying health, then proceeding, all driven from the spec.
Where does an operator run? As a normal Deployment in the cluster, typically in its own namespace with a scoped ServiceAccount and leader election enabled.
When is an operator the wrong choice? When plain manifests or a Helm chart already deploy and manage the workload safely; the added long-lived complexity is not justified.
Related
- CRDs & Operators Basics
- How Controllers Work (controller-runtime)
- Using Operators in Production
- Operators vs Helm vs Plain Manifests
- CRDs & Operators Best Practices
Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).