Extending Kubernetes: The Big Idea

Summary

• Kubernetes is not a fixed product but a platform for building platforms: its API server is a generic, extensible store of typed objects.
• Insight: The core scheduler, kubelet, and controllers are just the first clients of an API that anyone can extend with new object types.
• Key Concepts: Custom Resource Definition (CRD), Custom Resource (CR), controller, reconciliation, declarative desired state, control loop.
• When to Use: Reach for extension when you want your own concepts (a Database, a Certificate, a KafkaTopic) to be first-class API objects that behave like built-in ones.
• Limitations/Trade-offs: A CRD alone stores data; without a controller acting on it, nothing happens. Extensibility adds operational surface area you must own.
• Related Topics: The operator pattern, controller-runtime, CRD versioning, and choosing extension mechanisms.

Foundations

Most systems ship a fixed set of nouns. Kubernetes ships a small set of built-in nouns and an API server designed to accept new ones.

A Pod, a Deployment, and a Service are all just structured records in etcd, served over a REST API with strong typing and validation.

The genuinely powerful idea is that these built-in types are not special to the API server. It treats them the same way it treats types you define.

That means you can register a new kind, say Database, and the API server will store it, validate it, version it, authorize access to it via RBAC, and serve it through kubectl, all without changing Kubernetes itself.

A Custom Resource Definition is the object that registers a new kind. Once applied, kubectl get databases works exactly like kubectl get pods.

But a stored record is inert. The declarative model only comes alive when something continuously reads the desired state and drives the real world toward it.

That something is a controller: a program running a control loop that watches objects and reconciles actual state with the spec the user declared.

Mechanics & Interactions

The API server is the hub. It persists objects to etcd and exposes a watch mechanism so clients get a stream of add, update, and delete events.

Built-in controllers live in the kube-controller-manager. The Deployment controller watches Deployments and creates ReplicaSets; the ReplicaSet controller watches ReplicaSets and creates Pods.

Nothing about that chain is privileged. Your controller can watch your Database custom resources and create Deployments, Services, Secrets, and PersistentVolumeClaims in response.

This is the key interaction: extension objects and built-in objects sit in the same API, so a custom controller composes cleanly with the rest of the platform.

The loop is level-triggered, not edge-triggered. A controller reconciles toward the current desired state rather than replaying a sequence of events, which makes it robust to missed messages and restarts.

Extension is not limited to CRDs. The aggregation layer lets you register an entire API service (like metrics.k8s.io) that the API server proxies to your own server.

For most teams CRDs are the right tool because they need no custom API server and inherit storage, validation, and RBAC for free.

Admission webhooks add another extension seam: they intercept create and update requests to validate or mutate objects before they are persisted.

Advanced Considerations & Applications

Extensibility is a design commitment, not a feature you toggle. Every CRD you introduce becomes a public API your users depend on.

CRDs are cluster-scoped registrations even when the resources themselves are namespaced, so naming and ownership matter across teams.

Because custom resources share the same API server, they share its limits. Very large or very numerous objects pressure etcd, and a chatty controller can overwhelm the API server with requests.

The payoff is that you can encode domain knowledge as software. An operator for a stateful database can automate backups, failover, and version upgrades that a plain Deployment never could.

This is why platform teams standardize on CRDs and controllers to expose golden paths: developers request a high-level resource, and a controller assembles the low-level machinery.

The trade-off is ownership. A built-in controller is maintained by the Kubernetes project; your custom controller is maintained by you, including its upgrades, security, and on-call burden.

For a fuller treatment of when that cost is justified, see The Operator Pattern Explained.

Common Misconceptions

"A CRD makes Kubernetes do something new." No. A CRD only teaches the API server a new type. Behavior requires a controller reconciling those objects.

"Custom resources are second-class." They are served, validated, versioned, and secured by the same API server as Pods, and support the same subresources and RBAC.

"Extending Kubernetes means forking or patching it." The whole point is that you extend from the outside, via CRDs, aggregated APIs, and webhooks, with no changes to core code.

"Operators replace Helm and manifests." They are different mechanisms with different strengths, covered in Operators vs Helm vs Plain Manifests.

"More CRDs is always better platform engineering." Each CRD is a long-lived contract; unnecessary ones add versioning and support cost without value.

FAQs

Is a CRD the same as an operator? No. A CRD is the API type. An operator is a CRD plus a controller that acts on it. See CRDs & Operators Basics.

Do I need Go to extend Kubernetes? Not strictly. CRDs are declarative YAML, and controllers can be written in any language, though Go with controller-runtime is the dominant, best-supported path.

Where are custom resources stored? In the same etcd backing the whole cluster, through the API server, exactly like built-in objects.

Can custom resources use RBAC? Yes. Once a CRD is registered, you write Roles and RoleBindings against its group and resource name like any other API.

What is the aggregation layer for? For cases where a CRD is not enough and you need a full custom API server, such as serving metrics or non-etcd-backed data.

Does extending Kubernetes affect cluster upgrades? It can. Your CRDs and controllers must stay compatible across Kubernetes versions, which is why CRD versioning discipline matters.

Related

• CRDs & Operators Basics
• The Operator Pattern Explained
• How Controllers Work (controller-runtime)
• Operators vs Helm vs Plain Manifests
• CRDs & Operators Best Practices

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).