The Platform Engineer's Toolkit, Conceptually
Summary
- A production Kubernetes cluster is a bare kernel of scheduling and API surface; the "platform" is the set of tools you add on top to make it operable, secure, and self-service.
- Insight: Kubernetes deliberately ships without an ingress controller, certificate manager, DNS integration, or backup system, because it is an extensible substrate rather than a finished PaaS.
- Key Concepts: the control plane (API server, scheduler, controllers), controllers/operators that reconcile desired state, CRDs that extend the API, and add-ons installed as workloads.
- When to Use: think in tool categories whenever you stand up a new cluster or audit an existing one for gaps.
- Limitations/Trade-offs: every add-on is another controller to patch, secure, and reason about; a bloated platform is as risky as a bare one.
- Related Topics: cert-manager, external-dns, Velero, k9s and stern, GitOps delivery.
Foundations
A vanilla Kubernetes cluster does surprisingly little on its own.
It schedules pods onto nodes, exposes an API, and runs a handful of core controllers. Everything else is something you choose and install.
The mental model that helps most: Kubernetes is a platform for building platforms, not the platform itself.
Its power comes from the reconciliation pattern. A controller watches the desired state in the API and continuously drives the real world toward it.
Custom Resource Definitions (CRDs) let third-party tools add their own object types to the same API. cert-manager adds Certificate, Velero adds Backup, and Gateway API adds HTTPRoute.
So "the toolkit" is really a set of controllers and CRDs that fill specific gaps the core project leaves open on purpose.
Thinking in categories keeps you from either forgetting a critical capability or accumulating redundant tools that do the same job.
Mechanics & Interactions
Here are the gaps a real cluster needs filled, and the category that fills each.
Ingress and traffic routing. The API defines Ingress and the newer Gateway API (Gateway, HTTPRoute), but ships no controller to honor them.
You install an ingress or Gateway controller (for example NGINX, Envoy Gateway, or a cloud load-balancer controller) to turn those objects into real L7 routing.
TLS certificates. Browsers need trusted certificates, and issuing/renewing them by hand does not scale.
cert-manager watches Ingress and Gateway objects, solves ACME challenges against Let's Encrypt or a private CA, and stores the resulting certs in Secrets.
DNS. A LoadBalancer Service gets an external IP or hostname, but nothing publishes that to your DNS provider.
external-dns reconciles Services and Ingress into records in Route53, Cloud DNS, Azure DNS, or Cloudflare.
Storage and backup. PersistentVolumes give pods durable disks, but a deleted namespace still loses its objects and its data.
Velero backs up API resources and triggers volume snapshots through the CSI driver, so you can restore a namespace or migrate a cluster.
Observability. The API exposes metrics endpoints, but no long-term storage or dashboards.
Prometheus scrapes metrics, Grafana visualizes them, and OpenTelemetry standardizes traces and logs across services.
Autoscaling nodes. The scheduler places pods, but does not create nodes when capacity runs out.
Cluster Autoscaler or Karpenter provisions and removes nodes based on pending pods.
Security and supply chain. Nothing stops you from running a root container from an unscanned image by default.
Pod Security Standards (via the admission label), Trivy or Grype for scanning, and Sigstore/cosign for signing close that gap.
Delivery. kubectl apply works, but it is imperative and hard to audit at scale.
Argo CD or Flux reconcile the cluster from Git, giving you a reviewable, revertible source of truth.
Human debugging. The API is verbose to query by hand under pressure.
k9s gives a live terminal UI over cluster state, and stern tails logs across many pods at once.
These categories interact. cert-manager depends on an ingress controller and often on external-dns so the ACME HTTP-01 challenge resolves publicly.
Velero depends on your CSI driver for volume snapshots. Argo CD often installs all of the above as tracked applications.
Advanced Considerations & Applications
The hard part is not picking tools, it is treating them as a coherent, versioned platform.
Every add-on runs as a controller with cluster-wide RBAC, so each one expands your attack surface and your patching burden.
Prefer tools that are widely adopted, actively maintained, and installable as versioned Helm charts or manifests you pin by digest.
Manage the whole set through GitOps so upgrades are reviewed and reversible, not applied by hand at 2am.
Watch CRD lifecycle carefully. Upgrading a controller can require CRD schema migrations, and Helm does not upgrade CRDs automatically in many charts.
Right-size the platform to the team. A three-service startup does not need Velero, Karpenter, and a full service mesh on day one.
A regulated enterprise, by contrast, will need backup, signing, and audit from the start.
The goal is a documented, opinionated toolkit that new services inherit for free, so app teams never reinvent ingress or TLS per project.
Common Misconceptions
"Kubernetes includes an ingress controller." It defines the Ingress and Gateway APIs but ships no controller; nothing routes external traffic until you install one.
"Docker runs my pods in the cluster." Nodes run containerd via the CRI; dockershim was removed in 1.24, so Docker Engine is a build and dev tool, not the in-cluster runtime.
"PersistentVolumes are backups." A PV is live storage tied to a claim; deleting the namespace or corrupting the data loses it. You still need Velero or an equivalent.
"More tools means a better platform." Each controller adds RBAC scope, upgrade risk, and cognitive load; a lean, well-understood set beats a sprawling one.
"Cluster Autoscaler and HPA are the same thing." HPA scales pod replicas; Cluster Autoscaler and Karpenter scale the underlying nodes. You usually need both.
FAQs
Do I need all of these tools from day one? No. Start with an ingress controller, cert-manager, and observability; add DNS, backup, and node autoscaling as scale and compliance demand.
Where do these tools actually run?
As pods in the cluster, usually in dedicated namespaces like cert-manager or velero, reconciling state through their own CRDs.
How do I keep the toolkit consistent across clusters? Manage every add-on through GitOps (Argo CD or Flux) with pinned chart versions, so each cluster converges to the same declared set.
Is a service mesh part of the base toolkit? Not usually. Add a mesh only when you need mTLS everywhere, fine-grained traffic policy, or per-request observability that ingress alone cannot give.
What replaces these on managed Kubernetes? Cloud providers often bundle load-balancer controllers, CSI drivers, and DNS integrations, but you still typically add cert-manager, Velero, and your own delivery tooling.
Related
Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).