Technical Leadership Best Practices
How to Use This List
These are the practices that separate a platform team that ships from one that gatekeeps.
Read the metric group first. If you cannot measure time-to-safe-deploy, the rest is opinion.
Each group stands alone - adopt one, prove it, then take the next. Doing all five at once is how platform initiatives die.
A - Measure the Platform Like a Product
Track time-to-safe-deploy as the headline metric. Measure from a new service's first commit to it running in production, healthy, with probes, limits, policy, and alerting. This single number captures whether your golden path works, and it is the only platform metric an executive will remember.
Instrument the golden path, not the cluster. Node count and pod density describe your infrastructure; template adoption rate and PR-to-prod latency describe your product. Report the second set.
Count exceptions as a design signal. Three teams filing the same admission exception is a missing feature, not three problems. Review exception reasons quarterly and turn the repeats into template options.
Use DORA metrics per team, never as a ranking. Deploy frequency and change-failure rate are diagnostic - a team with a low deploy rate usually has a platform friction you own. The moment they become a leaderboard, they become fiction.
Survey your developers and publish the results. A quarterly two-question survey ("what slowed you down most?" and "what would you delete?") beats a dashboard for finding real friction. Publishing the answers is what makes teams answer honestly next quarter.
B - Pave the Road Before You Close the Dirt One
Ship the template before the policy. Enforcing restricted Pod Security Standards without a compliant base image teaches teams that the platform blocks work. The paved road comes first, always.
Run every new policy in audit mode first. Kyverno's validationFailureAction: Audit and Gatekeeper's dryrun tell you the blast radius before you own it. Clear the backlog, then flip to enforce.
Version templates as real software. SemVer the chart, publish it to an OCI registry, and stamp a template-version label on everything it renders. If you cannot query who is on which version, you cannot deprecate anything.
Send the migration PR yourself. The platform team that opens the PR gets adoption; the one that files tickets gets a backlog. A codemod plus a batch of PRs turns a quarter-long migration into a week.
Document the escape hatch. Every path meets a workload it does not fit, and undocumented escape destroys your inventory. Make leaving the path require a one-page ADR and a labeled exception with an expiry.
C - Make Review Cheap and Rare
Automate every rule you can state precisely. Required labels, probes, memory limits, and digest pinning are machine checks. Human review time is too expensive to spend on things a policy engine decides better.
Keep the human checklist to five items. A 40-item checklist gets rubber-stamped; five judgment calls get read. Resource justification, probe semantics, blast radius, template adherence, rollout strategy - that is the list.
Run the same policies in CI and admission. Divergence means green PRs that fail at deploy, which trains teams to distrust CI. kubectl apply --dry-run=server against the real webhooks keeps them honest.
Review the values file, not the rendered output. If your chart generates the manifest, reviewing generated YAML wastes the reviewer's time and insults the template. Review the input humans actually wrote.
Demand a source for every resource number. "Where did 512Mi come from?" catches more real problems than any policy. An observed percentile or a load-test link is an answer; "it's what the last service used" is not.
D - Mentor Platform Engineers Deliberately
Teach the control loop, not the kubectl command. An engineer who understands that controllers reconcile desired state against observed state can debug an add-on they have never seen. One who memorized commands is stuck the moment the output looks unfamiliar.
Rotate the pager and review the postmortems together. On-call is the highest-bandwidth teaching there is, and a blameless postmortem read as a group turns one person's bad night into the team's knowledge.
Pair on the upgrade, do not do it for them. Kubernetes minor upgrades are the platform team's core skill and the scariest thing to hand over. Drive once, navigate twice, then hand over the keys.
Give away the work you are best at. The thing you are fastest at is the thing your team cannot do without you, which makes it your bus factor. Delegating your strongest skill is uncomfortable and correct.
Insist that people write the ADR. Writing forces the decision to be understood, and a one-page ADR that a junior engineer wrote and a senior reviewed teaches more than an hour of whiteboarding.
E - Decide Add-Ons With Written Trade-Offs
Put the FTE cost in every build-vs-buy comparison. Self-hosted is not free; it costs roughly 0.25-0.5 engineer-years per add-on in upgrades alone. A table without that column is an argument, not an analysis.
Ask the bus-factor question out loud. If the person who installed this leaves next month, does it keep working? A "no" is a decision, not a risk to note.
Write the exit cost into the ADR. Add-ons that touch every app manifest are cheap to install and brutal to remove. Prefer standards at the app-facing boundary - HTTPRoute, ServiceMonitor, OTLP - so the next decision stays makeable.
Give every ADR a review date. The ecosystem moves fast enough that a two-year-old choice was made against a different landscape. "We decided this in 2024" is a date, not a reason.
Cap the number of supported add-ons. A hard limit forces the conversation about what leaves when something new arrives. Every add-on you accept joins your Kubernetes upgrade critical path.
When You Are Done
You will know the practices landed when a new service reaches production without the platform team touching it.
The second signal is quieter: teams stop asking permission and start reading the template.
If you adopt nothing else, adopt the metric. A platform team that measures time-to-safe-deploy will find the rest of this list on its own.
FAQs
What is a good time-to-safe-deploy? Under a day for a new service on the golden path, and under an hour for a change to an existing one. The absolute number matters far less than the trend.
Should the platform team have its own on-call? Yes, for the platform's own SLOs - control plane, ingress, CI. Taking app teams' pages makes you their operations department and kills your roadmap.
How do we say no without becoming gatekeepers? Say no to the request and yes to the requirement. "Not that annotation, but here is the template value that does it" is a different conversation than "denied."
How big should a platform team be? Three engineers is roughly the floor for self-hosting a meaningful add-on set. Below that, buy managed and spend your people on the golden path.
Do we need a formal ADR process? You need the artifact, not the ceremony. One markdown file per decision in the platform repo, reviewed by PR, is enough.
How do we handle a team that ignores the standard? With a schedule and data, not escalation theater. "This chart version fails admission on 2026-09-01, and you have 14 workloads on it" resolves most of these.
Related
- Leading a Platform, Not Just Running It
- Tech Lead Basics on Platforms
- Golden Path Governance
- Manifest Review Standards
- Build vs Buy Add-Ons
Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).