Admission Best Practices
Admission control is powerful enough to protect a cluster and powerful enough to break it. These practices keep your policies safe to roll out, easy to maintain, and versioned alongside the clusters they run on.
How to Use This List
Read each group in order; the early groups reduce blast radius, the later ones cover authoring and operations.
Treat every practice as a default you deviate from only with a reason. Apply them incrementally, starting with the safety-scoping group before you enforce anything.
A - Roll Out Safely
- Start in audit or warn, never deny. Ship every new rule with Kyverno's
Audit, Gatekeeper'sdryrun, or a CEL binding'sWarn/Auditaction first, and only promote to enforcement once reports are clean. - Measure impact against real objects. Use background scans (Kyverno
PolicyReport, Gatekeeperstatus.violations) to see how many existing workloads would fail before you block anything. - Promote per environment. Enforce strictly in production namespaces while auditing in sandbox, using namespace-scoped action overrides rather than separate policies.
- Roll out one rule at a time. Bundling ten new rules into one enforcing change makes a rejection hard to diagnose; land them individually.
B - Protect the Cluster From Your Policies
- Exclude control-plane namespaces. Always add
kube-system,kube-node-lease, and the policy engine's own namespace toexcludedNamespacesor anamespaceSelectorso a policy outage cannot wedge the control plane. - Choose failurePolicy deliberately per rule.
Failis stronger for security but blocks writes when the webhook is down;Ignorefavors availability. Decide based on what the rule protects. - Keep timeouts short. Set webhook
timeoutSecondsto 5-10 so a slow endpoint does not stall every write; admission runs synchronously in the request path. - Run the engine with headroom. Give Kyverno or Gatekeeper multiple replicas plus requests and limits, because the engine's availability is now part of your write path.
- Scope webhooks tightly. Match only the
apiGroups,resources, andoperationsyou need, and useobjectSelectorto skip objects that should never be gated.
C - Author Clear, Testable Policies
- Prefer the simplest engine for the rule. Use built-in Pod Security Admission or a CEL
ValidatingAdmissionPolicyfor simple checks, and reserve Kyverno or Gatekeeper for logic they cannot express. - Write actionable messages. Every deny message should name the offending field and the fix, so a developer resolves it without reading the policy source.
- Keep mutations minimal and idempotent. Mutating rules run before validation and can conflict; set
reinvocationPolicy: IfNeededwhen a rule must see the final object. - Reuse community libraries. Adopt vetted Kyverno policies or Gatekeeper templates for common controls (no
:latest, required probes, allowed registries) instead of hand-writing them.
D - Test Before the Cluster Sees It
- Gate pull requests on rendered manifests. Run
conftest,kyverno test, orgator testagainst the output ofhelm templateandkubectl kustomize, not the source templates. - Add a server-side dry-run stage. Validate against a staging cluster with production policy configuration using
kubectl apply --dry-run=serverfor exact parity. - Unit test the policies themselves. Use
conftest verifyand engine test files so a broken policy that silently allows everything is caught in CI. - Keep CI rules and cluster rules in sync. Share policy source or automate the mirror, because drift between them is the main way policy testing gives false confidence.
E - Operate and Version as Code
- Store policies in Git and deploy via GitOps. Manage policies as Kubernetes manifests through Argo CD or Flux so changes are reviewed, auditable, and reversible.
- Version policies with the cluster. Pin and bump policies alongside Kubernetes minor upgrades, since API shapes, CEL features, and Pod Security defaults change across releases.
- Monitor rejections and violations. Alert on admission denials and rising violation counts; a spike often signals a broken policy or a bad deploy, not just user error.
- Review and prune regularly. Remove policies that no longer apply and reconcile audit reports, so the rule set stays understandable and fast.
When You Are Done
You should have a policy set that begins in audit, excludes control-plane namespaces, and enforces only after reports are clean.
Your rules should live in Git, deploy through GitOps, run in CI against rendered manifests, and carry a version that moves with each cluster upgrade.
If a policy engine outage cannot block your control plane and a developer can read any rejection message and fix it unaided, the guardrails are production-ready.
FAQs
Should every policy start in audit mode? Yes. Enforcing a new rule cold risks blocking legitimate workloads; audit first, review reports, then promote to deny.
How do I keep policies from breaking on upgrades? Version them with the cluster and test against each new Kubernetes minor, because Pod Security defaults, CEL, and API fields evolve.
Which engine should I standardize on? Use the simplest that expresses your rules: Pod Security Admission or CEL for basics, Kyverno for YAML-native teams, Gatekeeper for a central Rego library.
What is the single most important safety setting? Excluding control-plane and engine namespaces from enforcing webhooks, so an engine outage cannot lock you out.
Where should policies live? In Git, deployed via Argo CD or Flux, so every change is reviewed and reversible like any other manifest.
How do I stop CI from drifting from the cluster? Share the same policy source or add a server-side dry-run gate against a cluster that mirrors production policy configuration.
Related
- Where Kubernetes Enforces Your Rules
- Admission Basics
- Kyverno Policies
- OPA Gatekeeper
- Policy Testing
Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).