Platform Delivery Basics

This section covers how platform changes differ from app deploys, and the basic building blocks - SLOs, error budgets, change windows, and GitOps - that keep cluster changes from breaking the teams on top.

Prerequisites

• Kubernetes 1.36.2 cluster access with kubectl configured (containerd is the node runtime via CRI).
• Helm 3 and Kustomize for packaging platform components.
• Argo CD or Flux for GitOps delivery of cluster config.
• Prometheus and Grafana for measuring platform SLIs.
• kubectl install on Debian/Ubuntu:

# install kubectl matching the cluster minor
curl -LO "https://dl.k8s.io/release/v1.36.2/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client

Basic Examples

1. App deploy cadence vs platform change cadence

App teams deploy many times a day; platform changes are rarer and higher blast radius.

# an app deploy: rolls one Deployment, affects one team
kubectl set image deployment/checkout web=registry.example.com/checkout:1.8.3
 
# a platform change: rotates node pool, affects every pod on those nodes
kubectl drain node-pool-a-xyz --ignore-daemonsets --delete-emptydir-data

• App deploys are frequent, small, and scoped to one workload.
• Platform changes are infrequent but touch shared surfaces every team depends on.
• Blast radius, not frequency, is what makes platform delivery need change windows.
• Keep the two cadences on separate pipelines so an app deploy never waits on a cluster upgrade.

2. A platform SLI you can measure today

Start with API server request latency, a signal every team feels.

# fraction of API reads served under 1s over 30 days
sum(rate(apiserver_request_duration_seconds_bucket{verb="GET",le="1"}[30d]))
/
sum(rate(apiserver_request_duration_seconds_count{verb="GET"}[30d]))

• apiserver_request_duration_seconds is exported by the API server itself.
• A bucketed histogram lets you compute the fraction under a latency threshold.
• Compare the result to your SLO target, for example 0.999.
• This ratio is the SLI; the target you promise on it is the SLO.

3. Declare an SLO for the platform

Write the objective down so it is a contract, not a vibe.

# a plain-text SLO record kept in the platform repo
slo:
  name: apiserver-read-availability
  sli: fraction of GET requests < 1s
  target: 0.999        # 99.9% over a 30-day rolling window
  window: 30d
  owner: platform-team

• Every SLO needs an SLI, a target, a window, and an owner.
• A 30-day rolling window smooths out single bad days.
• 99.9% over 30 days is roughly 43 minutes of error budget.
• Store it in Git next to the cluster config it governs.

4. GitOps for cluster config

Deliver platform changes declaratively so they are reviewable and revertible.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: platform-baseline
  namespace: argocd
spec:
  project: platform
  source:
    repoURL: https://github.com/example/platform-config.git
    targetRevision: main
    path: baseline
  destination:
    server: https://kubernetes.default.svc
    namespace: platform
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

• Cluster config lives in Git, and Argo CD reconciles the live state to it.
• selfHeal reverts drift; prune removes objects deleted from Git.
• A change is a pull request, so it gets review and an audit trail.
• Rollback is a git revert, not a frantic manual edit.

5. Requests, limits, and probes on a platform component

Platform add-ons need the same resource hygiene you expect from apps.

resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    memory: 256Mi
readinessProbe:
  httpGet:
    path: /healthz
    port: 8080
  periodSeconds: 10

• Requests drive scheduling; limits cap runaway usage.
• Set a memory limit, but avoid a hard CPU limit on latency-sensitive components to prevent throttling.
• A readiness probe keeps traffic off a pod until it is actually ready.
• Right-sized requests keep scheduling latency inside its SLO.

6. A minimal change-window policy

Decide when disruptive platform work is allowed before you need it.

change-window:
  routine:    "Tue-Thu 10:00-16:00 UTC"   # low-risk, business hours
  freeze:     "Fri 12:00 - Mon 09:00 UTC" # no risky platform changes
  emergency:  "any time, with incident approval"

• Routine windows favor daytime hours when the team is watching.
• A weekend freeze protects the period with the thinnest on-call coverage.
• Emergencies always override, but require explicit approval.
• Publishing the policy lets app teams plan their launches around it.

7. Track DORA basics

Measure delivery health so you know if the platform is helping.

# deployment frequency: successful app rollouts per day
sum(increase(deployment_rollout_success_total[1d]))

• The four DORA metrics are deployment frequency, lead time, change failure rate, and time to restore.
• A good platform raises frequency and lowers change failure rate.
• Track them per team to spot who the paved road is failing.
• Trends matter more than any single day's number.

Intermediate Examples

8. Gate risky sync on the error budget

Let policy, not gut feel, decide whether a change ships.

# pseudo-gate in a pipeline: block sync if budget is nearly gone
BUDGET_REMAINING=$(curl -s "$PROM/api/v1/query?query=slo_error_budget_remaining_ratio" \
  | jq -r '.data.result[0].value[1]')
 
if (( $(echo "$BUDGET_REMAINING < 0.10" | bc -l) )); then
  echo "Error budget below 10% - freezing platform changes"
  exit 1
fi
argocd app sync platform-baseline

• The gate reads the remaining error budget from Prometheus.
• Below a threshold, the pipeline refuses to apply risky changes.
• This turns the freeze policy into automation instead of a Slack message.
• Emergency fixes bypass the gate through a separate, approved path.

9. Safe control-plane upgrade, one minor at a time

Never skip minor versions; Kubernetes supports upgrading one minor at a time.

# check current version, then upgrade the control plane one minor up
kubectl version
# on a kubeadm control-plane node:
sudo kubeadm upgrade plan
sudo kubeadm upgrade apply v1.36.2
# then drain, upgrade kubelet, and uncordon each node in turn
kubectl drain node-1 --ignore-daemonsets

• Upgrade the control plane first, then the nodes.
• Move one minor at a time; skipping minors is unsupported.
• Drain each node so pods reschedule before its kubelet restarts.
• Do upgrades inside a change window, watching the error budget as you go.

10. Protect the shared cluster from a noisy tenant

Quotas and priority keep one team from burning everyone's budget.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: team-checkout
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    pods: "150"

• A ResourceQuota caps how much of the shared cluster one namespace can claim.
• This protects scheduling latency and capacity for every other tenant.
• Pair it with PriorityClass so platform-critical pods win under pressure.
• Multi-tenant safety is a platform SLO concern, not an afterthought.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).