Liveness, Readiness & Startup Probes
Summary
- Kubernetes offers three probes with three distinct jobs: startup (is it done booting?), readiness (can it take traffic now?), and liveness (is it hopelessly stuck?).
- Insight: The single most common outage-amplifier is a liveness probe that checks dependencies. It restarts healthy Pods during a downstream blip and turns a brownout into a crash loop.
- Key Concepts:
httpGet/tcpSocket/exec/grpchandlers,initialDelaySeconds,periodSeconds,failureThreshold, endpoint removal vs restart. - When to Use: readiness on every serving Pod; startup for slow boots; liveness only for genuine deadlocks.
- Limitations: probes measure a signal you define. A shallow probe can report healthy while the app is broken, and an over-eager probe can cause the failures it is meant to catch.
Recipe
Add a readiness probe that reflects real ability to serve, a startup probe for slow starters, and a shallow liveness probe that never touches dependencies.
readinessProbe:
httpGet: { path: /readyz, port: 8080 }
periodSeconds: 5
failureThreshold: 3
startupProbe:
httpGet: { path: /healthz, port: 8080 }
periodSeconds: 5
failureThreshold: 30 # allow up to ~150s to boot
livenessProbe:
httpGet: { path: /healthz, port: 8080 }
periodSeconds: 10
failureThreshold: 3Working Example
A Deployment with all three probes wired to separate endpoints, showing how they cooperate.
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders
spec:
replicas: 3
selector:
matchLabels: { app: orders }
template:
metadata:
labels: { app: orders }
spec:
containers:
- name: orders
image: registry.example.com/orders@sha256:7de1...
ports:
- name: http
containerPort: 8080
startupProbe:
httpGet: { path: /healthz, port: http }
periodSeconds: 5
failureThreshold: 30
readinessProbe:
httpGet: { path: /readyz, port: http }
periodSeconds: 5
timeoutSeconds: 2
failureThreshold: 3
livenessProbe:
httpGet: { path: /healthz, port: http }
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3Here /healthz is shallow (process is alive) and /readyz reflects whether the app can serve, including checks like a warmed cache.
Deep Dive
What each probe does on failure
A failing readiness probe removes the Pod from Service endpoints. Traffic stops flowing to it, but the container keeps running.
A failing liveness probe restarts the container. The kubelet kills and recreates it per the restart policy.
A failing startup probe also restarts the container, but it runs first and suspends the other two until it succeeds once.
Probe handlers
httpGet expects a 2xx or 3xx status. tcpSocket just checks the port accepts a connection. exec runs a command and checks exit code 0. grpc uses the standard gRPC health protocol.
Prefer httpGet for web services because it exercises the request path. Use exec only when nothing else fits, since it is the most expensive.
Timing fields
initialDelaySeconds waits before the first check. With a startup probe you usually set this to 0 and let the startup probe handle slow boots.
periodSeconds is the interval, timeoutSeconds is how long a single check may take, and failureThreshold is the consecutive failures needed to act. successThreshold for readiness can require several passes before flipping back.
Startup vs a long initialDelay
Before startup probes, teams used a large initialDelaySeconds on liveness. That is a blunt instrument: too short and it kills slow boots, too long and it delays detecting real deadlocks.
The startup probe replaces that guess. It gives the app a generous budget to boot (failureThreshold * periodSeconds), then hands off to a tight liveness probe.
Gotchas
Liveness that checks the database. When the database blips, every Pod fails liveness and restarts at once, deepening the outage. Keep liveness shallow and put dependency checks in readiness.
Readiness stays failed forever because it checks a hard dependency at startup. If /readyz requires a downstream that is down, no Pod ever becomes Ready and the rollout stalls. Distinguish "temporarily not ready" from "will never be ready."
Probe timeout too tight under load. A timeoutSeconds: 1 on a GC pause can fail spuriously. Give a realistic budget and account for tail latency.
Same endpoint for liveness and readiness. If they share logic, a dependency failure both drains traffic and restarts the Pod. Separate /healthz and /readyz.
Forgetting a startup probe on slow apps. JVM services with heavy warmup get liveness-killed mid-boot into a crash loop. Add a startup probe with a big enough budget.
Alternatives
No liveness probe at all. Many mature services run with only startup and readiness probes. If your app cannot deadlock in a way a restart fixes, skipping liveness avoids self-inflicted restarts.
Sidecar or mesh health signals. A service mesh can report endpoint health, but in-Pod probes remain the kubelet's source of truth for restarts and endpoint membership.
tcpSocket vs httpGet. Use tcpSocket for non-HTTP protocols. For HTTP services, httpGet is strictly more informative because it exercises the handler.
FAQs
What is the difference between liveness and readiness? Readiness controls traffic (endpoint membership); liveness controls restarts. One drains, the other kills.
Should every container have a liveness probe? No. Add it only when a restart can recover a stuck process. A bad liveness probe causes more outages than it prevents.
Why did my Pod never become Ready? Its readiness probe never passed, often because it checks a dependency that is down, or the endpoint or port is wrong.
Do startup probes replace initialDelaySeconds? Effectively yes for slow boots. The startup probe gates the others, so you avoid guessing a fixed delay.
Can probes reference a named port? Yes. Use port: http if the container declares a port named http, so the number lives in one place.
What handler is cheapest? httpGet and tcpSocket are lightweight. exec forks a process each check and is the most costly, so avoid it in tight periods.
Related
Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).