Right-Sizing Requests & Limits

Summary

Right-sizing means setting each container's CPU and memory requests close to its real measured usage instead of a padded guess.
Insight: Over-requesting is usually the single largest source of Kubernetes waste, because inflated requests force extra nodes into existence.
Key Concepts: request (reservation), limit (cap), p95/p99 usage, Vertical Pod Autoscaler (VPA), and CPU throttling versus OOM kill.
When to Use: Do this before enabling node autoscaling or spot, so autoscalers scale against honest numbers.
Limitations/Trade-offs: Requests set too low cause scheduling pressure and OOM kills, so aim for measured p95 plus a small buffer, not the floor.
Related Topics: Cost basics, Karpenter consolidation, and cost visibility tooling that surfaces over-provisioned workloads.

Foundations

A request is a promise the scheduler keeps. When a pod requests 1 CPU, the scheduler reserves a full core on some node and never hands that core to anyone else.

That reservation holds even if the pod idles at two percent CPU. You pay for the reserved core, not the used one.

Multiply this across hundreds of pods and the effect is stark. Padded requests inflate the reserved total, the reserved total drives bin-packing, and bin-packing drives node count.

The result is a cluster that looks 90 percent full to the scheduler while burning 25 percent of real capacity.

Right-sizing closes that gap. You measure what a workload actually uses, set the request to match, and let the scheduler pack tighter onto fewer nodes.

The default failure mode is copy-paste. A team pastes cpu: 1, memory: 2Gi from a template, ships it everywhere, and never revisits it.

Mechanics & Interactions

CPU and memory behave differently, and that difference drives how you set requests versus limits.

CPU is compressible. When a container hits its CPU limit, the kernel throttles it - the container slows down but keeps running.

Memory is not compressible. When a container exceeds its memory limit, the kernel OOM-kills it, and the pod restarts.

This asymmetry gives a practical rule. Set CPU requests from real usage and usually leave CPU without a hard limit, so pods can burst into idle headroom.

Set memory requests from real usage and set the memory limit equal to the request, so a pod cannot balloon and destabilize its node.

To pick the numbers, measure over a representative window that includes peaks.

kubectl top pod -A --sort-by=cpu

For durable analysis, query Prometheus for the p95 of container_cpu_usage_seconds_total and the max of container_memory_working_set_bytes over one to two weeks.

Set the request to roughly p95 usage plus a modest buffer. This covers normal peaks without reserving for rare spikes.

The Vertical Pod Autoscaler automates this loop. Its recommender watches actual usage and emits suggested requests.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout
  updatePolicy:
    updateMode: "Off"

updateMode: "Off" runs the recommender in observe-only mode, so you get numbers without VPA restarting pods.

Once you trust the recommendations, updateMode: "Auto" lets VPA apply them, though applying new requests evicts and recreates pods.

Advanced Considerations & Applications

VPA and Horizontal Pod Autoscaler can conflict when both act on CPU or memory, because HPA scales replica count off the same signal VPA is adjusting.

The common pattern is HPA on CPU for scaling out, and VPA on memory only, or VPA in recommendation mode feeding a periodic manual update.

Guardrails matter at scale. A LimitRange sets default requests and limits for pods that omit them, so a forgotten field does not silently reserve zero or inherit a huge default.

A ResourceQuota caps a namespace's total reserved capacity, which turns right-sizing into an enforced budget rather than a suggestion.

Quality of Service class is a side effect worth knowing. A pod with requests equal to limits on both CPU and memory is Guaranteed and is evicted last under node pressure.

A pod with requests below limits is Burstable, and a pod with no requests or limits is BestEffort and is killed first. Critical workloads usually want Guaranteed.

The payoff compounds. Tighter requests raise real utilization, which lets node consolidation remove empty nodes, which is exactly what Karpenter acts on.

Common Misconceptions

"Bigger requests make my app faster." Requests reserve capacity; they do not grant more CPU cycles at runtime. A pod can burst above its CPU request into idle headroom regardless.

"I should set requests to the peak spike." Sizing to a rare spike reserves capacity you almost never use. Size to p95 plus a buffer and let short bursts share headroom.

"Setting a CPU limit protects my app." A CPU limit throttles your own container. Unless you have a specific noisy-neighbor problem, leaving CPU unlimited but requested is often better.

"Requests and limits should always be equal." For memory, equal is a good default. For CPU, equal often causes needless throttling and wastes burst headroom.

"VPA and HPA work together automatically." They can fight over the same metric. Split their responsibilities or run VPA in recommendation mode.

FAQs

What usage percentile should I target? Roughly p95 over a window that includes real peaks, plus a small safety buffer. This avoids both waste and frequent eviction.

Should I ever set a CPU limit? Only when you need hard isolation, such as untrusted tenants or benchmarking. Otherwise requested-but-unlimited CPU uses idle headroom efficiently.

Why did my pod get OOM-killed after right-sizing? The memory request or limit was set below true peak working set. Raise memory to the observed maximum, not the average.

Can VPA and HPA both run on a Deployment? Yes, if they act on different signals. A common split is HPA on CPU and VPA on memory, or VPA in recommend-only mode.

How often should requests be revisited? Whenever traffic patterns shift, or continuously via VPA recommendations surfaced in a cost dashboard.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

Right-Sizing Requests & Limits

Summary

Right-sizing means setting each container's CPU and memory requests close to its real measured usage instead of a padded guess.
Insight: Over-requesting is usually the single largest source of Kubernetes waste, because inflated requests force extra nodes into existence.
Key Concepts: request (reservation), limit (cap), p95/p99 usage, Vertical Pod Autoscaler (VPA), and CPU throttling versus OOM kill.
When to Use: Do this before enabling node autoscaling or spot, so autoscalers scale against honest numbers.
Limitations/Trade-offs: Requests set too low cause scheduling pressure and OOM kills, so aim for measured p95 plus a small buffer, not the floor.
Related Topics: Cost basics, Karpenter consolidation, and cost visibility tooling that surfaces over-provisioned workloads.

Foundations

A request is a promise the scheduler keeps. When a pod requests 1 CPU, the scheduler reserves a full core on some node and never hands that core to anyone else.

That reservation holds even if the pod idles at two percent CPU. You pay for the reserved core, not the used one.

Multiply this across hundreds of pods and the effect is stark. Padded requests inflate the reserved total, the reserved total drives bin-packing, and bin-packing drives node count.

The result is a cluster that looks 90 percent full to the scheduler while burning 25 percent of real capacity.

Right-sizing closes that gap. You measure what a workload actually uses, set the request to match, and let the scheduler pack tighter onto fewer nodes.

The default failure mode is copy-paste. A team pastes cpu: 1, memory: 2Gi from a template, ships it everywhere, and never revisits it.

Mechanics & Interactions

CPU and memory behave differently, and that difference drives how you set requests versus limits.

CPU is compressible. When a container hits its CPU limit, the kernel throttles it - the container slows down but keeps running.

Memory is not compressible. When a container exceeds its memory limit, the kernel OOM-kills it, and the pod restarts.

This asymmetry gives a practical rule. Set CPU requests from real usage and usually leave CPU without a hard limit, so pods can burst into idle headroom.

Set memory requests from real usage and set the memory limit equal to the request, so a pod cannot balloon and destabilize its node.

To pick the numbers, measure over a representative window that includes peaks.

kubectl top pod -A --sort-by=cpu

For durable analysis, query Prometheus for the p95 of container_cpu_usage_seconds_total and the max of container_memory_working_set_bytes over one to two weeks.

Set the request to roughly p95 usage plus a modest buffer. This covers normal peaks without reserving for rare spikes.

The Vertical Pod Autoscaler automates this loop. Its recommender watches actual usage and emits suggested requests.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout
  updatePolicy:
    updateMode: "Off"

updateMode: "Off" runs the recommender in observe-only mode, so you get numbers without VPA restarting pods.

Once you trust the recommendations, updateMode: "Auto" lets VPA apply them, though applying new requests evicts and recreates pods.

Advanced Considerations & Applications

VPA and Horizontal Pod Autoscaler can conflict when both act on CPU or memory, because HPA scales replica count off the same signal VPA is adjusting.

The common pattern is HPA on CPU for scaling out, and VPA on memory only, or VPA in recommendation mode feeding a periodic manual update.

Guardrails matter at scale. A LimitRange sets default requests and limits for pods that omit them, so a forgotten field does not silently reserve zero or inherit a huge default.

A ResourceQuota caps a namespace's total reserved capacity, which turns right-sizing into an enforced budget rather than a suggestion.

Quality of Service class is a side effect worth knowing. A pod with requests equal to limits on both CPU and memory is Guaranteed and is evicted last under node pressure.

A pod with requests below limits is Burstable, and a pod with no requests or limits is BestEffort and is killed first. Critical workloads usually want Guaranteed.

The payoff compounds. Tighter requests raise real utilization, which lets node consolidation remove empty nodes, which is exactly what Karpenter acts on.

Common Misconceptions

"Bigger requests make my app faster." Requests reserve capacity; they do not grant more CPU cycles at runtime. A pod can burst above its CPU request into idle headroom regardless.

"I should set requests to the peak spike." Sizing to a rare spike reserves capacity you almost never use. Size to p95 plus a buffer and let short bursts share headroom.

"Setting a CPU limit protects my app." A CPU limit throttles your own container. Unless you have a specific noisy-neighbor problem, leaving CPU unlimited but requested is often better.

"Requests and limits should always be equal." For memory, equal is a good default. For CPU, equal often causes needless throttling and wastes burst headroom.

"VPA and HPA work together automatically." They can fight over the same metric. Split their responsibilities or run VPA in recommendation mode.

FAQs

What usage percentile should I target? Roughly p95 over a window that includes real peaks, plus a small safety buffer. This avoids both waste and frequent eviction.

Should I ever set a CPU limit? Only when you need hard isolation, such as untrusted tenants or benchmarking. Otherwise requested-but-unlimited CPU uses idle headroom efficiently.

Why did my pod get OOM-killed after right-sizing? The memory request or limit was set below true peak working set. Raise memory to the observed maximum, not the average.

Can VPA and HPA both run on a Deployment? Yes, if they act on different signals. A common split is HPA on CPU and VPA on memory, or VPA in recommend-only mode.

How often should requests be revisited? Whenever traffic patterns shift, or continuously via VPA recommendations surfaced in a cost dashboard.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

Right-Sizing Requests & Limits

Search Documentation

Right-Sizing Requests & Limits