Autoscaling Best Practices

Autoscaling only works when the pod, metric, and node layers agree. These practices keep the three axes cooperating instead of fighting.

They target platform engineers and tech leads running HPA, VPA, Cluster Autoscaler or Karpenter, and KEDA in production.

How to Use This List

Treat each group as a checkpoint, not a menu. Skipping the foundations group breaks everything above it.

Adopt them in order: get requests and metrics right first, then tune the HPA, then wire in node and event scaling.

Re-read the scale-down and safety group before you ship, because that is where autoscaling causes outages.

Set resource requests on every container. The HPA measures usage as a percentage of the CPU request, and the scheduler and Cluster Autoscaler both reason from requests. No request means no reliable scaling.

Install and verify the metrics pipeline. Confirm kubectl top pods returns data before creating any HPA. A <unknown> metric almost always traces back to a missing metrics-server or absent request.

Right-size requests before automating. Use VPA in updateMode: "Off" or usage history to set honest requests. Inflated requests waste nodes; starved requests trigger OOM kills and false scaling.

Keep limits sane. Set memory limits to prevent a leak from taking a node down, and be cautious with CPU limits since aggressive limits throttle pods and distort the signal the HPA reads.

B - Choosing the Right Axis

Scale stateless throughput horizontally. The HPA on CPU or a custom metric is the default for services that parallelize across replicas.

Scale unsplittable workloads vertically. Use the VPA for single-threaded processes or fixed-topology apps where more pods do not help.

Never point HPA and VPA at the same metric. They oscillate. Let the HPA own CPU or a custom metric and the VPA own memory, or keep the VPA in recommendation mode.

Use KEDA for event and queue signals. When load is driven by queue depth, cron, or an external metric - or when idling to zero matters - KEDA is the right tool rather than a hand-rolled adapter.

C - Tuning the HPA

Scale up fast, scale down slow. Use a zero-second scale-up stabilization window and a longer scale-down window so you add capacity quickly but remove it cautiously.

Pick a metric that reflects real load. CPU lags for I/O-bound services; requests-per-second or queue depth track demand more truthfully.

Set min and max replicas deliberately. The minimum protects baseline availability; the maximum caps blast radius and cost. Do not leave the maximum at a value your nodes cannot back.

List multiple metrics when safety demands it. The HPA takes the highest desired count across metrics, so the most-stressed dimension wins.

D - Node Scaling and Cost

Back the HPA with node autoscaling. Replicas that land as Pending help no one. The Cluster Autoscaler or Karpenter must supply capacity behind pod scaling.

Budget for provisioning latency. New nodes take minutes to join, so keep headroom or run low-priority over-provisioning pods for latency-sensitive spikes.

Prefer Karpenter for packing on AWS. Just-in-time, right-sized nodes and consolidation usually beat fixed node groups on both cost and bin-packing.

Watch for scale-down blockers. Pods with local storage, no controller, or safe-to-evict: "false" pin nodes and quietly inflate your bill.

E - Scale-Down Safety and Validation

Define PodDisruptionBudgets. A PDB stops node drains and consolidation from evicting too many replicas at once and starving a service.

Test scale-down, not just scale-up. Most incidents come from removing capacity too aggressively. Drive load down in staging and watch what gets evicted.

Guard scale-to-zero with activation thresholds. With KEDA, separate the zero-to-one activation from ongoing scaling so a trickle of events does not thrash a cold workload.

Protect single-replica workloads. VPA Auto mode and node drains both recreate pods. Give critical services a second replica or use non-disruptive modes.

F - Observability

Watch HPA events and decisions. kubectl describe hpa explains every scale action and surfaces <unknown> metrics before they page you.

Alert on saturation and pending pods. Sustained max replicas or lingering Pending pods mean the current limits or node capacity are wrong.

Track cost against utilization. Autoscaling that never scales down, or nodes that never pack, show up as flat spend under variable load.

When You Are Done

You should have accurate requests, a healthy metrics pipeline, an HPA with tuned behavior, node autoscaling behind it, and PDBs guarding scale-down.

Run a load test that spikes and then drains, and confirm the system grows, places pods on new nodes, and consolidates back down without dropping traffic.

If scale-down never happens or pods sit Pending, revisit groups A, D, and E before adding more metrics.

FAQs

What is the single most common autoscaling mistake? Missing resource requests. Without them the HPA cannot compute utilization and the Cluster Autoscaler cannot bin-pack.

Can I use HPA and VPA on the same workload? Yes, but not on the same metric. Split CPU to the HPA and memory to the VPA, or keep the VPA in Off mode as an advisor.

Why does my cluster never scale down? Look for scale-down blockers: strict PDBs, pods with local storage, unmanaged pods, or over-inflated requests keeping utilization artificially high.

Do I still need the Cluster Autoscaler if I have the HPA? Yes. The HPA adds replicas but cannot create nodes. Without node scaling it stalls the moment the cluster fills.

How do I scale to zero? Use KEDA with a minReplicaCount of zero and an activation threshold. The stock HPA does not idle to zero on its own.

How fast can autoscaling react? Pod scaling is seconds once capacity exists; node scaling is minutes. Design headroom around the slower node axis.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

Autoscaling Best Practices

Autoscaling only works when the pod, metric, and node layers agree. These practices keep the three axes cooperating instead of fighting.

They target platform engineers and tech leads running HPA, VPA, Cluster Autoscaler or Karpenter, and KEDA in production.

How to Use This List

Treat each group as a checkpoint, not a menu. Skipping the foundations group breaks everything above it.

Adopt them in order: get requests and metrics right first, then tune the HPA, then wire in node and event scaling.

Re-read the scale-down and safety group before you ship, because that is where autoscaling causes outages.

A - Foundations

Right-size requests before automating. Use VPA in updateMode: "Off" or usage history to set honest requests. Inflated requests waste nodes; starved requests trigger OOM kills and false scaling.

Keep limits sane. Set memory limits to prevent a leak from taking a node down, and be cautious with CPU limits since aggressive limits throttle pods and distort the signal the HPA reads.

B - Choosing the Right Axis

Scale stateless throughput horizontally. The HPA on CPU or a custom metric is the default for services that parallelize across replicas.

Scale unsplittable workloads vertically. Use the VPA for single-threaded processes or fixed-topology apps where more pods do not help.

Never point HPA and VPA at the same metric. They oscillate. Let the HPA own CPU or a custom metric and the VPA own memory, or keep the VPA in recommendation mode.

Use KEDA for event and queue signals. When load is driven by queue depth, cron, or an external metric - or when idling to zero matters - KEDA is the right tool rather than a hand-rolled adapter.

C - Tuning the HPA

Scale up fast, scale down slow. Use a zero-second scale-up stabilization window and a longer scale-down window so you add capacity quickly but remove it cautiously.

Pick a metric that reflects real load. CPU lags for I/O-bound services; requests-per-second or queue depth track demand more truthfully.

Set min and max replicas deliberately. The minimum protects baseline availability; the maximum caps blast radius and cost. Do not leave the maximum at a value your nodes cannot back.

List multiple metrics when safety demands it. The HPA takes the highest desired count across metrics, so the most-stressed dimension wins.

D - Node Scaling and Cost

Back the HPA with node autoscaling. Replicas that land as Pending help no one. The Cluster Autoscaler or Karpenter must supply capacity behind pod scaling.

Budget for provisioning latency. New nodes take minutes to join, so keep headroom or run low-priority over-provisioning pods for latency-sensitive spikes.

Prefer Karpenter for packing on AWS. Just-in-time, right-sized nodes and consolidation usually beat fixed node groups on both cost and bin-packing.

Watch for scale-down blockers. Pods with local storage, no controller, or safe-to-evict: "false" pin nodes and quietly inflate your bill.

E - Scale-Down Safety and Validation

Define PodDisruptionBudgets. A PDB stops node drains and consolidation from evicting too many replicas at once and starving a service.

Test scale-down, not just scale-up. Most incidents come from removing capacity too aggressively. Drive load down in staging and watch what gets evicted.

Guard scale-to-zero with activation thresholds. With KEDA, separate the zero-to-one activation from ongoing scaling so a trickle of events does not thrash a cold workload.

Protect single-replica workloads. VPA Auto mode and node drains both recreate pods. Give critical services a second replica or use non-disruptive modes.

F - Observability

Watch HPA events and decisions. kubectl describe hpa explains every scale action and surfaces <unknown> metrics before they page you.

Alert on saturation and pending pods. Sustained max replicas or lingering Pending pods mean the current limits or node capacity are wrong.

Track cost against utilization. Autoscaling that never scales down, or nodes that never pack, show up as flat spend under variable load.

When You Are Done

You should have accurate requests, a healthy metrics pipeline, an HPA with tuned behavior, node autoscaling behind it, and PDBs guarding scale-down.

Run a load test that spikes and then drains, and confirm the system grows, places pods on new nodes, and consolidates back down without dropping traffic.

If scale-down never happens or pods sit Pending, revisit groups A, D, and E before adding more metrics.

FAQs

What is the single most common autoscaling mistake? Missing resource requests. Without them the HPA cannot compute utilization and the Cluster Autoscaler cannot bin-pack.

Can I use HPA and VPA on the same workload? Yes, but not on the same metric. Split CPU to the HPA and memory to the VPA, or keep the VPA in Off mode as an advisor.

Why does my cluster never scale down? Look for scale-down blockers: strict PDBs, pods with local storage, unmanaged pods, or over-inflated requests keeping utilization artificially high.

Do I still need the Cluster Autoscaler if I have the HPA? Yes. The HPA adds replicas but cannot create nodes. Without node scaling it stalls the moment the cluster fills.

How do I scale to zero? Use KEDA with a minReplicaCount of zero and an activation threshold. The stock HPA does not idle to zero on its own.

How fast can autoscaling react? Pod scaling is seconds once capacity exists; node scaling is minutes. Design headroom around the slower node axis.

Stack versions: This page was written for Kubernetes 1.36.2, Docker Engine 29.6.1 (BuildKit default), containerd (CRI runtime on nodes), Helm 3, Compose v2, Argo CD (latest - verify at build), and Gateway API (GA - verify controller support at build).

Autoscaling Best Practices

Search Documentation

Autoscaling Best Practices