SproutStackSproutStack home
IntermediateKubernetes

The autoscaler never scales

A HorizontalPodAutoscaler targets CPU, but the pods have no resource requests.

The setup today

A Node.js API runs as 4 pods with a HorizontalPodAutoscaler targeting 70% CPU utilization. The deployment was copied from an example and sets no resource requests or limits.

Current configuration

HPA: minReplicas 4, maxReplicas 40, target CPU utilization 70%
containers: resources: {}   # no requests, no limits
nodes: 8 vCPU, 32 GB each

What you know

  • During peaks, latency triples but the HPA stays at 4 replicas.
  • Pods are occasionally killed when a node runs low on memory, and some land on nodes that are already full.

The new requirement

Peaks are getting bigger. The API must scale out automatically and stop being killed unpredictably.

Make the call

2 questions · first-try answers earn the most XP

1. Why does the autoscaler not scale?
2. What should the resources look like?
What a strong answer also covers
  • CPU is a poor scaling signal for I/O-bound services; scaling on requests per second or queue depth may track load better.
  • Configure HPA scale-down stabilization so it does not flap after a short peak.
  • The cluster autoscaler must be able to add nodes, or new pods sit Pending.
  • Node.js memory: set the heap size flag below the container limit, or the process ignores the limit until it is killed.
Variations to think through

Change one fact and decide again. There is no single right answer here.

  • Traffic jumps tenfold within a minute at 9 a.m. Can a CPU-based HPA react fast enough?
  • Each pod needs 60 seconds to warm up. What does that mean for autoscaling during a spike?
  • The service is mostly waiting on other services. Which metric would you scale on instead?

The background

More drills