Autoscaling in Kubernetes is not one feature but a stack of loosely coupled control loops, and most scale-related production incidents trace back to those loops fighting each other rather than to any single one being misconfigured. The Horizontal Pod Autoscaler adjusts replica counts, KEDA translates external events into scaling signals, and Karpenter or the cluster autoscaler turns pending pods into actual nodes. Each layer reasons from different inputs on a different timescale.
Treating them as one system, with honest resource requests as the shared foundation every layer reads, is what separates clusters that absorb traffic spikes quietly from clusters that page someone every time marketing sends an email.
Pod Layer: HPA and Its Blind Spots
The HPA is the workhorse, and CPU utilization is its default diet, which is exactly the problem for latency-sensitive services. CPU is a trailing indicator: by the time utilization crosses the threshold, queues have already formed. Scale on the leading signal instead, requests per second, queue depth, or in-flight connections, exposed through the Prometheus adapter or the custom metrics API. Tune the stabilization windows deliberately: a short scale-up window with a long scale-down window buys responsiveness without flapping.
Two standing caveats. The HPA cannot scale to zero, so idle services keep their floor replicas and their cost. And never let the Vertical Pod Autoscaler and the HPA manage the same resource metric on the same workload, VPA rewriting requests underneath HPA utilization math produces oscillation that looks like load and is not.
Event Layer: KEDA for Queues, Streams, and Schedules
KEDA fills both HPA gaps for asynchronous workloads. Its scalers read the systems that actually hold the pending work, Kafka consumer lag, SQS and RabbitMQ queue depth, Prometheus queries, even cron schedules for predictable daily peaks, and it can scale deployments to zero between bursts, which transforms the economics of spiky batch and consumer workloads. Mechanically, a ScaledObject creates and manages an HPA under the hood, which yields a simple rule: never attach both a ScaledObject and a hand-written HPA to the same workload, because two owners of one replica count is a control-loop fight you will lose at 3 a.m.
Node Layer: Karpenter Versus the Cluster Autoscaler
The cluster autoscaler thinks in node groups: predefined pools that grow and shrink. It is stable and well understood, but every new pod shape means another group to define and balance. Karpenter inverts the model, it reads the pending pods' actual requirements and provisions the best-fitting instances directly from the full menu of types, within the constraints of a NodePool definition. Its consolidation loop then continuously replaces under-utilized or drifted nodes with cheaper arrangements, and it handles spot interruption notices natively. The trade-off is a livelier cluster: nodes come and go more often, which is precisely why disruption budgets and graceful shutdown stop being optional hygiene and become load-bearing configuration.
Making the Layers Cooperate
The classic failure chain: traffic spikes, HPA adds replicas, replicas pend for lack of capacity, node provisioning takes anywhere from tens of seconds to a few minutes, and users eat the gap. Closing that gap is an end-to-end tuning exercise:
1. Set resource requests honestly first. Every layer, HPA math, KEDA thresholds, Karpenter bin packing, reasons from requests, and lies poison all three at once. 2. Put PodDisruptionBudgets on everything that matters; consolidation and spot reclaim both respect them, but only if they exist. 3. Align timescales: HPA scale-down stabilization should be long enough that Karpenter is not consolidating nodes the HPA is about to need back. 4. Reserve headroom for latency-critical services with low-priority placeholder pods that get preempted the moment real work arrives, capacity that is warm but not wasted. 5. Load test the whole chain and measure time from trigger to serving pod, not just node launch time. The end-to-end number is the only one users experience.
Done well, layered autoscaling converts capacity from a standing cost into a variable one without sacrificing the latency budget: the queue drains overnight on spot instances, the API rides its daily curve without a human touching a replica count, and the finance conversation shifts from why is the cluster so big to a curve that tracks revenue-generating traffic. Elasticity that actually works end to end is one of the few investments that shows up simultaneously in the reliability numbers and on the invoice.