Kubernetes clusters are expensive when nodes are oversized for the actual workload. Right-sizing the node fleet with Vertical Pod Autoscaler (VPA), Goldilocks recommendations, and deliberate bin-packing can cut infrastructure spend 25-40% without compromising reliability.
The risk is under-provisioning that causes throttling or OOMKills. The teams that succeed treat right-sizing as a controlled experiment with clear rollback criteria, not a one-time cleanup.
Why Most Clusters Are Oversized
The default behavior for new workloads is to request more CPU and memory than needed. Developers copy example manifests that request 500m CPU and 512Mi for a service that actually uses 80m and 120Mi under load. The scheduler packs those requests onto nodes, leaving large idle capacity.
Over time the gap between requested and actual usage grows. A cluster that shows 70% average CPU utilization on the nodes may be running at 25% of requested resources. The node count is sized for the requests, not the reality.
VPA: The Foundation of Right-Sizing
Vertical Pod Autoscaler observes actual usage over a rolling window and proposes new request and limit values. In recommendation mode it surfaces the numbers. In update mode it patches the deployments automatically.
Most teams start in recommendation mode for 30 days. They review the proposed values, adjust for known spikes, and then enable update mode on non-critical workloads. Production services stay in recommendation mode with a human gate until confidence is high.
VPA works best on stateless services with predictable load patterns. Stateful workloads and services with large memory caches need manual tuning on top of the VPA recommendations.
Goldilocks: Visualizing the Waste
Goldilocks is a dashboard that shows the ratio of requested to actual resources across the cluster. It highlights the biggest offenders and the potential savings if requests were lowered.
The tool is useful for prioritization. A namespace with 50 deployments requesting 2 CPU each but using 200m on average is a clear target. The same namespace with tight requests is left alone.
Goldilocks also surfaces the "burst" pattern where requests are low but limits are high. That pattern wastes node capacity because the scheduler reserves the limit even if the workload never reaches it.
Bin-Packing and Node Group Strategy
Right-sized pods still need nodes that match the actual shape of the workload. A cluster with many small pods benefits from smaller node sizes or higher bin-packing density. A cluster with a few large pods needs larger nodes to avoid fragmentation.
Spot instances add another dimension. Fault-tolerant workloads can run on Spot with a 60-70% discount. The node group strategy mixes on-demand for critical pods and Spot for the rest, with cluster autoscaler handling the transitions.
The target is 80-85% average node utilization after right-sizing. Below 70% there is still waste. Above 90% the risk of scheduling pressure increases.
Measured Results and Risk Controls
A SaaS company with 180 nodes reduced the fleet to 120 nodes over six weeks while maintaining the same latency and error budget. The change cut $18k per month in compute spend.
The risk controls were explicit. Any workload that showed increased latency or restart rate above baseline triggered an immediate rollback of that deployment's requests. The team also maintained a 15% buffer of idle capacity for unexpected spikes.
Rollback is cheap when the manifests are in Git and the change is a single number. The expensive failure mode is discovering under-provisioning during a traffic spike with no easy path back.
When Right-Sizing Is Not Enough
Some workloads are inherently bursty or have large memory requirements that make dense packing difficult. In those cases, the savings come from architecture changes (moving to serverless, splitting monoliths, or adopting different data stores) rather than node tuning.
Right-sizing is also not a substitute for eliminating unused workloads. A namespace that runs only during business hours can be scaled to zero overnight. That lever often saves more than node optimization.
Have a Kubernetes cluster whose bill needs this level of attention? Talk to Rutagon or call 907-841-8407. We run the utilization analysis and surface the safe right-sizing path in the first call.
FAQ
How long does VPA need to observe before recommendations are reliable?
VPA needs at least 7-14 days of representative load. Workloads with weekly or monthly cycles need a full cycle. The recommendation quality improves with more data, but the first two weeks usually surface 80% of the opportunity.
Can VPA and HPA run together?
Yes. VPA adjusts requests and limits. HPA adjusts replica count based on observed metrics. The two work together when the HPA target is set on the right metric (usually custom metrics, not CPU). Setting HPA on CPU while VPA is also tuning CPU requests creates oscillation. Use separate metrics.
What about memory limits and OOMKills?
VPA recommendations include both requests and limits. The safe pattern is to set limits 20-30% above the recommended request for most workloads. Memory-hungry services need a higher buffer or manual tuning. The rollback criteria should include OOMKill rate.
Does right-sizing affect cluster autoscaler behavior?
Yes. Tighter requests allow the autoscaler to pack more pods per node, which can reduce node count. The autoscaler also reacts faster to scale-down when requests are realistic. The combination often produces larger savings than either lever alone.
How do we convince developers to accept lower requests?
Show the before-and-after numbers on actual production traffic, not synthetic benchmarks. Pair the change with a clear rollback plan and a 30-day observation period. Most developers accept the change once they see that latency and error rates do not move.
---
*Cross-reference: See Reduce AWS Bill: Cost Optimization for the broader FinOps program that makes node right-sizing one lever among many.*