Kubernetes clusters are commonly 30-60% over-provisioned. The combination of overly conservative resource requests, under-used reserved node capacity, and missing autoscaling configuration means most production clusters are paying for significantly more compute than the workloads consume.
This guide covers the Kubernetes cost optimization techniques that produce the largest savings in production environments.
The Over-Provisioning Problem
Kubernetes resource management has two layers that must both be right for efficient resource utilization:
-
Pod resource requests/limits: What each container declares it needs (
requests) and the maximum it can use (limits). These determine scheduling and affect actual resource consumption. -
Node capacity: The underlying EC2/GCE instances that form the cluster. If pods are under-utilizing their requests, nodes are under-utilized, and you're paying for idle compute.
The cascade: If a pod requests 1 CPU but only uses 0.2 CPU, the Kubernetes scheduler treats 1 CPU as consumed. A node with 4 CPUs can only schedule 4 of these pods, even though actual usage is only 0.8 CPU. You're paying for 4 CPUs to do the work of 0.8 CPUs.
Fixing the resource request problem is the foundational step in Kubernetes cost optimization.
Step 1: Measure Actual Resource Utilization
You cannot right-size requests without data. Set up metrics first:
Prometheus + Grafana (standard):
Deploy kube-state-metrics, node-exporter, and Prometheus. The key metrics:
- container_cpu_usage_seconds_total vs kube_pod_container_resource_requests{resource="cpu"}
- container_memory_working_set_bytes vs kube_pod_container_resource_requests{resource="memory"}
The ratio of actual usage to requested resources is your "utilization efficiency" metric. Target: 60-80% utilization of requests (not 20-30%).
Goldilocks (simpler approach): Goldilocks is a Kubernetes tool that runs VPA in recommendation mode and generates a dashboard showing recommended request/limit values per container. For teams without deep Prometheus expertise, Goldilocks provides actionable right-sizing recommendations without requiring manual metric analysis.
Kubecost: For complete cost attribution and optimization recommendations across namespaces, workloads, and teams, Kubecost provides both visibility and recommendations in a single tool. The free tier covers most single-cluster use cases.
Step 2: Right-Size Resource Requests and Limits
With utilization data in hand, update resource requests to reflect actual usage plus a safety margin:
Recommended sizing formula:
- requests.cpu = P90 CPU usage × 1.2 (20% headroom)
- requests.memory = P99 memory usage × 1.15 (15% headroom — memory OOM kills are bad)
- limits.cpu = 2-4× requests (allow bursting, prevent starvation)
- limits.memory = 1.2-1.5× requests (tighter — memory limits protect against runaway allocations)
Example: A web service with measured P90 CPU of 200m and P99 memory of 256 MB:
resources:
requests:
cpu: "250m" # 200m × 1.2
memory: "295Mi" # 256Mi × 1.15
limits:
cpu: "1000m" # 4× requests (allow bursting)
memory: "450Mi" # 1.5× requests
Don't set CPU limits too tight: CPU throttling at the container level is one of the most common performance issues in Kubernetes. If your CPU limit is too close to your CPU request and your workload has traffic bursts, you'll see latency spikes from CPU throttling even when the node has available CPU. Setting CPU limits to 2-4× requests allows burst capacity.
Step 3: Vertical Pod Autoscaler (VPA)
VPA automatically adjusts resource requests and limits based on measured usage. It's the production-scale solution to manual right-sizing.
VPA modes:
- Off: Recommendations only, no automatic changes
- Initial: Sets resources at pod creation, doesn't update running pods
- Auto: Automatically updates resources, recreates pods when changes are significant (disrupts running pods)
Recommended VPA deployment:
Start with Off mode — use VPA as a recommendation engine. Review recommendations weekly and update deployment manifests. Graduating to Initial mode for stable workloads reduces ongoing maintenance. Auto mode is appropriate for stateless workloads that tolerate occasional pod restarts.
VPA + HPA note: VPA and HPA (Horizontal Pod Autoscaler) conflict on CPU-based HPA. Don't use both VPA and CPU-based HPA on the same deployment. Use VPA with HPA based on custom metrics or memory, or use VPA alone for scaling.
Step 4: Cluster Autoscaler and Node Right-Sizing
Workload right-sizing reduces per-pod resource consumption. Cluster autoscaler ensures nodes scale down when those smaller pods are scheduled, converting workload optimization into actual cost reduction.
Cluster Autoscaler configuration for cost:
- Set scale-down-delay-after-add to a reasonable window (10-30 minutes)
- Ensure pods have PodDisruptionBudgets configured to allow safe draining
- Verify expander: least-waste to select the node type that wastes the least capacity
Node group sizing strategy: Multiple node groups with different instance types allow the cluster autoscaler to select the best fit: - General workload group: m5.xlarge or m5.2xlarge (cost-effective, balanced) - Memory-intensive group: r5.xlarge or r5.2xlarge (for high-memory workloads) - Spot instance group: m5.xlarge spot (for fault-tolerant, interruptible workloads)
ARM/Graviton nodes: Graviton3-based instances (m7g, r7g, c7g) are 20% cheaper than comparable x86 instances. For applications without x86-specific requirements, Graviton node groups reduce node costs immediately. Kubernetes workloads running standard runtimes (Python, Node.js, Java, Go) are typically ARM-compatible without changes.
Step 5: Spot Instances for Fault-Tolerant Workloads
Spot instances provide 60-90% discounts vs on-demand pricing. For Kubernetes workloads tolerant of interruption (typically stateless services), spot is the most impactful cost reduction.
Safe spot instance patterns: - Stateless API services with graceful shutdown handling - Background job workers - CI/CD runners - Development and staging environments
Unsafe for spot without careful design: - Databases (stateful, interruption dangerous) - Singleton services without HA (interruption causes downtime) - Long-running batch jobs without checkpointing
Karpenter (EKS): Karpenter is the successor to Cluster Autoscaler for EKS and provides smarter node provisioning, including automatic selection of spot instances from multiple instance families (reducing interruption risk from any single instance type being reclaimed).
Mixed node group strategy: Run a small on-demand base (20-30% of capacity) plus a spot group for the remainder. This provides reliability while capturing spot savings for the majority of workload capacity.
Namespace and Team Cost Attribution
For multi-team clusters, cost visibility per team drives accountability and optimization behavior.
Kubecost or OpenCost applied to namespace resource consumption generates per-namespace cost reports. When teams see their own cost data: - Engineers optimize resource requests (they see the waste they're causing) - Over-provisioned development environments get scaled down - Cost-per-feature becomes a real metric in planning
Implementation: Monthly cost report by namespace/team, sent to team tech leads. Set cost targets per team per quarter.
Rutagon provides Kubernetes cost optimization and cloud FinOps services for engineering teams. Contact us to discuss a cluster cost audit.
Frequently Asked Questions
How much can Kubernetes cost optimization realistically save?
For clusters with unoptimized resource requests, the typical savings range from 30-60% of cluster node costs. The biggest wins come from: right-sizing over-provisioned resource requests (often 20-40% savings), spot instances for stateless workloads (additional 40-60% on those nodes), and removing over-provisioned buffer capacity that was added "just in case."
Does right-sizing resource requests affect application performance?
When done correctly, no. Right-sizing should maintain or improve performance by giving pods accurate resource allocations. The risk is setting requests too low and causing resource starvation during load spikes. This is why P90/P99 metrics with safety margins are used, rather than average utilization, and why CPU limits should be set generously to allow bursting.
What's the difference between requests and limits in Kubernetes?
Requests are what the scheduler uses to place pods on nodes — a pod with a 250m CPU request is guaranteed 250m CPU on whatever node it's placed on. Limits are the maximum a container can use — if a container tries to use more CPU than its limit, it's throttled; if it uses more memory than its limit, it's OOM-killed. Setting requests accurately is critical for scheduling efficiency; setting limits appropriately prevents runaway resource consumption.
How does VPA handle stateful workloads?
VPA in Auto mode recreates pods when it updates resource requests, which is disruptive for stateful workloads. For stateful services (databases, caches, message queues), use VPA in Off or Initial mode only — use recommendations to inform manual resource spec updates during maintenance windows. StatefulSet pods should not be automatically recreated.
Is Karpenter better than Cluster Autoscaler?
Karpenter (EKS-specific) provides faster scaling, better spot instance diversity, and more intelligent node selection compared to Cluster Autoscaler. For new EKS clusters, Karpenter is generally recommended. For existing clusters using Cluster Autoscaler, migration to Karpenter is worth planning for clusters with significant spot instance requirements or variable load patterns. Cluster Autoscaler is more widely applicable (any Kubernetes cluster) while Karpenter is EKS-specific.