EKS clusters are expensive by default. The combination of EC2 node costs, EKS control plane fee ($0.10/hour = $72/month per cluster), load balancer costs, data transfer, and EBS volume overhead adds up quickly. Teams running multiple EKS clusters for dev, staging, and production often discover their Kubernetes bill is 40-60% higher than it needs to be — not from overuse, but from default configurations that favor reliability over cost efficiency.
The Four Main EKS Cost Drivers
1. Overprovisioned node groups: Nodes sized for peak load running at 20-30% average utilization. The gap between requested resources and actual usage is money wasted.
2. Default cluster autoscaler vs. Karpenter: The default Kubernetes Cluster Autoscaler is slow to scale down (default 10-minute cooldown) and doesn't optimize node type selection. Karpenter is significantly more cost-efficient for variable workloads.
3. No Spot Instance usage: EKS workloads that can tolerate interruption (batch processing, non-critical services, dev/staging) should run on Spot, not on-demand. Spot saves 60-90% on EC2.
4. Non-production clusters running 24/7: Dev and staging clusters that are only used 8 hours per day are paying 100% of on-demand pricing for 66% idle time.
Right-Sizing with Actual Resource Data
Before changing anything, get the actual resource utilization data:
# Install metrics-server if not already running
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
# Check current pod resource usage vs. requests
kubectl top pods --all-namespaces
# See node utilization
kubectl top nodes
# Get per-namespace resource request vs. limit totals
kubectl resource-capacity --sort cpu.util
# (requires kubectl-capacity plugin: https://github.com/robscott/kube-capacity)
Compare your resource requests (resources.requests.cpu/memory in your deployment specs) to actual usage from metrics-server. A service requesting 1000m CPU that never exceeds 200m actual is wasting 80% of its CPU allocation — and driving node sizing unnecessarily high.
Right-sizing process:
1. Run under production load for 1-2 weeks and collect p99 resource consumption
2. Set resources.requests to ~1.3× the p90 actual consumption (buffer for spikes)
3. Set resources.limits to 2× the requests (protection against runaway processes)
4. Evaluate if nodes can be right-sized based on new bin packing efficiency
This process typically reveals 30-50% reductions in required node capacity for production clusters.
Karpenter: Replace Cluster Autoscaler for Variable Workloads
Karpenter makes node provisioning decisions that the Cluster Autoscaler can't — selecting the optimal instance type and purchase option (Spot vs. on-demand) per pending pod's requirements.
# Karpenter NodePool — defines provisioning rules
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
name: default
spec:
template:
metadata:
labels:
karpenter.sh/nodepool: default
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"] # Prefer Spot, fall back to on-demand
- key: kubernetes.io/arch
operator: In
values: ["amd64", "arm64"] # Allow Graviton instances (cheaper)
- key: node.kubernetes.io/instance-type
operator: In
values:
- m5.large
- m5.xlarge
- m6i.large
- m6i.xlarge
- m6g.large # Graviton
- m6g.xlarge # Graviton
nodeClassRef:
name: default
limits:
cpu: 200
memory: 400Gi
disruption:
consolidationPolicy: WhenUnderutilized
consolidationAfterEmpty: 30s # Much faster than CA's 10 minutes
Karpenter consolidation — the most impactful cost feature — continuously evaluates whether your existing nodes are under-utilized and can be compacted. If 3 nodes are running at 30% utilization, Karpenter can drain and terminate 2 of them and reschedule pods onto 1 more densely packed node. The default Cluster Autoscaler only scales down truly empty nodes.
Spot Instance Strategy for EKS
Spot Instances for EKS require designing pods to be interruptible (2-minute warning before termination). The right architecture:
Stateless services: Ideal for Spot. Pods receive SIGTERM on spot interruption, can drain connections and terminate gracefully. Most web services and API handlers are stateless and appropriate for Spot.
Batch processing: Perfect for Spot. Jobs that are interrupted simply restart from their last checkpoint. Design batch jobs with checkpoint/resume logic from the start.
Stateful services, databases, single-instance critical services: Not appropriate for Spot. Run on on-demand.
Mixed Spot/on-demand node groups:
# In your Karpenter NodePool — express Spot preference
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"] # Karpenter prefers Spot when available
# Use taints to control which workloads land on Spot
taints:
- key: "karpenter.sh/capacity-type"
value: "spot"
effect: NoSchedule
Then add tolerations to pods that can handle interruption:
# Deployment that accepts Spot nodes
spec:
template:
spec:
tolerations:
- key: "karpenter.sh/capacity-type"
operator: "Equal"
value: "spot"
effect: "NoSchedule"
Non-Production Cost: Scale to Zero
Dev and staging clusters that don't need to run overnight should scale down:
# Cron job to scale down non-prod at end of day
# In your CI/CD or EventBridge Scheduler
# Scale down all deployments in dev cluster
kubectl scale deployment --all --replicas=0 -n default
# Or: set autoscaler min to 0 and let natural scale-down happen
# Then scale back up in morning via CI job or scheduled EventBridge trigger
Alternatively, for non-production environments, consider EKS on Fargate (pay per pod rather than per node) or AWS Batch for truly intermittent workloads.
For dev environments used only during business hours, shutting down at 7pm and restarting at 7am cuts EC2 node costs by ~58% without any change to daytime developer experience.
EKS Control Plane Cost (Multiple Clusters)
At $0.10/hour per cluster, multiple clusters add up: - 5 clusters (prod, staging, dev, qa, sandbox): $360/month in control plane fees alone - Plus node costs for each
Consider cluster consolidation for non-production: one cluster with namespaces for dev, qa, and sandbox is often sufficient. Reserve dedicated clusters for production and staging only. This saves $144-$216/month in control plane fees and reduces operational overhead.
Rutagon Optimizes EKS Infrastructure
If your EKS costs are higher than expected or your cluster architecture needs rationalization, Rutagon audits and implements cost optimization across your Kubernetes infrastructure. Contact us at rutagon.com/contact.
See also: AWS Reserved Instances strategy and cloud infrastructure capabilities.
FAQ
Is Karpenter better than Cluster Autoscaler for all EKS deployments?
Karpenter is better for dynamic, variable workloads where fast scale-up and aggressive scale-down matter. Cluster Autoscaler is more battle-tested and has broader community support — it works well for stable workloads with well-defined node groups. Karpenter provides more cost efficiency for variable workloads; Cluster Autoscaler provides more predictable behavior for stable ones. Teams running primarily steady-state production workloads often prefer Cluster Autoscaler's predictability.
How much does Spot save on EKS node costs?
Spot discounts are 60-90% off on-demand pricing, varying by instance type and real-time availability. m5.xlarge on-demand: ~$0.19/hr; m5.xlarge Spot: often $0.05-0.08/hr. For a 20-node EKS cluster, moving batch and non-critical workloads to Spot can save $2,000-$4,000/month on EC2 alone.
How do you handle Spot interruptions gracefully in EKS?
Configure a preStop hook in your pod spec to handle graceful shutdown: drain in-flight requests, close database connections, complete checkpoint. Use a terminationGracePeriodSeconds of 120 seconds (matching the Spot 2-minute interruption warning). Implement retry logic in your clients so upstream services handle pod restarts without customer impact. Test interruption handling deliberately using Fault Injection Simulator.
What resource requests and limits are appropriate for most web services?
Starting point for a typical API service: - requests.cpu: 100m-250m (actual usage baseline) - requests.memory: 256Mi-512Mi - limits.cpu: 500m-1000m (protection against CPU hogging) - limits.memory: 512Mi-1Gi (OOMkill protection)
Profile your actual service under load before setting these — they're highly service-specific. Use VPA (Vertical Pod Autoscaler) in recommendation mode to get data-driven suggestions for your specific workloads.
How do I check which nodes are over-provisioned in my EKS cluster?
Run kubectl describe node NODE_NAME for each node and look at the "Allocated resources" section — it shows total requested CPU/memory vs. node capacity, plus actual usage. Nodes consistently at 20-30% actual CPU usage against 90% requested are candidates for right-sizing the underlying EC2 instance type. Automate this with kubectl kube-capacity for cluster-wide views.