An API without rate limiting is one noisy client, one misconfigured integration, or one bad actor away from an outage. We consistently see rate limiting treated as an afterthought — bolted on after an incident rather than designed in from the start. Here's how we approach it for production APIs.
Rate Limiting Algorithms: Which One and Why
Token Bucket
Each client has a "bucket" that refills at a fixed rate, with requests consuming tokens. This allows short bursts up to the bucket's capacity while enforcing a sustained average rate over time — generally our default choice because it accommodates realistic usage patterns better than a hard per-second cap.
Sliding Window Log
Tracks exact request timestamps within a rolling window, providing precise rate enforcement at the cost of more memory overhead per client. We use this when exact fairness matters more than burst tolerance — for example, enforcing strict per-minute quotas on a metered billing API.
Fixed Window Counter
Simplest to implement, but has a boundary problem: a client can send double their limit by timing requests around a window boundary. We avoid this for anything where the limit is meant to be a hard guarantee.
Implementation on AWS API Gateway
UsagePlan:
Type: AWS::ApiGateway::UsagePlan
Properties:
Throttle:
BurstLimit: 200
RateLimit: 50
Quota:
Limit: 100000
Period: MONTH
ApiStages:
- ApiId: !Ref RestApi
Stage: production
For more granular control than API Gateway's native throttling provides, we implement token bucket logic in a Lambda authorizer or at the application layer using Redis (via ElastiCache) as a shared counter store across all API instances:
def check_rate_limit(client_id: str, limit: int, window_seconds: int) -> bool:
key = f"ratelimit:{client_id}"
current = redis_client.incr(key)
if current == 1:
redis_client.expire(key, window_seconds)
return current <= limit
Per-Tier Rate Limits
Most production APIs need different limits per customer tier, not a single global limit. We design the rate limit configuration as data, not code, so adjusting a specific customer's limits doesn't require a deployment:
TIER_LIMITS = {
"free": {"requests_per_minute": 60, "burst": 10},
"pro": {"requests_per_minute": 600, "burst": 100},
"enterprise": {"requests_per_minute": 6000, "burst": 1000},
}
Graceful Degradation, Not Just Rejection
Rate limiting shouldn't be a binary "allowed or 429 rejected." We build in graceful degradation where it makes sense — for example, serving cached or slightly stale data to a client approaching their limit rather than outright rejecting the request, when staleness is acceptable for that endpoint.
Communicating Limits to Clients
Every rate-limited response includes standard headers so well-behaved clients can self-throttle:
X-RateLimit-Limit: 600
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1735689600
Retry-After: 30
Clients that respect these headers back off automatically; clients that ignore them still get a clear, standards-based signal rather than an opaque failure.
Protecting Against Distributed Abuse
Rate limiting per API key or client ID doesn't stop abuse distributed across many accounts or IP addresses. For public-facing APIs, we layer in AWS WAF rate-based rules at the edge, catching volumetric abuse patterns before they even reach the rate-limited application layer.
Results
For a client whose API had suffered repeated outages from a single misbehaving integration partner overwhelming shared infrastructure, implementing tiered token-bucket rate limiting with Redis-backed shared state eliminated the outage pattern entirely, while legitimate high-volume enterprise customers saw no degradation in their normal usage.
Building an API that needs to survive real-world traffic patterns? Talk to us about your AWS architecture — 907-841-8407 or contact@rutagon.com.
Frequently Asked Questions
What's the difference between token bucket and sliding window rate limiting?
Token bucket allows short bursts up to a capacity limit while enforcing a sustained average rate, fitting realistic client usage patterns. Sliding window log tracks exact request timestamps for precise, strict enforcement, at higher memory cost per client.
Should rate limits be the same for every customer?
Usually not. Most production APIs benefit from tiered limits based on customer plan or contract, configured as adjustable data rather than hardcoded values so limits can change without a deployment.
How do you rate limit across multiple API server instances?
A shared counter store like Redis (via AWS ElastiCache) lets all API instances check and increment the same rate limit counters, ensuring consistent enforcement regardless of which instance handles a given request.
Does rate limiting protect against distributed abuse from many IP addresses?
Not on its own — per-client rate limiting only protects against a single account or key exceeding its limit. Edge-level protections like AWS WAF rate-based rules add a layer that catches volumetric abuse distributed across many sources.
What should an API return when a client exceeds their rate limit?
A 429 status code along with standard headers (X-RateLimit-Limit, X-RateLimit-Remaining, Retry-After) so well-behaved clients can programmatically back off and retry at the appropriate time.