Have you ever watched your serverless application crumble under unexpected traffic? Last month, our AI-powered image generator went viral on social media, and within hours we were drowning in requests. Our traditional rate limiting setup couldn’t keep up with the distributed load across Cloudflare’s edge network.
This experience taught me that rate limiting in serverless environments requires a fundamentally different approach. Here’s how I built a production-ready rate limiter using Cloudflare Durable Objects that handles thousands of concurrent requests while running at the edge.
The Traditional Approach Falls Short
Most developers reach for Redis when implementing rate limiting. It’s fast, reliable, and has excellent libraries. But in a serverless environment, especially one distributed across multiple edge locations, Redis introduces several challenges:
Latency Issues: Every rate limit check requires a round trip to your Redis instance, adding 50-200ms of latency depending on geographic distance.
Single Point of Failure: Your entire rate limiting system depends on Redis availability. If Redis goes down, you either block all traffic or allow unlimited requests.
Cold Start Problems: Serverless functions need to establish Redis connections on cold starts, further increasing response times.
Geographic Complexity: Running Redis replicas across multiple regions for low latency gets expensive quickly and introduces data consistency challenges.
Enter Durable Objects
Cloudflare Durable Objects solve these problems elegantly by providing stateful compute primitives that run directly at the edge. Each Durable Object instance maintains its own persistent state and can handle concurrent requests while ensuring strong consistency.
Think of Durable Objects as tiny, persistent computers that live at Cloudflare’s edge locations. They wake up when needed, maintain state between requests, and automatically migrate to follow your traffic patterns.
Building the Rate Limiter
Here’s the complete implementation of a production-ready rate limiter using Durable Objects:
import { DurableObject } from "cloudflare:workers";
interface ThrottleState {
limitTimes: number;
limitEndTimeMs: number;
executedTimesCurrentCycle: number;
currentCycle: number;
}
export interface TryApplyOptions {
limitCycleExecutionTimes: number;
limitCycleTimeMs: number;
}
export interface ThrottlerResponse {
granted: boolean;
state: ThrottleState;
}
export class ThrottlerDO extends DurableObject {
limitCycleExecutionTimes = 10; // Default: 10 requests per cycle
limitCycleTimeMs = 10 * 60 * 1000; // Default: 10 minutes
constructor(ctx: DurableObjectState, env: Env) {
super(ctx, env);
}
async getState(): Promise<ThrottlerResponse> {
let state = await this.ctx.storage.get('throttle_state') as ThrottleState | null;
if (!state) {
state = {
limitTimes: 0,
limitEndTimeMs: 0,
executedTimesCurrentCycle: 0,
currentCycle: 0,
};
}
const currentMs = Date.now();
// Reset state if cycle has expired
if (state.limitEndTimeMs > 0 && currentMs > state.limitEndTimeMs) {
state = {
...state,
limitEndTimeMs: 0,
executedTimesCurrentCycle: 0,
};
}
const granted = state.executedTimesCurrentCycle < this.limitCycleExecutionTimes;
return { granted, state };
}
async tryApply(options?: TryApplyOptions): Promise<ThrottlerResponse> {
if (options) {
this.limitCycleExecutionTimes = options.limitCycleExecutionTimes;
this.limitCycleTimeMs = options.limitCycleTimeMs;
}
let granted = false;
let state = await this.ctx.storage.get('throttle_state') as ThrottleState | null;
if (!state) {
state = {
limitTimes: 0,
limitEndTimeMs: 0,
executedTimesCurrentCycle: 0,
currentCycle: 0,
};
}
const currentMs = Date.now();
// Reset cycle if expired
if (state.limitEndTimeMs > 0 && currentMs > state.limitEndTimeMs) {
state.limitEndTimeMs = 0;
state.executedTimesCurrentCycle = 0;
}
// Check if request can be granted
if (state.executedTimesCurrentCycle < this.limitCycleExecutionTimes) {
state.executedTimesCurrentCycle++;
granted = true;
} else {
state.limitTimes++;
granted = false;
}
// Initialize new cycle if needed
if (state.limitEndTimeMs === 0) {
state.limitEndTimeMs = currentMs + this.limitCycleTimeMs;
state.currentCycle++;
if (state.currentCycle >= 65535) {
state.currentCycle = 1;
}
}
await this.ctx.storage.put('throttle_state', state);
return { granted, state };
}
}
Using the Rate Limiter
Integrating this rate limiter into your Worker is straightforward:
export default {
async fetch(request: Request, env: Env) {
// Create DO instance based on user identifier
const userId = request.headers.get('user-id') || 'anonymous';
const id = env.THROTTLER.idFromName(userId);
const throttler = env.THROTTLER.get(id);
// Check rate limit
const result = await throttler.tryApply({
limitCycleExecutionTimes: 5,
limitCycleTimeMs: 60 * 1000, // 1 minute
});
if (!result.granted) {
return new Response('Rate limit exceeded', {
status: 429,
headers: {
'Retry-After': '60'
}
});
}
// Process the actual request
return processRequest(request);
}
};
Real-World Performance
I’ve been running this rate limiter in production for several months now. The results have been impressive:
Latency: Average rate limit checks complete in under 5ms, compared to 80-150ms with our previous Redis setup.
Reliability: Zero downtime related to rate limiting failures since deployment. Durable Objects automatically handle failover and state migration.
Cost Efficiency: Running at roughly 30% of our previous Redis costs while serving 10x more requests.
Geographic Performance: Users in Asia Pacific see the same low latency as users in North America, thanks to Cloudflare’s global network.
This architecture has been battle-tested at Fastjrsy, where we process thousands of AI-generated jersey design requests daily. During traffic spikes, the rate limiter seamlessly scales to handle 500+ concurrent requests per second while maintaining fair usage policies across our global user base.
Key Takeaways
Durable Objects excel when you need:
- Stateful logic at the edge
- Strong consistency guarantees
- Automatic scaling and geographic distribution
- Low latency for global applications
They may not be the right choice if you need:
- Cross-platform compatibility beyond Cloudflare
- Complex query capabilities like traditional databases
- Shared state across multiple applications
What’s Your Experience?
Have you implemented rate limiting in serverless environments? I’d love to hear about your approach and any challenges you’ve encountered. Are there specific use cases where you think Durable Objects would be particularly valuable?
Drop a comment below and let’s discuss the future of edge computing architecture!
