Summary
The current rate limiting implementation in src/proxy.ts uses a module-scoped Map (rateLimitMap) to track request counts per client identifier. Because this is process-local, rate limit counters are not shared across server instances or replicas, making it easy to bypass limits in multi-instance/horizontal deployments.
Problem
rateLimitMap is isolated to a single Node.js process- Restarts reset all counters
- Requests routed to different replicas are each rate-limited independently, so a client can exceed the intended threshold by the number of replicas
Proposed Solution
Migrate rateLimitMap to a shared store such as Redis or a Next.js KV provider:
- Initialize a single Redis/KV client used by the proxy handler
- Store per-identifier counters and expiry in the shared store
- Use atomic increment + TTL operations (e.g.,
INCR + EXPIRE) to implement count/resetTime logic
- Update
checkRateLimit and shouldBypassRateLimit to read/write from the shared store
References
Raised by @Producdevity.
Summary
The current rate limiting implementation in
src/proxy.tsuses a module-scopedMap(rateLimitMap) to track request counts per client identifier. Because this is process-local, rate limit counters are not shared across server instances or replicas, making it easy to bypass limits in multi-instance/horizontal deployments.Problem
rateLimitMapis isolated to a single Node.js processProposed Solution
Migrate
rateLimitMapto a shared store such as Redis or a Next.js KV provider:INCR+EXPIRE) to implement count/resetTime logiccheckRateLimitandshouldBypassRateLimitto read/write from the shared storeReferences
Raised by @Producdevity.