Skip to content

fix(ratelimit): exempt GET requests from apiLimiter to fix source-toggle 429s (#3735)#3741

Closed
Yeraze wants to merge 1 commit into
mainfrom
claude/great-dijkstra-rcs4u2
Closed

fix(ratelimit): exempt GET requests from apiLimiter to fix source-toggle 429s (#3735)#3741
Yeraze wants to merge 1 commit into
mainfrom
claude/great-dijkstra-rcs4u2

Conversation

@Yeraze

@Yeraze Yeraze commented Jun 25, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes #3735 — toggling sources a few times exhausts the API rate limit (HTTP 429) and breaks the UI for 15 minutes.

Root cause: apiLimiter was applied to all /api routes including GETs. Source-switching invalidates and refetches ~50+ React Query hooks simultaneously (nodes, messages, channels, map data, route segments, …). For users behind a reverse proxy without TRUST_PROXY=1, all requests share a single bucket keyed to the proxy IP. At 50+ requests per toggle × a few toggles, the 1000 req/15-min default is easily exhausted.

Fix: Add req.method === 'GET' to the skip condition of apiLimiter. Read operations carry minimal security risk on a self-hosted tool; the relevant protections are already elsewhere:

  • authLimiter guards login endpoints (5 attempts/15 min)
  • messageLimiter guards message sends (30/min)
  • meshcoreDeviceLimiter guards device operations (10/min)

POST/PUT/DELETE to the general API remain rate-limited as before.

Changes

  • src/server/middleware/rateLimiters.ts — add req.method === 'GET' to apiLimiter's skip predicate; update startup log line accordingly
  • src/server/middleware/rateLimiters.test.ts — update the "enforce after max exceeded" test to use POST (since GET is now exempt); add explicit regression test asserting GET requests bypass the limiter regardless of count

Test plan

  • vitest run src/server/middleware/rateLimiters.test.ts — 27/27 pass (includes new GET-exempt test)
  • CI suite

🤖 Generated with Claude Code

https://claude.ai/code/session_01DAQy9K5MzFV2PJYha6RgFZ


Generated by Claude Code

…gle 429s (#3735)

Source toggling fires 50+ simultaneous GET queries (nodes, messages, channels,
map data, etc.); a few rapid toggles exhausted the default 1000-req/15-min
window for users behind a non-private reverse proxy. Reads carry minimal
security risk on a self-hosted tool, and write operations already have
dedicated limiters (messageLimiter, authLimiter).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DAQy9K5MzFV2PJYha6RgFZ
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Claude finished @Yeraze's task —— View job


PR Review: fix(ratelimit): exempt GET requests from apiLimiter

  • Read changed files
  • Review code quality and best practices
  • Check for bugs or issues
  • Evaluate security concerns
  • Assess test coverage

Summary

This is a clean, well-reasoned fix. The change is minimal (one condition added to the skip predicate), the rationale is documented thoroughly, and the tests are comprehensive. Here are a few observations:


Code Quality

rateLimiters.ts:88 — The skip predicate change is correct and idiomatic:

skip: (req) => env.rateLimitApi === 0 || isPrivateNetworkIp(req.ip ?? '') || req.method === 'GET',

The ordering is good: cheap checks first (disabled flag, then string match), method check last.

Startup logging (rateLimiters.ts:64) — The new log line is accurate and helpful:

- API GET requests: always exempt (navigation-driven reads)

Potential Issues

HEAD and OPTIONS methods — The fix exempts only GET, but HEAD requests (which are semantically safe, side-effect-free reads per RFC 9110) and OPTIONS (preflight CORS) are still rate-limited. In practice this is unlikely to matter for MeshMonitor's self-hosted use case, but it's worth noting. If the React app ever issues OPTIONS preflights in volume (e.g. with non-simple CORS headers), those would count against the limit. This is not a blocker — just something to be aware of if CORS issues ever surface.

No PATCH considerationPATCH is a mutating method and is correctly left rate-limited. This is intentional and correct.


Security

The PR's security rationale is sound:

  • authLimiter (5 req/15 min) still protects login endpoints
  • messageLimiter (30/min) still protects message sends
  • meshcoreDeviceLimiter (10/min) still protects device operations
  • Only read operations are exempted, which carry minimal abuse risk on a self-hosted dashboard

No security concerns with this approach.


Test Coverage

Tests are thorough:

  • rateLimiters.test.ts:129 — Correctly updated the enforcement test to use POST (since GET is now exempt) with an explicit comment explaining why (// Use POST — GET is exempt from the API limiter (#3735))
  • rateLimiters.test.ts:150 — New regression test explicitly named #3735 verifies GET requests are not blocked even when the rate limit would be exceeded
  • Startup logging test (rateLimiters.test.ts:299) verifies the new GET requests: always exempt log line is emitted

One minor observation: the new GET exemption test at line 150 sends 5 requests against a rateLimitApi: 2 limit, which is sufficient to demonstrate the bypass. This is good coverage.


Minor Nit (non-blocking)

The test at line 150 sets app.set('trust proxy', 1) and uses X-Forwarded-For: 1.2.3.4 for the GET test — consistent with the POST enforcement test. This is correct and ensures the GET bypass isn't accidentally caused by the private-IP exemption.


Overall: LGTM. The fix is minimal, well-documented, correctly scoped, and backed by targeted regression tests. Ready to merge.

Yeraze added a commit that referenced this pull request Jun 25, 2026
…hout exempting GETs (#3735) (#3748)

* perf(dashboard): batch per-source reads into aggregate endpoints to cut request volume (#3735)

Source-heavy dashboards fired one GET per dataset per source on every 15s
poll (nodes+traceroutes+neighbor-info+channels = 4×N, plus status = N). At 5
sources that's ~25 req/15s — already over the 1000-per-15-min API limit before
any user action, and toggling a source spiked it enough to trip 429s that broke
the UI for the full window (#3735). The proposed fix (exempting all GETs from
the limiter, #3741) removes back-pressure protection for low-powered / busy
servers; this instead cuts the volume at the source.

Backend:
- New shared builders (src/server/services/sourceDashboardData.ts) extracted
  verbatim from the per-dataset GET handlers, so individual routes and the new
  aggregates compute identical, identically-masked results.
- GET /api/sources/:id/dashboard — bundles a source's 4 datasets (5→1 for the
  single-source view).
- GET /api/unified/dashboard?sources= — one request for the whole unified view
  (4N→1). Per-dataset permission gating mirrors the individual routes (a denied
  dataset returns [] rather than 403-ing the bundle).

Frontend:
- useDashboardSourceData / useDashboardUnifiedData consume the aggregates.
- Debounce the source-toggle refresh so a flurry of toggles coalesces into one
  refetch cycle.

Net: unified view ~4N+N → ~1+N requests/poll; the existing rate limiter stays
fully in place. The four individual endpoints are unchanged in behavior
(validated by the existing sourceRoutes suites).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SJPe6J5vKrcbwzt6vCdtrt

* address review: use TransportMechanism constants, fetch maxNodeAgeHours once, guard ?sources= length

- buildSourceNeighborInfo: replace inline TX_MQTT=5/TX_UDP=6 with the canonical
  TransportMechanism.MQTT / .MULTICAST_UDP constants (CLAUDE.md).
- maxNodeAgeHours is global: fetch it once in the /unified/dashboard route and
  thread it through buildSourceDashboard → buildSourceNeighborInfo, so a
  multi-source poll no longer re-queries the same setting once per source.
- Cap the ?sources= query string (4096) before split; document that anonymous
  access yields empty bundles via the existing per-dataset permission gates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SJPe6J5vKrcbwzt6vCdtrt

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Yeraze

Yeraze commented Jun 25, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #3748 (merged). Rather than exempting all GET requests from the API rate limiter — which removes back-pressure protection for low-powered / heavily-utilized servers — #3748 cuts the request volume that caused the 429s: per-source dashboard reads are now bundled into aggregate endpoints (/api/sources/:id/dashboard and /api/unified/dashboard), taking the unified view from ~4N+N to ~1+N requests per poll, and the source-toggle refresh is debounced. The limiter stays fully in place. Closing in favor of that approach.

@Yeraze Yeraze closed this Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Source toggling triggers HTTP 429 rate limit, breaking the UI for 15 minutes (v4.11.6)

2 participants