Summary

Addresses gaps documented in issue #3740 (sub-task of #3735). The rate limiter plugin had several correctness issues that made it silently ineffective in multi-instance deployments.

Gaps closed

Gap 1 (HIGH) — No shared state across instances: each gateway process had its own `_store` dict, making the effective per-user limit `N × configured_limit` where N = number of instances. Fixed with `RedisBackend` using an atomic Lua INCR+EXPIRE script — all instances share the same counter.

Gap 2 (MEDIUM) — No store eviction: the in-memory `_store` dict grew unboundedly as unique users accumulated. Fixed with a background asyncio sweep task in `MemoryBackend` that evicts expired windows every 0.5s.

Gap 3 (MEDIUM) — No thread safety: dict access was not atomic under concurrent asyncio tasks. Fixed with `asyncio.Lock` in `MemoryBackend.allow()`.

Gap 5 (LOW) — `prompt_pre_fetch` silently ignored `by_tool` config. Fixed by adding the same `by_tool` lookup to the prompt hook.

Gap 6 (MEDIUM) — Malformed config (`"abc/m"`) raised `ValueError` at request time. Fixed with `_validate_config()` called at `init` — fails fast at startup.

Gap 7 (LOW) — Unsupported rate unit (`"60/d"`) raised `ValueError` at request time. Fixed by `_parse_rate()` raising early during `_validate_config()`.

Gap 8 (MEDIUM) — Any unexpected exception in the plugin propagated to the caller. Fixed with `try/except Exception` wrapping both hooks; logs and returns permissive result on unexpected error.

Gap 4 (sliding window) intentionally deferred — the fixed window burst-at-boundary is a known trade-off. The `test_fixed_window_burst_at_boundary` xfail test documents the gap.

Additional hardening

PII fix — violation descriptions previously included `user` and `tenant` identifiers in the human-readable string (e.g. `"Rate limit exceeded for tool search, user alice@example.com, or tenant acme-corp"`). This string is logged directly in permissive mode and embedded in `PluginViolationError` messages in enforce mode. Fixed to only include the tool/prompt name — structured metadata remains in the `details` dict.

Bypass resistance documented — three bypass vectors identified and tested:

• Whitespace-only user identity creates its own bucket (xfail, documented)
• Tool name case sensitivity bypasses `by_tool` limits (xfail, documented)
• Tool name whitespace bypasses `by_tool` limits (xfail, documented)

README rewritten — the previous README described the old pre-fix implementation (in-memory only, Redis listed as a TODO). Now documents both backends, full configuration reference, response headers, multi-instance deployment guidance, and an accurate limitations table.

Architecture

• `MemoryBackend`: `asyncio.Lock` + background TTL sweep; single-process only
• `RedisBackend`: atomic Lua INCR+EXPIRE; shared across all gateway instances; falls back to `MemoryBackend` on Redis unavailability if `redis_fallback: true`
• `_validate_config()`: parses all rate strings at `init`; collects all errors and raises `ValueError` with a full report
• Both hooks: `try/except Exception` with permissive degradation on unexpected errors

Test results

Unit tests: 70 passed, 4 xfailed

• Core enforcement, headers, multi-dimensional limiting
• Permissive vs enforce mode (via `PluginExecutor`)
• Redis fallback (broken Redis → memory, limit still enforced, no-fallback graceful degradation)
• Cross-tenant isolation and counter bleed prevention
• Header accuracy (Retry-After bounds, Reset consistency, Remaining decrement)
• Bypass resistance (None/whitespace identity, tool name normalisation gaps)
• PII: violation description contains no user or tenant identifiers

Integration tests: 16/16 passed (`tests/integration/test_rate_limiter.py`, `--with-integration`)

Plugin framework tests: 53/53 passed

Performance profiling (`tests/performance/test_plugins_performance.py`, 1000 iterations):

Redis adds ~0.075ms per hook call — well under 1ms.

Multi-instance load test (`make benchmark-rate-limiter`, 3 gateway instances, 1 user at 60 req/min = 2× the 30/m limit, 120s):

*40% observed vs ~50% theoretical: the first 30 requests in each 60-second window are always allowed before blocking starts. Over a longer run it converges toward ~50%.

Redis MONITOR confirmed all 3 instances write to the same atomic counter:
```
[172.18.0.8] EVAL "rl:user:admin@example.com:60" "60"
[172.18.0.9] EVAL "rl:user:admin@example.com:60" "60"
[172.18.0.10] EVAL "rl:user:admin@example.com:60" "60"
```

Closes #3740