feat: middleware.ClientIP, a replacement for middleware.RealIP#967
Conversation
convto
left a comment
There was a problem hiding this comment.
I’ve checked the implementation, and I completely agree with the approach.
It addresses the concerns beautifully, and I’m very grateful for the effort put into this!
c2354ea to
4b70ef1
Compare
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Covers single/multiple trusted proxy chains, spoofing attempts, prefix boundary values, and no-prefix baseline as requested in PR go-chi#967.
This comment was marked as outdated.
This comment was marked as outdated.
Rework the new ClientIP* middlewares per reviewer feedback (c2h5oh, adam-p, rezmoss, Saku0512) and align with the documented attack patterns in GHSA-3fxj-6jh8-hvhx, GHSA-rjr7-jggh-pgcp, GHSA-9g5q-2w5x-hmxf. API: - Rename ClientIPFromXFFHeader -> ClientIPFromXFF. - Add ClientIPFromXFFTrustedProxies(n) for dynamic proxy pools where enumerating CIDRs isn't practical (autoscaling, ephemeral containers). - Add GetClientIPAddr alongside GetClientIP; both back to a single netip.Addr stored in context. r.RemoteAddr is never mutated. - Drop IsLoopback/IsPrivate filtering: the user's explicit trust configuration is authoritative (k8s nginx-ingress and similar legitimately surface those values). - Merge multiple X-Forwarded-For header instances before walking (RFC 2616), defeating duplicate-header attacks. - Deprecate middleware.RealIP with citations to all three advisories and guidance pointing at the new API. Docs: - Per-function godoc explains exactly when each variant applies. - Example_clientIP is a single, consolidated decision guide rendered on pkg.go.dev. Tests: 56 subtests including explicit PoC reproductions of each advisory, /24 boundary cases (Saku0512), IPv6, multi-header merging, spoofing prevention, and middleware chaining (rezmoss). Co-authored-by: Cursor <cursoragent@cursor.com> Co-Authored-By: Claude Sonnet 4.7 <noreply@anthropic.com>
2268552 to
716b67a
Compare
Saku0512 correctly flagged in #967 that the godoc told users to register ClientIPFromRemoteAddr *after* ClientIPFromXFFTrustedProxies for fallback behavior. With last-write-wins semantics (482e855) that recipe is actively wrong: on the happy path RemoteAddr would silently overwrite the legitimate XFF-derived client IP with the immediate proxy's address — reintroducing the spoofing-prone behavior this API was written to prevent. Rather than just reverse the order in the example, drop the chaining hint entirely. The whole point of 482e855 was to push users toward picking exactly one ClientIPFrom* middleware; documenting a chaining recipe in one specific function works against that. The honest statement is "no client IP is set and GetClientIP returns \"\"" — what the caller does about it is on them. Also rename TestClientIPChaining -> TestClientIPLastWriteWins and reword its subtests to describe the property being locked down rather than endorsing a fallback recipe. Same coverage, no implicit recommendation.
Per PR #967 review, demonstrate three concrete gaps that the rightmost-untrusted XFF algorithm and ClientIPFromRemoteAddr don't catch: 1. v4-mapped IPv6 in XFF escapes a v4 trusted prefix (netip.Prefix.Contains is false for ::ffff:a.b.c.d vs a.b.c.0/N). Internal-address spoof and loopback spoof variants. 2. Zoned IPv6 in XFF escapes a v6 trusted prefix (netip.Prefix.Contains is false for any zoned address). 3. v4-mapped IPv6 RemoteAddr is returned in its v6 form, splitting one logical client across two rate-limit buckets / log keys / prefix checks depending on dual-stack listener configuration. These tests are EXPECTED TO FAIL on this commit; the follow-up commit will land the fixes (Unmap() + zone-strip at parse time). The previously discussed "empties don't shift TrustedProxies slot" property is not a bypass — current behavior is already correct — so it will be added as a regression-pin test alongside the fixes rather than committed here as failing. Co-authored-by: Cursor <cursoragent@cursor.com>
…ration Fixes three gaps surfaced by the PR #967 review (failing tests added in the previous commit now pass): 1. v4-mapped IPv6 in XFF escaping a v4 trusted prefix. netip.Prefix.Contains returns false for ::ffff:a.b.c.d checked against a.b.c.0/N, so an attacker able to inject the v4-mapped form of an internal address could escape the trust list. Fix: a single parseHeaderAddr helper Unmap()s before the prefix check and before storage. Applied uniformly in ClientIPFromHeader, the rightmost-untrusted XFF walk, and the ClientIPFromXFFTrustedProxies slot lookup. 2. Zoned IPv6 in XFF escaping a v6 trusted prefix. netip.Prefix.Contains returns false for any zoned IP, and zone identifiers carry no meaning across the network, so an attacker-injected zone suffix bypassed the trust check. Fix: parseHeaderAddr strips zones via WithZone("") at parse time; zones are by definition not valid in header-sourced IPs. 3. ClientIPFromRemoteAddr returning ::ffff:a.b.c.d for v4 clients on dual-stack listeners, splitting one logical client into two rate-limit / log / ACL keys. Fix: Unmap() before storing. Zone is preserved here because RemoteAddr can legitimately be link-local. Also: - Pre-canonicalize the user-supplied header name in ClientIPFromHeader at construction time, matching the realip.go pattern and saving a per-request canonicalization on the hot path. A package-level xForwardedForHeader constant documents the canonical XFF spelling. - Per-function godoc explicitly notes the v4-mapped folding and zone stripping so downstream callers understand the normalization contract. - New regression-pin tests: empties don't shift the ClientIPFromXFFTrustedProxies slot index; positive normalization tests for v4-mapped and zoned inputs in each entry point. - README: deprecate the RealIP row, add ClientIPFrom* entries, and introduce a "Choosing a ClientIP middleware" section with copy-paste recipes for the common deployments (direct internet, Cloudflare, CloudFront, dynamic-IP proxy pools). Co-authored-by: Cursor <cursoragent@cursor.com>
PR #967 Copilot review: ClientIPFromHeader uses r.Header.Get(header) to read the trusted single-IP header. Get returns only the FIRST value Go saw on the wire. If the trusted header reaches us with two values (a client-supplied "spoofed" plus a proxy-added "real"), we surface the client's value -- exactly the bypass shape this middleware was written to prevent. The trusted hop is the one CLOSEST to us in the chain, which is the LAST value, not the first. Same rightmost-untrusted spirit as ClientIPFromXFF: values from hops further from us are by definition less trustworthy than what our nearest hop produced. So the post-fix contract is: - Read r.Header.Values(header) (all instances). - Take the last entry. - parseHeaderAddr it; if it doesn't parse (garbage, empty), no IP is set -- do NOT fall back to earlier values (those came from less-trusted hops). TestClientIPFromHeader_MultiValueLastWins pins five cases: 1. single_value_unchanged (regression pin) 2. attacker_then_proxy -- today returns attacker's value 3. three_values_last_wins -- today returns attacker's value 4. last_unparseable_no_fallback -- today falls back to earlier value 5. last_empty_no_fallback -- today falls back to earlier value Cases 2-5 are EXPECTED TO FAIL on this commit; the follow-up flips ClientIPFromHeader from Get to Values()[len-1]. Co-authored-by: Cursor <cursoragent@cursor.com>
PR #967 Copilot review: the previous implementation read the trusted single-IP header with r.Header.Get(header), which returns only the first value. If the trusted header reaches us with multiple values -- a client-supplied "spoofed" entry plus the proxy's real entry, or two proxy-set entries because of upstream chain quirks -- we surfaced the attacker-controlled first value. Switch to r.Header.Values(header) and take the LAST entry. The last value is the one added by the hop closest to us in the chain, which is by definition the most trusted (the rightmost-untrusted principle we already apply to X-Forwarded-For). Fail-closed on garbage at the last position: we do NOT fall back to earlier values, as those came from less-trusted hops further from us. Correctly configured proxies (single value, proxy overwrites instead of appends) are unaffected -- len(values)==1, so first == last == that value. The four cases pinned in the previous commit's TestClientIPFromHeader_MultiValueLastWins now pass: - attacker_then_proxy (last value wins) - three_values_last_wins (last value wins) - last_unparseable_no_fallback (fail-closed) - last_empty_no_fallback (fail-closed) Existing ClientIPFromHeader tests stay green. Godoc updated with one short paragraph naming the multi-value contract. Co-authored-by: Cursor <cursoragent@cursor.com>
PR #967 Copilot review claimed walkXFF is O(n^2) in the number of XFF entries because it calls strings.LastIndexByte on progressively shorter substrings, and suggested a DoS-prone CPU cost on attacker-supplied large XFF headers. Benchmark refutes the claim. walkXFF is strictly O(M) in total chain length M -- linear in the entry count n, near-constant for the single-trusted-hop case (visitor returns true on the first entry). BenchmarkWalkXFF (visitor walks every entry, worst case for ClientIPFromXFF when all entries are inside trusted prefixes): n=1 5.6 ns/op n=10 57.6 ns/op (5.8 ns/entry) n=100 541.1 ns/op (5.4 ns/entry) n=1000 5228 ns/op (5.2 ns/entry) n=10000 55148 ns/op (5.5 ns/entry) BenchmarkWalkXFF_RightmostStop (visitor stops at the first entry, the common case for ClientIPFromXFF with no trusted prefixes and ClientIPFromXFFTrustedProxies(1)): n=1 5.2 ns/op n=10 6.0 ns/op n=100 6.1 ns/op n=1000 6.1 ns/op n=10000 6.1 ns/op -- truly constant Why the analysis is O(M): strings.LastIndexByte scans BACKWARD from the end of its input and stops at the first match. After we slice off the rightmost entry, the next call only scans up to the next comma -- O(L) where L is that entry's length. Sum over n iterations is O(n*L) = O(M). On arm64/amd64 it's also SIMD-optimized in the Go runtime, so a hand-rolled byte loop would lose, not win. Benchmark stays in the codebase as a regression pin: any future refactor that accidentally drops back to O(n^2) gets caught here. Zero allocations preserved across both shapes. Co-authored-by: Cursor <cursoragent@cursor.com>
…-continue PR #967 Copilot review: the godoc said "Calling with no arguments returns the rightmost parseable XFF IP", implying skip-and-continue past garbage. That phrasing predates the fail-closed change in 5dd2243; the actual behavior is now "try the rightmost; if it doesn't parse, no IP is set". Update the line to match. Co-authored-by: Cursor <cursoragent@cursor.com>
This comment was marked as resolved.
This comment was marked as resolved.
PR #967 fresh review: r.Header.Values(key) re-canonicalizes the key through textproto.CanonicalMIMEHeaderKey on every call. xForwardedForHeader is already a const in canonical form, so the canonicalization is redundant work in the request hot path. Replace r.Header.Values(xForwardedForHeader) with the direct map read r.Header[xForwardedForHeader] in both XFF middleware handlers. Safe because: - Headers received by net/http are stored under canonical keys. - Headers set programmatically via r.Header.Set/Add are also stored under canonical keys. - r.Header[non-canonical-key] only misses if user code bypasses the canonicalization via direct map writes; that's a test-only foot-gun and not the regime this middleware runs in. Per-lookup cost measured side by side on an M4 Pro: Header.Values 16.0 ns/op 0 B/op 0 allocs/op DirectMap 4.3 ns/op 0 B/op 0 allocs/op ~12 ns saved per request per XFF middleware on the hot path. Test suite (~70 tests) stays green; walkXFF semantics unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
PR #967 fresh review: the inline comment summarizing the XFF middleware said it "walks right-to-left, skipping trusted entries" but stopped short of mentioning the fail-closed-on-garbage contract that landed in 5dd2243. Add three words to match the godoc. Co-authored-by: Cursor <cursoragent@cursor.com>
PR #967 Copilot review: the godoc on this test still described the pre-fix behavior in present tense ("Today we surface values[0]"), even though a8e3dbf already switched the implementation to read r.Header.Values() and use the last entry. Reword the godoc to frame the test as a regression pin for the gap fixed in a8e3dbf -- past-tense bug description, present-tense post-fix contract. Same body, same test cases, same assertions. Co-authored-by: Cursor <cursoragent@cursor.com>
PR #967 Copilot review: the BenchmarkWalkXFF godoc referenced rightmostUntrustedXFF, an internal helper that was consolidated into the single walkXFF primitive back in 1863339 and no longer exists in the package. Rename the reference to ClientIPFromXFF (the public caller whose worst case this benchmark actually exercises). No code change. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks to Copilot and Claude for additional hardening of this PR. This PR is now ready for final human reviews & merge. ✅ |
|
@VojtechVitek one thing before mrg, middleware.realip itself is unchanged, i verified that x-forwarded-for 127.0.0.1 still spoofs r.remoteaddr, if these three gh get a "patched version", govulncheck & dependabot will mark every chi ver after that release as safe, even though code still using middleware.realip remains fully exploitable, suggest keeping them as "no fix, realip deprecated, migrate to clientipfrom" instead of assigning a patched ver |
|
@rezmoss I've marked |
…(#49) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [github.com/go-chi/chi/v5](https://github.com/go-chi/chi) | `v5.2.5` → `v5.3.0` |  |  | --- ### Release Notes <details> <summary>go-chi/chi (github.com/go-chi/chi/v5)</summary> ### [`v5.3.0`](https://github.com/go-chi/chi/releases/tag/v5.3.0) [Compare Source](go-chi/chi@v5.2.5...v5.3.0) #### What's Changed - Use strings.ReplaceAll where applicable by [@​JRaspass](https://github.com/JRaspass) in [#​1046](go-chi/chi#1046) - Propagate inline middlewares across mounted subrouters by [@​LukasJenicek](https://github.com/LukasJenicek) in [#​1049](go-chi/chi#1049) - add go 1.26 to ci by [@​pkieltyka](https://github.com/pkieltyka) in [#​1052](go-chi/chi#1052) - Remove last uses of io/ioutil by [@​JRaspass](https://github.com/JRaspass) in [#​1054](go-chi/chi#1054) - Simplify chi.walk with slices.Concat by [@​JRaspass](https://github.com/JRaspass) in [#​1053](go-chi/chi#1053) - Apply the stringscutprefix modernizer by [@​JRaspass](https://github.com/JRaspass) in [#​1051](go-chi/chi#1051) - Bump minimum Go to 1.23, always use request.Pattern by [@​JRaspass](https://github.com/JRaspass) in [#​1048](go-chi/chi#1048) - middleware: fix httpFancyWriter.ReadFrom double-counting bytes with Tee by [@​alliasgher](https://github.com/alliasgher) in [#​1085](go-chi/chi#1085) - Fix typo in Route doc comment by [@​gouwazi](https://github.com/gouwazi) in [#​1073](go-chi/chi#1073) - fix: set Request.Pattern from RoutePattern() by [@​leno23](https://github.com/leno23) in [#​1097](go-chi/chi#1097) - feat: middleware.ClientIP, a replacement for middleware.RealIP by [@​VojtechVitek](https://github.com/VojtechVitek) in [#​967](go-chi/chi#967) #### New Contributors - [@​LukasJenicek](https://github.com/LukasJenicek) made their first contribution in [#​1049](go-chi/chi#1049) - [@​alliasgher](https://github.com/alliasgher) made their first contribution in [#​1085](go-chi/chi#1085) - [@​gouwazi](https://github.com/gouwazi) made their first contribution in [#​1073](go-chi/chi#1073) - [@​leno23](https://github.com/leno23) made their first contribution in [#​1097](go-chi/chi#1097) #### SECURITY: middleware.ClientIP, a replacement for middleware.RealIP PR [#​967](go-chi/chi#967) introduced `middleware.ClientIP`, a replacement for `middleware.RealIP` that closes the three open spoofing advisories: - [GHSA-9g5q-2w5x-hmxf](GHSA-9g5q-2w5x-hmxf) — IP spoofing via XFF in `RemoteAddr` resolution (convto) - [GHSA-rjr7-jggh-pgcp](GHSA-rjr7-jggh-pgcp) — RealIP allows IP spoofing via unvalidated XFF (rezmoss) - [GHSA-3fxj-6jh8-hvhx](GHSA-3fxj-6jh8-hvhx) — IP spoofing in `middleware.RealIP` (Saku0512, Critical / 9.3) It also addresses issues outlined at: - [#​708](go-chi/chi#708) - <https://adam-p.ca/blog/2022/03/x-forwarded-for/> - [#​711](go-chi/chi#711) - [#​453](go-chi/chi#453) - [#​908](go-chi/chi#908) `middleware.RealIP` is deprecated in this PR with pointers to the new API. The deprecation only adds a `// Deprecated:` doc comment; the function keeps working for backward compatibility. ##### Why a new middleware (not "fix RealIP in place") `RealIP` has two unfixable design choices: it mutates `r.RemoteAddr`, and it tries to be a one-size-fits-all default by walking a hard-coded list of headers any client can supply. Per [adam-p's "The perils of the 'real' client IP"](https://adam-p.ca/blog/2022/03/x-forwarded-for/) (which calls chi out by name on this), there is no safe default — the user must pick their trust source explicitly. ##### The new API Four middlewares, two accessors. Pick exactly one middleware based on your infrastructure, read the result with one of the two accessors: ```go // One of the four. There is no safe default — pick exactly one. func ClientIPFromHeader(trustedHeader string) func(http.Handler) http.Handler func ClientIPFromXFF(trustedIPPrefixes ...string) func(http.Handler) http.Handler func ClientIPFromXFFTrustedProxies(numTrustedProxies int) func(http.Handler) http.Handler func ClientIPFromRemoteAddr(h http.Handler) http.Handler // Read the result. func GetClientIP(ctx context.Context) string // for logs, rate-limit keys func GetClientIPAddr(ctx context.Context) netip.Addr // for typed work ``` #### Example usage: ```go // Pick a single ClientIP middleware based on your deployment // Cloudflare. r.Use(middleware.ClientIPFromHeader("CF-Connecting-IP")) // Nginx with ngx_http_realip_module. r.Use(middleware.ClientIPFromHeader("X-Real-IP")) // Apache with mod_remoteip. r.Use(middleware.ClientIPFromHeader("X-Client-IP")) // AWS CloudFront, or any proxy fleet with known CIDRs. r.Use(middleware.ClientIPFromXFF( "13.32.0.0/15", // CloudFront IPv4 "52.46.0.0/18", // CloudFront IPv4 "2600:9000::/28", // CloudFront IPv6 )) // Behind exactly 2 trusted proxies with dynamic IPs (autoscaling pools, // ephemeral containers, dynamic CDN edges). r.Use(middleware.ClientIPFromXFFTrustedProxies(2)) // Server directly on the public internet, no proxy in front. r.Use(middleware.ClientIPFromRemoteAddr) ``` And in your handler or downstream middleware: ```go clientIP := middleware.GetClientIP(r.Context()) // log it, use it as a rate-limit key, etc. ``` *** Thanks to [@​adam-p](https://github.com/adam-p), [@​c2h5oh](https://github.com/c2h5oh), [@​rezmoss](https://github.com/rezmoss), [@​Saku0512](https://github.com/Saku0512), [@​convto](https://github.com/convto), [@​Dirbaio](https://github.com/Dirbaio), [@​jawnsy](https://github.com/jawnsy), [@​lrstanley](https://github.com/lrstanley), [@​mfridman](https://github.com/mfridman), [@​n33pm](https://github.com/n33pm), [@​pkieltyka](https://github.com/pkieltyka) for the prior discussions, detailed reviews, advisory reports, and test contributions that shaped this PR. **Full Changelog**: <go-chi/chi@v5.2.5...v5.3.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL21pbm9yIl19--> Reviewed-on: https://git.erwanleboucher.dev/eleboucher/apoci/pulls/49
|
@VojtechVitek could request cve plz |
A replacement for
middleware.RealIPthat closes the three open spoofing advisories against it:RemoteAddrresolution (convto)middleware.RealIP(Saku0512, Critical / 9.3)It also addresses issues outlined at:
middleware.RealIPis deprecated in this PR with pointers to the new API.The deprecation only adds a
// Deprecated:doc comment; the function keeps working for backward compatibility.Why a new middleware (not "fix RealIP in place")
RealIPhas two unfixable design choices: it mutatesr.RemoteAddr, and it tries to be a one-size-fits-all default by walking a hard-coded list of headers any client can supply. Per adam-p's "The perils of the 'real' client IP" (which calls chi out by name on this), there is no safe default — the user must pick their trust source explicitly.The new API
Four middlewares, two accessors. Pick exactly one middleware based on your
infrastructure, read the result with one of the two accessors:
Example usage:
And in your handler or downstream middleware:
Thanks to @adam-p, @c2h5oh, @rezmoss, @Saku0512, @convto, @Dirbaio, @jawnsy, @lrstanley, @mfridman, @n33pm, @pkieltyka for the prior discussions, detailed reviews, advisory reports, and test contributions that shaped this PR.