feat(web_fetch): proxy support with SSRF-safe HTTP CONNECT / SOCKS5 tunnelling#3581
Merged
Conversation
…unnelling
web_fetch intentionally bypasses netclient proxy settings (source comment
in netclient.go). This means users behind corporate firewalls or GFW
cannot use web_fetch to reach external sites (GitHub, npm, etc.), even
when they have a working proxy configured in [network].
Wire web_fetch to the existing [network] proxy configuration while
preserving full SSRF protection:
1. netclient.ResolveProxyURL — new exported function that resolves
ProxySpec to URL string (reuses existing customProxyURL logic)
2. webFetch.proxyURL — new field set during tool registration
3. ssrfGuardedClient(proxyURL) — enhanced to tunnel through proxy:
- HTTP/HTTPS proxy: dial proxy -> CONNECT with ORIGINAL hostname
(proxy resolves DNS remotely, essential for GFW users) -> tunnel
- IP literal targets still get SSRF-checked via net.ParseIP
- SOCKS5/SOCKS5h: Transport.Proxy (Go standard library handles it)
- No proxy: existing direct SSRF-safe behavior unchanged
4. ConfineWebFetch + boot.go plumbing for both workspace and fallback
paths
| File | Change |
|------|--------|
| internal/netclient/netclient.go | +ResolveProxyURL() |
| internal/tool/builtin/webfetch.go | webFetch.proxyURL, ssrfGuardedClient proxy tunnel |
| internal/tool/builtin/workspace.go | Workspace.ProxyURL |
| internal/tool/builtin/confine.go | +ConfineWebFetch() |
| internal/boot/boot.go | addBuiltins proxyURL param + fallback fix |
| internal/boot/boot_test.go | test signature update |
| internal/tool/builtin/web_fetch_proxy_test.go | new proxy tests |
- go build ./... compiles
- go test ./internal/tool/builtin/ PASS
- go test ./internal/boot/ PASS
- SSRF still blocks 169.254.169.254, 10.0.0.1, 192.168.1.1 through proxy
The SOCKS branch only set Transport.Proxy, so the proxy itself was dialed through the SSRF-checking directDialContext — a SOCKS proxy on a private/LAN address (anything but loopback) was refused by the guard. Route SOCKS through golang.org/x/net/proxy with a plain dialer to the trusted proxy, and keep the IP-literal target SSRF check in a wrapping DialContext, mirroring the HTTP CONNECT path. Adds tests for both the preserved block and the proxy-not-blocked case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased @ningqipeng's #3357 onto current main-v2 (resolving the boot/workspace conflicts from the bash-timeout changes) and fixed the SOCKS5 branch.
What it does
Wires
web_fetchto the[network]proxy config while preserving the SSRF guard:CONNECTwith the original hostname (proxy resolves DNS — needed behind the GFW). IP-literal targets stay SSRF-checked.Fix on top of the original
The original SOCKS branch only set
Transport.Proxy, so the proxy itself was dialed through the SSRF-checkingdirectDialContext— a SOCKS proxy on a private/LAN address (common; anything but loopback) was refused by the guard. This routes SOCKS throughgolang.org/x/net/proxywith a plain dialer to the trusted proxy and keeps the IP-literal target check in a wrappingDialContext, mirroring the HTTP CONNECT path. Added tests for both the preserved block and the proxy-not-blocked case.Testing
go test ./internal/tool/builtin ./internal/boot ./internal/netclientgreenTestSSRFBlocksPrivateTargetThroughSOCKS5,TestSOCKS5ProxyOnPrivateAddressNotSSRFBlockedCloses #3357
Original work by @ningqipeng.