feat: add HTTP proxy support for web tools#587
Conversation
c5c9574 to
71c2669
Compare
|
hi @nayihz PR / Linter (pull_request)Failing after 33s Please fix that |
71c2669 to
c873db5
Compare
|
Done. PTAL. |
lxowalle
left a comment
There was a problem hiding this comment.
Maybe you can configure proxies separately for Brave, DuckDuckGo, and Perplexity to handle situations where proxies need to be selected based on the region.
nikolasdehor
left a comment
There was a problem hiding this comment.
This PR supersedes #447 (which I submitted) with a cleaner implementation. Reviewing as this is an area I am familiar with.
What works well:
- Single proxy config field at the tools.web level is the right granularity. Per-provider proxy configs (as lxowalle suggested) would add config complexity that most users do not need -- the common case is a single corporate/regional proxy for all outbound HTTP.
- createHTTPClient() centralizes the proxy + transport setup nicely. All three search providers and the fetch tool use it consistently.
- Proxy URL validation happens at client creation time via url.Parse, which will catch malformed URLs before any request is made.
- NewWebFetchToolWithProxy() preserves backward compatibility -- NewWebFetchTool() still works for callers that do not need proxy.
- Test coverage is thorough: config loading, proxy propagation to all providers, client creation with valid/invalid proxy URLs.
- Environment variable support (PICOCLAW_TOOLS_WEB_PROXY) is useful for containerized deployments.
Issues:
-
SOCKS5 proxy support is documented but not verified -- The PR description lists socks5:// as a supported format, but url.Parse + http.ProxyURL alone does not handle SOCKS5. Go's net/http Transport only handles HTTP/HTTPS proxies natively. For SOCKS5 you need golang.org/x/net/proxy or a custom dialer. If SOCKS5 support is intended, it needs either an explicit golang.org/x/net/proxy import and dialer setup, or documentation clarifying that only HTTP/HTTPS proxies are supported. If SOCKS5 is not a priority, just remove it from the description and add a comment in createHTTPClient() noting the limitation.
-
Proxy auth credentials in config file -- The description shows http://user:pass@proxy.example.com:8080 as a supported format. This means credentials could end up in config.json in plaintext. Consider adding a note in the example config that PICOCLAW_TOOLS_WEB_PROXY env var is the recommended approach when auth is needed, to avoid credentials in version-controlled config files.
-
Minor: shell_test.go changes are unrelated -- The 0755 to 0o755 and map[string]interface{} to map[string]any changes in shell_test.go are style cleanups unrelated to proxy support. Not blocking, but ideally these would be in a separate commit to keep the diff focused.
Overall this is solid work. The core proxy plumbing is correct and well-tested. The SOCKS5 claim in the description is the main item to address -- either implement it or document the limitation.
nikolasdehor
left a comment
There was a problem hiding this comment.
Re-reviewed after the latest push. The proxy plumbing is clean — createHTTPClient centralizes the transport setup, all search providers and WebFetchTool propagate the proxy correctly, and the test coverage is thorough (proxy config loading, invalid URL, provider propagation for all 3 search backends).
Remaining items from original review:
-
SOCKS5 support — http.ProxyURL only supports HTTP/HTTPS proxies. Users behind SOCKS5 proxies (common in corporate environments and in China where this project has significant adoption) will get a silent failure. Consider validating the proxy scheme and returning a clear error for unsupported schemes.
-
Proxy credentials in config — The proxy field accepts a URL which can contain credentials (http://user:pass@proxy:8080). These will appear in log output if the config is ever logged. Worth documenting that users should use environment-based proxy auth (HTTP_PROXY env var) for credentials, or at minimum redact the URL in any log output. Not a blocker but worth a comment in config.example.json.
-
Unrelated shell_test.go changes — The 0755 to 0o755 and interface{} to any changes are cosmetic cleanup that should go in a separate PR. This was flagged in the original review and is still present.
The core proxy implementation is solid. Item 3 is the easiest to address (just revert that file). Items 1 and 2 are suggestions for hardening.
3630222 to
2e3ca28
Compare
2e3ca28 to
76f2b42
Compare
|
Thanks for your review.
Actually, |
There was a problem hiding this comment.
LGTM! The implementation is solid:
- ✅ Native SOCKS5 support (Go 1.9+) is correctly implemented
- ✅ createHTTPClient centralizes proxy + transport setup
- ✅ Test coverage is thorough
- ✅ Environment variable support (PICOCLAW_TOOLS_WEB_PROXY)
- ✅ Backward compatible with NewWebFetchTool()
Note for future: CheckRedirect configuration could be moved into createHTTPClient for better organization, but this is not blocking.
|
@nayihz Proxy support for web tools is a practical addition, especially for users behind restricted networks. Covering HTTP, SOCKS5, and auth variants makes it flexible out of the box. We have a PicoClaw Dev Group on Discord where contributors share ideas and collaborate. If you're interested, drop an email to |
feat: add HTTP proxy support for web tools
Add HTTP proxy configuration for web search and web fetch tools to enable users in restricted network environments to access external services.
Changes:
proxyfield to WebToolsConfig in configConfiguration example:
{
"tools": { "web": { "proxy": "http://127.0.0.1:7890" } } }
Supported proxy formats:
📝 Description
🗣️ Type of Change
🤖 AI Code Generation
🔗 Related Issue
📚 Technical Context (Skip for Docs)
🧪 Test Environment
📸 Evidence (Optional)
Click to view Logs/Screenshots
☑️ Checklist