Skip to content

feat: add HTTP proxy support for web tools#587

Merged
yinwm merged 1 commit intosipeed:mainfrom
nayihz:feat_webtool_proxy
Feb 24, 2026
Merged

feat: add HTTP proxy support for web tools#587
yinwm merged 1 commit intosipeed:mainfrom
nayihz:feat_webtool_proxy

Conversation

@nayihz
Copy link
Contributor

@nayihz nayihz commented Feb 21, 2026

Add HTTP proxy configuration for web search and web fetch tools to enable users in restricted network environments to access external services.

Changes:

  • Add proxy field to WebToolsConfig in config
  • Implement createHTTPClient() with proxy support
  • Add proxy field to all search providers (Brave, DuckDuckGo, Perplexity)
  • Add proxy field to WebFetchTool with NewWebFetchToolWithProxy()
  • Add logging to show when proxy is being used
  • Update config.example.json with proxy field example

Configuration example:
{
"tools": { "web": { "proxy": "http://127.0.0.1:7890" } } }

Supported proxy formats:

📝 Description

🗣️ Type of Change

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 📖 Documentation update
  • ⚡ Code refactoring (no functional changes, no api changes)

🤖 AI Code Generation

  • 🤖 Fully AI-generated (100% AI, 0% Human)
  • 🛠️ Mostly AI-generated (AI draft, Human verified/modified)
  • 👨‍💻 Mostly Human-written (Human lead, AI assisted or none)

🔗 Related Issue

📚 Technical Context (Skip for Docs)

  • Reference URL:
  • Reasoning:

🧪 Test Environment

  • Hardware:
  • OS:
  • Model/Provider:
  • Channels:

📸 Evidence (Optional)

Click to view Logs/Screenshots

☑️ Checklist

  • My code/docs follow the style of this project.
  • I have performed a self-review of my own changes.
  • I have updated the documentation accordingly.

@nayihz nayihz force-pushed the feat_webtool_proxy branch from c5c9574 to 71c2669 Compare February 21, 2026 07:40
@yinwm
Copy link
Collaborator

yinwm commented Feb 21, 2026

hi @nayihz

PR / Linter (pull_request)Failing after 33s

Please fix that

@nayihz nayihz force-pushed the feat_webtool_proxy branch from 71c2669 to c873db5 Compare February 22, 2026 00:41
@nayihz
Copy link
Contributor Author

nayihz commented Feb 22, 2026

Done. PTAL.

Copy link
Collaborator

@lxowalle lxowalle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you can configure proxies separately for Brave, DuckDuckGo, and Perplexity to handle situations where proxies need to be selected based on the region.

Copy link

@nikolasdehor nikolasdehor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR supersedes #447 (which I submitted) with a cleaner implementation. Reviewing as this is an area I am familiar with.

What works well:

  • Single proxy config field at the tools.web level is the right granularity. Per-provider proxy configs (as lxowalle suggested) would add config complexity that most users do not need -- the common case is a single corporate/regional proxy for all outbound HTTP.
  • createHTTPClient() centralizes the proxy + transport setup nicely. All three search providers and the fetch tool use it consistently.
  • Proxy URL validation happens at client creation time via url.Parse, which will catch malformed URLs before any request is made.
  • NewWebFetchToolWithProxy() preserves backward compatibility -- NewWebFetchTool() still works for callers that do not need proxy.
  • Test coverage is thorough: config loading, proxy propagation to all providers, client creation with valid/invalid proxy URLs.
  • Environment variable support (PICOCLAW_TOOLS_WEB_PROXY) is useful for containerized deployments.

Issues:

  1. SOCKS5 proxy support is documented but not verified -- The PR description lists socks5:// as a supported format, but url.Parse + http.ProxyURL alone does not handle SOCKS5. Go's net/http Transport only handles HTTP/HTTPS proxies natively. For SOCKS5 you need golang.org/x/net/proxy or a custom dialer. If SOCKS5 support is intended, it needs either an explicit golang.org/x/net/proxy import and dialer setup, or documentation clarifying that only HTTP/HTTPS proxies are supported. If SOCKS5 is not a priority, just remove it from the description and add a comment in createHTTPClient() noting the limitation.

  2. Proxy auth credentials in config file -- The description shows http://user:pass@proxy.example.com:8080 as a supported format. This means credentials could end up in config.json in plaintext. Consider adding a note in the example config that PICOCLAW_TOOLS_WEB_PROXY env var is the recommended approach when auth is needed, to avoid credentials in version-controlled config files.

  3. Minor: shell_test.go changes are unrelated -- The 0755 to 0o755 and map[string]interface{} to map[string]any changes in shell_test.go are style cleanups unrelated to proxy support. Not blocking, but ideally these would be in a separate commit to keep the diff focused.

Overall this is solid work. The core proxy plumbing is correct and well-tested. The SOCKS5 claim in the description is the main item to address -- either implement it or document the limitation.

Copy link

@nikolasdehor nikolasdehor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after the latest push. The proxy plumbing is clean — createHTTPClient centralizes the transport setup, all search providers and WebFetchTool propagate the proxy correctly, and the test coverage is thorough (proxy config loading, invalid URL, provider propagation for all 3 search backends).

Remaining items from original review:

  1. SOCKS5 support — http.ProxyURL only supports HTTP/HTTPS proxies. Users behind SOCKS5 proxies (common in corporate environments and in China where this project has significant adoption) will get a silent failure. Consider validating the proxy scheme and returning a clear error for unsupported schemes.

  2. Proxy credentials in config — The proxy field accepts a URL which can contain credentials (http://user:pass@proxy:8080). These will appear in log output if the config is ever logged. Worth documenting that users should use environment-based proxy auth (HTTP_PROXY env var) for credentials, or at minimum redact the URL in any log output. Not a blocker but worth a comment in config.example.json.

  3. Unrelated shell_test.go changes — The 0755 to 0o755 and interface{} to any changes are cosmetic cleanup that should go in a separate PR. This was flagged in the original review and is still present.

The core proxy implementation is solid. Item 3 is the easiest to address (just revert that file). Items 1 and 2 are suggestions for hardening.

@nayihz nayihz force-pushed the feat_webtool_proxy branch 2 times, most recently from 3630222 to 2e3ca28 Compare February 24, 2026 09:14
@nayihz nayihz force-pushed the feat_webtool_proxy branch from 2e3ca28 to 76f2b42 Compare February 24, 2026 09:17
@nayihz
Copy link
Contributor Author

nayihz commented Feb 24, 2026

Thanks for your review.

SOCKS5 support — http.ProxyURL only supports HTTP/HTTPS proxies.

Actually, http.ProxyURL supports http/https/socks5
Item 1/2/3 have been addressed. PLAT at your convenience.

Copy link
Collaborator

@yinwm yinwm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! The implementation is solid:

  • ✅ Native SOCKS5 support (Go 1.9+) is correctly implemented
  • ✅ createHTTPClient centralizes proxy + transport setup
  • ✅ Test coverage is thorough
  • ✅ Environment variable support (PICOCLAW_TOOLS_WEB_PROXY)
  • ✅ Backward compatible with NewWebFetchTool()

Note for future: CheckRedirect configuration could be moved into createHTTPClient for better organization, but this is not blocking.

@yinwm yinwm merged commit fd26fa7 into sipeed:main Feb 24, 2026
2 checks passed
@yinwm yinwm mentioned this pull request Feb 24, 2026
8 tasks
@Orgmar
Copy link
Contributor

Orgmar commented Feb 25, 2026

@nayihz Proxy support for web tools is a practical addition, especially for users behind restricted networks. Covering HTTP, SOCKS5, and auth variants makes it flexible out of the box.

We have a PicoClaw Dev Group on Discord where contributors share ideas and collaborate. If you're interested, drop an email to support@sipeed.com with the subject [Join PicoClaw Dev Group] nayihz and we'll send you the invite link.

This was referenced Feb 27, 2026
hyperwd pushed a commit to hyperwd/picoclaw that referenced this pull request Mar 5, 2026
feat: add HTTP proxy support for web tools
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants