Skip to content

fix(web): respect HTTP_PROXY/HTTPS_PROXY env vars in web_fetch#46538

Closed
BenediktSchackenberg wants to merge 2 commits intoopenclaw:mainfrom
BenediktSchackenberg:fix/proxy-support-web-tools
Closed

fix(web): respect HTTP_PROXY/HTTPS_PROXY env vars in web_fetch#46538
BenediktSchackenberg wants to merge 2 commits intoopenclaw:mainfrom
BenediktSchackenberg:fix/proxy-support-web-tools

Conversation

@BenediktSchackenberg
Copy link
Copy Markdown

Summary

web_fetch fails with getaddrinfo EAI_AGAIN when running behind an HTTP proxy because it uses fetchWithWebToolsNetworkGuard in strict mode, which creates a pinned DNS dispatcher that bypasses proxy environment variables.

The fix adds useEnvProxy: true to the web_fetch network guard call, matching the behavior already used by web_search (via withTrustedWebToolsEndpoint). This routes requests through the configured proxy (via undici's EnvHttpProxyAgent) when HTTP_PROXY/HTTPS_PROXY env vars are set, with NO_PROXY respected automatically.

What Changed

  • src/agents/tools/web-fetch.ts: Added useEnvProxy: true to the fetchWithWebToolsNetworkGuard call

Why web_search already worked

web_search uses withTrustedWebToolsEndpoint which internally sets useEnvProxy: true. Only web_fetch was missing this flag.

Change Type

  • Bug fix

Scope

  • src/agents/tools/web-fetch.ts

Security Impact

No security impact. The useEnvProxy flag is already used by web_search and other trusted endpoint fetchers. It delegates to undici's EnvHttpProxyAgent which respects NO_PROXY exclusions. The SSRF policy is still applied.

Fixes #46306

Copilot AI review requested due to automatic review settings March 14, 2026 19:45
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: XS labels Mar 14, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 14, 2026

Greptile Summary

This PR fixes web_fetch failing with getaddrinfo EAI_AGAIN in proxy environments by adding useEnvProxy: true to its fetchWithWebToolsNetworkGuard call. Previously, web_fetch defaulted to strict mode (pinned DNS dispatcher, bypassing OS proxy variables), while web_search already used withTrustedWebToolsEndpoint which internally sets useEnvProxy: true.

  • The one-line fix correctly mirrors the established proxy pattern used throughout the codebase (withTrustedWebToolsEndpoint, pi-embedded-runner/run/attempt.ts, etc.)
  • The SSRF guard (resolvePinnedHostnameWithPolicy) still runs a pre-flight DNS check before the proxy dispatches the request, preserving the security posture.
  • NO_PROXY exclusions are respected automatically via undici's EnvHttpProxyAgent, and useEnvProxy only activates the proxy path when hasProxyEnvConfigured() returns true — no behavioral change for non-proxy environments.
  • The SSRF policy for web_fetch intentionally does not include dangerouslyAllowPrivateNetwork, which is correct since URLs here are user-supplied (unlike web_search's trusted endpoints).

No issues found — the change is minimal, safe, and consistent with existing patterns.

Confidence Score: 5/5

  • This PR is safe to merge — it is a minimal, targeted bug fix with no security regressions.
  • The change is a single-line addition that aligns web_fetch with the well-established useEnvProxy pattern already used by web_search and other fetchers. The SSRF guard remains active, NO_PROXY is respected, and the fix only activates in environments where a proxy is explicitly configured. No new code paths are introduced.
  • No files require special attention.

Last reviewed commit: fb3e2d9

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts web_fetch networking behavior so outbound requests can honor HTTP_PROXY / HTTPS_PROXY environment variables when running behind an enterprise proxy, aligning it with other web tool fetch paths.

Changes:

  • Add useEnvProxy: true to the fetchWithWebToolsNetworkGuard call in web_fetch.

@@ -533,6 +533,7 @@ async function runWebFetch(params: WebFetchRuntimeParams): Promise<Record<string
url: params.url,
maxRedirects: params.maxRedirects,
timeoutSeconds: params.timeoutSeconds,
useEnvProxy: true,
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the DNS pinning tradeoff is real. In trusted_env_proxy mode we're deliberately delegating DNS resolution to the proxy, which means we can't pin locally. The assumption is that if you've configured an env proxy, you trust it to handle resolution. Added a comment in the code documenting this tradeoff.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed

The DNS pinning tradeoff is now documented in the code (see commit ). The comment at the call explains:

When a proxy is configured, DNS pinning is intentionally disabled because the proxy performs DNS resolution. This is a necessary tradeoff for proxy environments (e.g. corporate/K8s). SSRF protection is delegated to the proxy.

This pattern is consistent with , which uses (internally sets ) for the same reason. Implementing a separate proxy-respecting strict mode is beyond this PR's scope — the PR's explicit goal is to enable env proxy support for , which necessarily involves this tradeoff. A separate config option could be considered in a follow-up.

@openclaw-barnacle openclaw-barnacle Bot added the docs Improvements or additions to documentation label Mar 15, 2026
@BenediktSchackenberg BenediktSchackenberg force-pushed the fix/proxy-support-web-tools branch from 1e6688e to 016e27c Compare March 15, 2026 19:58
@openclaw-barnacle openclaw-barnacle Bot removed the docs Improvements or additions to documentation label Mar 15, 2026
@BenediktSchackenberg BenediktSchackenberg force-pushed the fix/proxy-support-web-tools branch from 7722b3a to 2a59fd6 Compare March 20, 2026 12:45
@openclaw-barnacle openclaw-barnacle Bot added the docs Improvements or additions to documentation label Mar 20, 2026
@BenediktSchackenberg BenediktSchackenberg force-pushed the fix/proxy-support-web-tools branch 4 times, most recently from b04db88 to 3a71aad Compare March 25, 2026 16:04
Deniable9570 pushed a commit to Deniable9570/openclaw that referenced this pull request Mar 26, 2026
web_fetch fails with getaddrinfo EAI_AGAIN behind HTTP proxy
because fetchWithWebToolsNetworkGuard uses strict mode with DNS
pinning, bypassing proxy env vars. Fix from openclaw#46306 root cause.

1. fetch-guard.ts: skip DNS pinning when routing through env
   proxy in TRUSTED_ENV_PROXY mode (proxy handles DNS)
2. web-fetch.ts: add useEnvProxy:true so web_fetch routes
   through configured env proxy

Test:
- fetch-guard.ssrf.test.ts: add DNS-pinning-skip test
- web-fetch.ssrf.test.ts: stub proxy env vars in beforeEach

Refs: openclaw#46306 openclaw#46318 openclaw#46538
@BenediktSchackenberg BenediktSchackenberg force-pushed the fix/proxy-support-web-tools branch from aef5142 to 0933269 Compare March 26, 2026 21:27
@openclaw-barnacle openclaw-barnacle Bot removed the docs Improvements or additions to documentation label Mar 26, 2026
Benedikt Schackenberg added 2 commits March 26, 2026 22:04
web_fetch was using fetchWithWebToolsNetworkGuard in strict mode,
which creates a pinned DNS dispatcher that bypasses env proxy settings.
Enable useEnvProxy so requests route through the configured proxy
(via undici EnvHttpProxyAgent) when HTTP_PROXY/HTTPS_PROXY are set.

web_search already worked correctly behind proxies because it uses
withTrustedWebToolsEndpoint which sets useEnvProxy: true.

Fixes openclaw#46306

AI-assisted: Built with Claude, reviewed by human.
Add code comment explaining that useEnvProxy intentionally disables DNS
pinning because the proxy performs DNS resolution. SSRF protection is
delegated to the proxy in these environments.
@obviyus
Copy link
Copy Markdown
Contributor

obviyus commented Mar 31, 2026

Closing as duplicate; this was superseded by #50650.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: XS

Projects

None yet

4 participants