Skip to content

feat(web-fetch): add allowPrivateNetwork config for web_fetch#130

Open
BingqingLyu wants to merge 4 commits into
mainfrom
fork-pr-39630-feat-web-fetch-allow-private-network
Open

feat(web-fetch): add allowPrivateNetwork config for web_fetch#130
BingqingLyu wants to merge 4 commits into
mainfrom
fork-pr-39630-feat-web-fetch-allow-private-network

Conversation

@BingqingLyu

@BingqingLyu BingqingLyu commented Apr 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add opt-in tools.web.fetch.allowPrivateNetwork config key (boolean, default false) so web_fetch can reach private/internal network addresses (localhost, 10.x, 192.168.x, 172.16-31.x) when explicitly enabled.
  • Default behavior is unchanged: private networks remain blocked unless the operator explicitly opts in.

Problem

web_fetch blocks all private/internal network addresses via the SSRF guard. There is no config-level way to opt in to private network access, blocking agent architectures where agents need to call local services:

[security] blocked URL fetch (url-fetch) target=http://127.0.0.1:9090/api/...
reason=Blocked hostname or private/internal/special-use IP address

The ToolsWebFetchSchema uses .strict(), so users cannot work around this without an upstream schema change.

Changes

  • src/infra/net/ssrf.ts: Add optional options.allowPrivateNetwork parameter to resolvePinnedHostname(). When true, skips blocked-hostname and private-IP checks; pins private IP literals directly without DNS lookup.
  • src/agents/tools/web-fetch.ts: Add resolveFetchAllowPrivateNetwork() resolver (defaults false). Thread allowPrivateNetwork through fetchWithRedirects and runWebFetch into resolvePinnedHostname.
  • src/config/zod-schema.agent-runtime.ts: Add allowPrivateNetwork: z.boolean().optional() to ToolsWebFetchSchema.
  • src/config/types.tools.ts: Add allowPrivateNetwork?: boolean to the fetch config type.
  • src/config/schema.ts: Add label and help text for the new config key.
  • src/agents/tools/web-fetch.ssrf.test.ts: Add two tests (allow private when enabled; block private when explicitly false). Fix mock to forward the new options parameter.
  • CHANGELOG.md: Add entry referencing [Feature]: Add tools.web.fetch.allowPrivateNetwork to allow private network access openclaw/openclaw#39604.

User-visible changes

New config key:

"tools": {
  "web": {
    "fetch": { "allowPrivateNetwork": true }
  }
}

When true, web_fetch can reach private/internal network addresses. Default remains false (no behavior change for existing users).

Security impact

  • New capability: opt-in only, default false.
  • Risk: operators enabling this in untrusted environments could expose internal services.
  • Mitigation: explicit config required; secure default; same trust model as operator-controlled gateway config.

Testing performed

  • pnpm lint -- 0 warnings, 0 errors
  • pnpm build -- clean
  • pnpm test -- 5947 passed, 2 failed (pre-existing lobster subprocess timeout, unrelated)
  • SSRF test suite: 7/7 passed (including 2 new tests for the allowPrivateNetwork path)

Closes openclaw#39604

Add opt-in `tools.web.fetch.allowPrivateNetwork` config key (boolean,
default `false`) so `web_fetch` can reach private/internal network
addresses (localhost, 10.x, 192.168.x, etc.) when explicitly enabled.

This unblocks agent architectures where agents call local services via
`web_fetch`. The SSRF guard's `.strict()` schema previously rejected
any unknown keys, so an upstream schema change was required.

Changes:
- Add optional `allowPrivateNetwork` option to `resolvePinnedHostname`
- Thread `allowPrivateNetwork` through fetchWithRedirects -> runWebFetch
- Add config key to zod schema, TypeScript types, and UI labels/help
- Add changelog entry referencing openclaw#39604
- Add tests for allow and block paths

Closes openclaw#39604
Address two review findings:

1. Always block cloud metadata (IMDS) endpoints even when
   allowPrivateNetwork is true. Maintain a separate denylist for
   169.254.169.254, fd00:ec2::254, and metadata.google.internal
   that is enforced unconditionally — both for literal hostnames/IPs
   and for DNS resolution results.

2. Include allowPrivateNetwork in the web_fetch cache key so that
   toggling the flag within the same process immediately invalidates
   entries fetched under the permissive setting.

Adds 2 new SSRF tests covering both IMDS scenarios.
Harden the IMDS denylist to handle IPv6-mapped forms of 169.254.169.254
(both dotted ::ffff:169.254.169.254 and hex ::ffff:a9fe:a9fe) that
could bypass the set-based check. Extract isImdsAddress() helper that
normalizes and re-checks mapped forms for both literal and DNS results.

Add test for localhost access with allowPrivateNetwork: true.
…passes

Replace string-set IMDS matching with canonical IPv6 group comparison
via parseIpv6Groups(). This blocks expanded forms like
fd00:ec2:0:0:0:0:0:254 that previously bypassed the compressed-form
set entry fd00:ec2::254.

Add test for expanded IPv6 IMDS variant.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add tools.web.fetch.allowPrivateNetwork to allow private network access

2 participants