feat(web-fetch): add allowPrivateNetwork config for web_fetch#130
Open
BingqingLyu wants to merge 4 commits into
Open
feat(web-fetch): add allowPrivateNetwork config for web_fetch#130BingqingLyu wants to merge 4 commits into
BingqingLyu wants to merge 4 commits into
Conversation
Add opt-in `tools.web.fetch.allowPrivateNetwork` config key (boolean, default `false`) so `web_fetch` can reach private/internal network addresses (localhost, 10.x, 192.168.x, etc.) when explicitly enabled. This unblocks agent architectures where agents call local services via `web_fetch`. The SSRF guard's `.strict()` schema previously rejected any unknown keys, so an upstream schema change was required. Changes: - Add optional `allowPrivateNetwork` option to `resolvePinnedHostname` - Thread `allowPrivateNetwork` through fetchWithRedirects -> runWebFetch - Add config key to zod schema, TypeScript types, and UI labels/help - Add changelog entry referencing openclaw#39604 - Add tests for allow and block paths Closes openclaw#39604
Address two review findings: 1. Always block cloud metadata (IMDS) endpoints even when allowPrivateNetwork is true. Maintain a separate denylist for 169.254.169.254, fd00:ec2::254, and metadata.google.internal that is enforced unconditionally — both for literal hostnames/IPs and for DNS resolution results. 2. Include allowPrivateNetwork in the web_fetch cache key so that toggling the flag within the same process immediately invalidates entries fetched under the permissive setting. Adds 2 new SSRF tests covering both IMDS scenarios.
Harden the IMDS denylist to handle IPv6-mapped forms of 169.254.169.254 (both dotted ::ffff:169.254.169.254 and hex ::ffff:a9fe:a9fe) that could bypass the set-based check. Extract isImdsAddress() helper that normalizes and re-checks mapped forms for both literal and DNS results. Add test for localhost access with allowPrivateNetwork: true.
…passes Replace string-set IMDS matching with canonical IPv6 group comparison via parseIpv6Groups(). This blocks expanded forms like fd00:ec2:0:0:0:0:0:254 that previously bypassed the compressed-form set entry fd00:ec2::254. Add test for expanded IPv6 IMDS variant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tools.web.fetch.allowPrivateNetworkconfig key (boolean, defaultfalse) soweb_fetchcan reach private/internal network addresses (localhost, 10.x, 192.168.x, 172.16-31.x) when explicitly enabled.Problem
web_fetchblocks all private/internal network addresses via the SSRF guard. There is no config-level way to opt in to private network access, blocking agent architectures where agents need to call local services:The
ToolsWebFetchSchemauses.strict(), so users cannot work around this without an upstream schema change.Changes
src/infra/net/ssrf.ts: Add optionaloptions.allowPrivateNetworkparameter toresolvePinnedHostname(). Whentrue, skips blocked-hostname and private-IP checks; pins private IP literals directly without DNS lookup.src/agents/tools/web-fetch.ts: AddresolveFetchAllowPrivateNetwork()resolver (defaultsfalse). ThreadallowPrivateNetworkthroughfetchWithRedirectsandrunWebFetchintoresolvePinnedHostname.src/config/zod-schema.agent-runtime.ts: AddallowPrivateNetwork: z.boolean().optional()toToolsWebFetchSchema.src/config/types.tools.ts: AddallowPrivateNetwork?: booleanto the fetch config type.src/config/schema.ts: Add label and help text for the new config key.src/agents/tools/web-fetch.ssrf.test.ts: Add two tests (allow private when enabled; block private when explicitly false). Fix mock to forward the newoptionsparameter.CHANGELOG.md: Add entry referencing [Feature]: Add tools.web.fetch.allowPrivateNetwork to allow private network access openclaw/openclaw#39604.User-visible changes
New config key:
When
true,web_fetchcan reach private/internal network addresses. Default remainsfalse(no behavior change for existing users).Security impact
false.Testing performed
pnpm lint-- 0 warnings, 0 errorspnpm build-- cleanpnpm test-- 5947 passed, 2 failed (pre-existing lobster subprocess timeout, unrelated)allowPrivateNetworkpath)Closes openclaw#39604