Skip to content

@openclaw/googlechat cert fetch fails: SSRF guard's per-request dispatcher bypasses NemoClaw's nemoclaw-http-proxy-fix.js global-agent hook #4687

@RandyAkerman

Description

@RandyAkerman

Summary

The @openclaw/googlechat plugin's JWT verification step fails every time inside a NemoClaw sandbox, with the gateway log line:

[googlechat] [default] Google Chat webhook auth rejected: Failed to retrieve verification certificates: fetch failed

This blocks every inbound Google Chat message — the bot receives Google's signed POST, tries to verify the JWT, can't fetch the signing certs, and rejects the request before replying. Reproducible across NemoClaw v0.0.56 with OpenClaw 2026.5.22, and the same code path exists unchanged in @openclaw/googlechat@2026.5.28 (current npm latest).

Reproduction

  1. Install @openclaw/googlechat into a NemoClaw sandbox (any version in 2026.5.22–2026.5.28).
  2. Configure the channel with a valid service account, an audience URL, and route Google Chat at the registered webhook URL.
  3. Send a real message from Google Chat to the bot. Watch /sandbox/gateway-manual.log (or wherever the gateway logs go).

Result: every inbound message yields Failed to retrieve verification certificates: fetch failed. The bot never replies.

Root cause

The plugin's createGoogleAuthFetch (in dist/api-*.js) wraps every Google API request — including the JWT signing-cert fetch from https://www.googleapis.com/service_accounts/v1/metadata/x509/chat@system.gserviceaccount.com — in fetchWithSsrFGuard from openclaw/plugin-sdk/ssrf-runtime. The dispatcher policy is picked by resolveGoogleAuthDispatcherPolicy in this priority order:

  1. googleAuthInit.proxy or agent set → mode: "explicit-proxy"
  2. process.env.HTTPS_PROXY set → mode: "env-proxy"
  3. TLS options → mode: "direct"
  4. Else → no dispatcherPolicy field (default fetch)

In all of 1, 2, and 3, the SSRF guard creates a per-request Agent / Dispatcher that bypasses the Node global agent — which is exactly where NemoClaw's nemoclaw-http-proxy-fix.js injects its proxy routing (loaded via NODE_OPTIONS=--require /tmp/nemoclaw-http-proxy-fix.js). The result:

Env Effective code path Outcome
HTTPS_PROXY=http://10.200.0.1:3128 plugin → SSRF guard → gaxios proxy-agent → L7 proxy CONNECT HTTP 403 policy_denied
(unset) plugin → SSRF guard → direct dispatcher (no NemoClaw hook) fetch failed (TCP)
(unset) bare gaxios.request() via Gaxios default fetch HTTP 200 OK
(unset) plain fetch(url) via Node global agent HTTP 200 OK

So the SSRF guard is doing the right thing in principle (defense-in-depth via dispatcher pinning) but in the NemoClaw runtime environment it actively undoes the host's proxy enforcement.

I have full timestamps from a single host:

  • 20:12:22, 20:17:14, 21:25:21, 21:27:08 — 4 consecutive failures across 2 gateway restarts (one with HTTPS_PROXY set, one without).

Fix shapes

Any of these would resolve it; (a) is the smallest:

(a) fetchWithSsrFGuard should detect the host's globalDispatcher (e.g. via undici.getGlobalDispatcher()) and reuse it when no explicit proxy / agent is supplied, instead of falling back to a new direct dispatcher that bypasses the host's hook.

(b) Bypass the SSRF guard for the small set of Google identity hosts the plugin already hardcodes (googleapis.com/oauth2/v1/certs, googleapis.com/robot/v1/metadata/x509/, googleapis.com/service_accounts/v1/metadata/x509/chat@system.gserviceaccount.com). These URLs are constants in the plugin source; an explicit carve-out is reasonable.

(c) Have resolveGoogleAuthDispatcherPolicy emit a new mode (e.g. "global") that maps to "use the existing globalDispatcher unchanged" in fetchWithSsrFGuard, and pick it when no other shape applies.

Adjacent context

  • NemoClaw v0.0.56 is currently the latest; we're on it. NemoClaw 2026.5.22 ships patch-openclaw-chat-send.js (scripts/patch-openclaw-chat-send.js in the image, applied at Step 25), but that patch targets the send path. The receive path's cert fetch (this issue) is unaffected.
  • We had a local-sandbox patch (scripts/chat-fix-sandbox.sh in our IaC) that modified openclaw/dist/ssrf-*.js to use createHttp1EnvHttpProxyAgent when HTTPS_PROXY was set. That patch only affects the openclaw runtime's SSRF guard; the chat plugin's bundled gaxios and google-auth-library under /sandbox/.openclaw/extensions/googlechat/node_modules/ don't go through the patched path.

Happy to PR (a) if it's the preferred shape.

Metadata

Metadata

Assignees

Labels

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions