feat(security): Add HTTP API security hooks for plugin scanning#6405
feat(security): Add HTTP API security hooks for plugin scanning#6405masterfung wants to merge 10 commits intoopenclaw:mainfrom
Conversation
Additional Comments (2)
The PR description calls out streaming limitations, but the code currently doesn’t invoke Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openresponses-http.ts
Line: 703:710
Comment:
[P0] Streaming `/v1/responses` never runs `http_response_sending`, so streaming responses bypass the new “response leak/exfil” hook entirely.
The PR description calls out streaming limitations, but the code currently doesn’t invoke `runHttpResponseSending` at all in the streaming path (only `http_request_received` is enforced). That means plugins can’t block/redact streaming output even at end-of-stream. If the intended behavior is “non-streaming only”, this is a security gap vs the stated goal of protecting the endpoint.
How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 430:476
Comment:
[P0] Streaming `/v1/chat/completions` bypasses `http_response_sending` (and can’t be blocked/redacted).
`http_response_sending` is only invoked in the non-streaming branch. In the SSE streaming branch, chunks are written directly from `onAgentEvent`/fallback without any hook call, so plugins cannot detect leaks or block/modify content for streamed responses. Given the PR’s goal (“protect HTTP API endpoints”), this is a meaningful coverage hole unless explicitly accepted and documented as such.
How can I resolve this? If you propose a fix, please make it concise. |
feb59f5 to
063c7a0
Compare
bee9ce5 to
e2be359
Compare
e2be359 to
8ad63df
Compare
Addressed Streaming Security Gap ✅Thanks for the thorough review @greptile-apps! The P0 concerns about streaming responses are valid and have been addressed: What we added:End-of-stream scanning (commit
// At end of stream:
if (hookRunner && httpCtx && accumulatedContent) {
hookRunner.runHttpResponseSending({
content: accumulatedContent,
isStreaming: true,
isFinalChunk: true,
}, httpCtx);
}Why audit-only for streaming?Blocking mid-stream would corrupt the SSE connection. Instead, plugins can:
This matches how other security gateways handle streaming (can't un-send bytes already transmitted). DocumentationAdded inline comments explaining this tradeoff in both Ready for review! Happy to address any other feedback. |
|
cc @steipete - would love your feedback on this security hooks feature when you have a moment! It closes a gap where HTTP API consumers bypass plugin hooks that protect messaging platforms. |
|
Perfect! I can see the implementation now. This is a solid approach to the streaming security challenge. ✅ What Works Well1. Proper content accumulation: // Content accumulated during streaming in both files
accumulatedText += content; // openresponses-http.ts
accumulatedContent += content; // openai-http.ts2. End-of-stream hook execution: 3. Clear documentation: SummaryThis is the right solution for streaming security - audit-mode scanning that:
The implementation properly balances security visibility with streaming performance. Security plugins can still detect and log issues for investigation, alert security teams, and take action on future requests. This addresses the P0 streaming gap concern effectively! 🎯 |
Why We Built This: Citadel Guard for OpenClaw 🛡️We're building Citadel Guard - an open-source security plugin that protects OpenClaw agents from prompt injection, jailbreaks, and data exfiltration. The Problem We HitOur plugin works great for messaging platforms (Telegram, Discord, Slack) because all messages flow through plugin hooks: But when users hit the HTTP API directly ( Same OpenClaw instance, completely different security posture depending on how you access it. What This PR EnablesWith these 4 hooks, Citadel can protect ALL entry points:
Real-World Use CaseWe have the plugin ready and tested: citadel-guard-openclaw
Why This Matters for OpenClawSecurity plugins are a huge value-add for enterprise adoption. This PR makes OpenClaw a platform where the community can build security layers - not just Citadel, but rate limiters, PII redactors, compliance filters, etc. Happy to jump on a call to demo if that helps! 🙏 |
|
@greptileai can you re-review the pr? |
Additional Comments (2)
If an SSE payload includes a Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 54:56
Comment:
[P1] `writeSse` uses `JSON.stringify` directly; this can throw and break streaming responses.
If an SSE payload includes a `bigint` or a circular structure (even via nested event data), `JSON.stringify` will throw synchronously and can terminate the streaming response unexpectedly. Consider using the same kind of “safe stringify” approach used in `/tools/invoke` or otherwise ensuring SSE payloads are always JSON-serializable.
How can I resolve this? If you propose a fix, please make it concise.
For content parts shaped like Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 78:83
Comment:
[P1] Likely bug in `extractTextContent`: `type === "input_text"` branch checks `text` instead of `input_text`.
For content parts shaped like `{ type: "input_text", input_text: "..." }`, this currently returns empty because it does `typeof text === "string"`. That means request scanning and prompt building may miss user content for some clients.
How can I resolve this? If you propose a fix, please make it concise. |
|
cc @cpojer @shakkernerd @justinhuangcode love your takes on this PR. Simple, powerful, and secure. |
|
@greptileai review |
a4e5ef4 to
5f29e10
Compare
Additional Comments (4)
If you want to preserve these safely, consider normalizing arrays (e.g. join with Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 104:122
Comment:
[P2] `buildHttpContext` silently drops multi-value headers (string[]) which can matter for auditing (e.g. multiple `set-cookie`, repeated `x-forwarded-for`). Right now only `typeof value === "string"` is kept; arrays are ignored.
If you want to preserve these safely, consider normalizing arrays (e.g. join with `,` or keep first) rather than dropping them entirely.
How can I resolve this? If you propose a fix, please make it concise.
If the intended contract is “replacement”, you likely want Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/tools-invoke-http.ts
Line: 666:673
Comment:
[P1] `toolArgs` is computed once before hooks and then mutated via `Object.assign(toolArgs, preHook.modifiedParams)`. If a plugin returns a *replacement* params object (not just partial overrides), `Object.assign` will merge keys but won’t remove unsafe keys from the original args. This can undermine sanitization (e.g. plugin tries to drop `unsafe` but it remains).
If the intended contract is “replacement”, you likely want `toolArgs = preHook.modifiedParams` (and ensure it stays in sync with the object passed into `execute`).
How can I resolve this? If you propose a fix, please make it concise.
If the contract is “sanitized replacement”, consider replacing Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 182:185
Comment:
[P1] `modifiedRequestBody` is applied via `Object.assign(payload, ...)`, which can’t remove fields and may leave the original unsafe data present (same issue as tool params). If a plugin intends to *replace* the request body or delete fields, merging won’t achieve that.
If the contract is “sanitized replacement”, consider replacing `payload` entirely (or explicitly applying deletions) so plugins can actually drop unsafe keys.
How can I resolve this? If you propose a fix, please make it concise.
If you want deterministic auditing, consider awaiting the hook (with a short timeout) before ending, or explicitly documenting that this is best-effort only. Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 320:345
Comment:
[P3] End-of-stream `http_response_sending` hook is invoked asynchronously and its promise isn’t awaited before `res.end()`. That’s fine for “audit-only”, but it means the process could exit early (or tests could complete) before the hook finishes/logs, and hook errors won’t affect request handling.
If you want deterministic auditing, consider awaiting the hook (with a short timeout) before ending, or explicitly documenting that this is best-effort only.
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise. |
8299021 to
c641a54
Compare
All Code Review Issues Addressed ✅Round 1 Fixes:
Round 2 Fixes:
Additional Self-Review Fixes (preemptive):
All checks passing ✅ (lint, format, typecheck) @greptileai review |
Comprehensive Code Review Fixes - All Issues Addressed ✅P0 Fixes:
P1 Fixes:
P2 Fixes:
P3 (Documented):
Files changed: All checks ✅ | All tests ✅ (20/20) @greptileai review |
Additional Comments (1)
Also appears at src/gateway/tools-invoke-http.ts:828-835. Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/tools-invoke-http.ts
Line: 771:774
Comment:
[P1] `/tools/invoke` ignores plugin-provided status codes when blocking tool invoke/result.
`PluginHookHttpToolInvokeResult` and `PluginHookHttpToolResultResult` don’t currently carry a `blockStatusCode`, and the gateway hard-codes `sendJson(res, 400, ...)` on blocks (here and again around `postHook?.block`). This makes it impossible for security plugins to distinguish policy blocks (e.g. 403/422) from fail-closed outages (e.g. 503) in a consistent way with `http_request_received`/`http_response_sending`, which both support `blockStatusCode`.
Also appears at src/gateway/tools-invoke-http.ts:828-835.
How can I resolve this? If you propose a fix, please make it concise. |
Final Fixes - blockStatusCode Consistency ✅
Now all 4 HTTP hooks consistently support
Plugins can now specify 400/403/422/500/503 etc. consistently across all hooks. All checks ✅ | All tests ✅ @greptileai review |
Additional Comments (1)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! Prompt To Fix With AIThis is a comment left during a code review.
Path: src/gateway/tools-invoke-http.ts
Line: 727:755
Comment:
`PluginHookHttpContext` (and the header redaction/normalization loop) is built unconditionally here, before checking whether `getGlobalHookRunner()` returns anything. When HTTP hooks are disabled/no plugins register them, this adds avoidable per-request overhead (and differs from the other HTTP handlers in this PR, which only build ctx when a hook runner exists). Consider moving the `safeHeaders`/`httpCtx` construction inside `if (hookRunner) { ... }` so the `/tools/invoke` path stays as cheap as before when hooks aren’t in use.
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise. |
These issues are already fixed ✅@greptileai - The issues you flagged have already been addressed in commit 2b04f6967: P1:
P2:
All 4 HTTP hooks now have
Please re-review with latest code. @greptileai review |
SECURITY FIX: HTTP API endpoints were bypassing all plugin security hooks, creating a critical vulnerability where direct API consumers (SDKs, curl) had zero protection against prompt injection and data exfiltration attacks. Added 4 new plugin hooks: - http_request_received: Scan/block incoming requests before LLM processing - http_response_sending: Scan/block outgoing responses for data leaks - http_tool_invoke: Scan/block tool invocations via /tools/invoke - http_tool_result: Scan/block tool results for indirect injection Protected endpoints: - /v1/chat/completions (OpenAI-compatible API) - /v1/responses (OpenResponses API) - /tools/invoke (Direct tool invocation) Security features: - FAIL-CLOSED by default (hook errors/timeouts = block request) - Sensitive headers redacted from hook context - Request ID for correlation/audit logging - Content modification support for sanitization/redaction Tests: 13 new unit tests covering all hook types, error handling, and priority This enables security plugins like Citadel Guard to protect ALL traffic, not just messaging platform traffic (Telegram, Discord, Slack). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove unused imports (beforeEach, HookRunner) from http-hooks.test.ts - Replace `void` with `undefined` in return types to fix redundant-type-constituent warnings - Replace `string | unknown` with `unknown` (unknown already covers string) - Format docs and skills files Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
P0 Fixes: - Add safeStringify() to handle BigInt/circular refs in tools-invoke-http.ts Prevents JSON.stringify from crashing before hooks run - Document streaming response limitation in openai-http.ts Streaming bypasses http_response_sending hook (by design - buffering breaks UX) P1 Fixes: - Exclude function_call_output from user content scanning in openresponses-http.ts Tool output should be scanned via http_tool_result hook, not request hook Prevents false positives when tool results contain benign text Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Document that streaming responses bypass http_response_sending hook in openresponses-http.ts (matching the existing documentation in openai-http.ts). This is a known limitation - output scanning requires buffering which defeats the streaming UX. Plugins should use message_sending hook at the messaging layer for output scanning, or accept that streaming responses cannot be scanned. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements audit-only scanning for streaming HTTP responses: - Accumulate streamed content during SSE delivery - Run http_response_sending hook at stream completion - Log blocked responses for audit (cannot unsend already-streamed content) - Update security documentation to clarify streaming behavior This addresses the streaming bypass concern raised in code review. For real-time blocking, users should disable streaming (stream: false) or use message_sending hooks at the messaging layer. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
P1 fixes: - writeSse: Wrap JSON.stringify in try-catch to handle bigint/circular refs - extractTextContent: Fix input_text type to use inputText variable - Hook modifications: Use modifiedParams/modifiedResult/modifiedRequestBody when returned by plugins (enables sanitization use cases) P2 fixes: - Remove duplicate toolArgs computation in tools-invoke-http.ts - Add lint disable comment for required `any` cast Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
P0 fix: - Add http_response_sending hook for non-streaming /v1/responses (was missing, defeating response-side leak scanning) P2 fixes: - Wrap URL parsing in try-catch in openai-http.ts, openresponses-http.ts, and tools-invoke-http.ts to handle malformed request URLs - Wrap JSON.stringify in try-catch in writeSseEvent (openresponses-http.ts) to match writeSse pattern and prevent streaming crashes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
P0: - Add http_response_sending hook for non-streaming /v1/responses P1 fixes: - modifiedRequestBody: Use full replacement (clear + assign) instead of merge to allow plugins to delete/redact keys (openai-http, openresponses-http) - modifiedParams: Same full replacement pattern for tool args sanitization - blockStatusCode: Add to PluginHookHttpResponseSendingResult type and respect it in both response handlers (consistent with request-side) P2 fixes: - URL parsing: Wrap in try-catch across all 3 HTTP handlers - writeSseEvent: Add JSON.stringify try-catch (matches writeSse pattern) - buildHttpContext: Handle multi-value headers (join arrays with ", ") across all 3 HTTP handlers for proper auditing - Test assertion: Use regex to match both direct and wrapped error messages Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add blockStatusCode to http_response_sending merge function - Add blockStatusCode to PluginHookHttpToolInvokeResult type - Add blockStatusCode to PluginHookHttpToolResultResult type - Add blockStatusCode to tool hook merge functions - Use blockStatusCode in tools-invoke-http.ts for both preHook and postHook This ensures plugins can specify custom status codes (403/422/503) consistently across all 4 HTTP hooks. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
| if (hookRunner && httpCtx && accumulatedContent) { | ||
| hookRunner | ||
| .runHttpResponseSending( | ||
| { |
There was a problem hiding this comment.
[P1] In the streaming path, the end-of-stream http_response_sending invocation only uses the result for block logging, and ignores modifiedContent / modifiedResponseBody. That makes the hook’s “modifying” fields effectively no-ops for streaming (even in audit-only mode), which can surprise plugins that expect redaction/sanitization to be applied consistently.
If streaming can never apply modifications, it may be better to omit/ignore modification fields for streaming explicitly (or document it in the event/result) so plugin authors don’t assume they’ll take effect.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/gateway/openai-http.ts
Line: 533:536
Comment:
[P1] In the streaming path, the end-of-stream `http_response_sending` invocation only uses the result for `block` logging, and ignores `modifiedContent` / `modifiedResponseBody`. That makes the hook’s “modifying” fields effectively no-ops for streaming (even in audit-only mode), which can surprise plugins that expect redaction/sanitization to be applied consistently.
If streaming can never apply modifications, it may be better to omit/ignore modification fields for streaming explicitly (or document it in the event/result) so plugin authors don’t assume they’ll take effect.
How can I resolve this? If you propose a fix, please make it concise.Move HTTP context construction inside the `if (hookRunner)` block to avoid per-request overhead when no HTTP hooks are registered. This matches the pattern used in the other HTTP handlers in this PR. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2b04f69 to
39d473c
Compare
Performance fix applied ✅Issue: Fix: Moved Also rebased on latest upstream/main. @greptileai review |
bfc1ccb to
f92900f
Compare
This comment was marked as spam.
This comment was marked as spam.
|
This pull request has been automatically marked as stale due to inactivity. |
Summary
This PR adds 4 new plugin hooks to protect HTTP API endpoints from prompt injection and data exfiltration attacks.
Related Discussion: #6098
Previous PR: #6099 (closed for rebase)
Problem
OpenClaw's HTTP API endpoints currently bypass ALL plugin security hooks:
/v1/chat/completions/v1/responses/tools/invokeAny application using the API directly (SDKs, curl) has zero protection against:
Solution
Added 4 new plugin hooks:
http_request_receivedhttp_response_sendinghttp_tool_invokehttp_tool_resultSecurity Features
Changes
src/plugins/types.tssrc/plugins/hooks.tssrc/gateway/openai-http.ts/v1/chat/completionssrc/gateway/openresponses-http.ts/v1/responsessrc/gateway/tools-invoke-http.ts/tools/invokesrc/plugins/http-hooks.test.tsTotal: 651 lines added, 9 lines removed
Test Plan
npx vitest run src/plugins/http-hooks.test.ts # ✓ 13 tests passUsage Example
Security plugins can now register for HTTP hooks:
Real-World Plugin
This PR enables Citadel Guard, a security plugin that:
The plugin already has conditional hook registration to work with both current OpenClaw (messaging hooks) and this PR (HTTP hooks).
Known Limitations
Breaking Changes
None. New hooks are additive and don't affect existing behavior.
Greptile Overview
Greptile Summary
This PR introduces four new plugin hooks to apply security scanning/blocking to direct HTTP API consumers that previously bypassed plugin security coverage: request/response scanning for
/v1/chat/completionsand/v1/responses, plus tool param/result scanning for/tools/invoke. The gateway handlers now build a redacted HTTP context (headers/IP/requestId), invoke the appropriate hook runners in fail-closed mode (errors/blocks return 4xx/5xx), and support request-body replacement and response content modification in non-streaming paths. For streaming responses, it accumulates content and runs an end-of-stream audit hook (cannot retroactively block already-sent data). Hook runner support and unit tests were added in the plugins layer to execute these hooks sequentially with priority ordering and merged results.Confidence Score: 4/5
next.field ?? acc?.fieldpattern allows later (lower-priority) handlers to explicitly undo earlier decisions (e.g., unblock), which may be undesirable for security and could surprise plugin authors.