Skip to content

feat(hooks): add before_llm_call plugin hook for LLM input interception#39195

Closed
zeroaltitude wants to merge 1 commit intoopenclaw:mainfrom
zeroaltitude:feat/hook-before-llm-call
Closed

feat(hooks): add before_llm_call plugin hook for LLM input interception#39195
zeroaltitude wants to merge 1 commit intoopenclaw:mainfrom
zeroaltitude:feat/hook-before-llm-call

Conversation

@zeroaltitude
Copy link
Copy Markdown
Contributor

TL;DR

Adds the before_llm_call modifying plugin hook — fires before every LLM API call, allowing plugins to inspect/modify the message context, filter tools, or block the call.

Split from #33916 (1 of 3). This PR covers input interception only. Tool-call gating (after_llm_call) and output policies (before_response_emit) follow in stacked PRs.

Hook contract

Field Type Semantics
messages AgentMessage[] First-writer-wins: security plugins can't be overridden
systemPrompt string First-writer-wins
tools Array<{ name }> Intersection latch: later handlers can only narrow, never widen
block boolean One-way latch: once blocked, stays blocked
blockReason string First-writer-wins

Architecture

Uses the streamFn wrapper pattern — the hook runs inside streamAssistantResponse() in the agentLoop's own async context. This is the outermost wrapper, so it sees the final context after all other transforms (cache trace, Ollama compat, xAI decode, Anthropic logging).

BeforeLlmCallBlockError is a sentinel class that propagates through pi-agent-core's error handling and is caught gracefully in attempt.ts — the run ends cleanly with no assistant response rather than surfacing an error.

Infrastructure fix

Moves clearInternalHooks() from server-startup.ts to server.impl.ts (before plugin registration). Without this, plugin hooks registered during loadGatewayPlugins would be cleared when bundled hooks load later in startGatewaySidecars.

Testing

  • 7 stream wrapper tests (block, modify, pass-through, error handling)
  • 8 merge semantics tests (first-writer-wins, intersection latch, block latch)
  • pnpm check clean (pre-existing streamSegments type mismatch excluded)

AI-Assisted Development 🤖

Developed with AI assistance (Claude/OpenClaw agent "Tank"). All changes reviewed and directed by Eddie Abrams.

Adds the before_llm_call modifying hook that fires before every LLM API
call within the agent loop. Plugins can inspect/modify the message
context, filter the tool list, or block the call entirely.

Key design decisions:
- First-writer-wins for messages/systemPrompt (security plugins can't
  be overridden by lower-priority handlers)
- Intersection latch for tools (a later handler can only narrow, never
  widen, the tool allowlist)
- BeforeLlmCallBlockError sentinel class for graceful blocking without
  surfacing errors to the user
- streamFn wrapper pattern (outermost wrapper, same context as agentLoop)

Also moves clearInternalHooks() from server-startup.ts to server.impl.ts
so plugin hooks registered during loadGatewayPlugins are preserved when
bundled hooks load later.

Files:
- src/plugins/types.ts: BeforeLlmCallEvent/Result types
- src/plugins/hooks.ts: runBeforeLlmCall runner
- src/agents/pi-embedded-runner/run/hook-stream-wrapper.ts: streamFn wrapper
- src/agents/pi-embedded-runner/run/attempt.ts: wrapper setup + iteration tracking
- src/gateway/server-startup.ts, server.impl.ts: clearInternalHooks fix

Tests: 7 stream wrapper + 8 merge semantics = 15 tests
@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime agents Agent runtime and tooling size: L labels Mar 7, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 7, 2026

Greptile Summary

This PR introduces the before_llm_call plugin hook, allowing plugins to inspect/modify message context, filter tools, or block LLM calls before they are dispatched. It also fixes a startup ordering bug where clearInternalHooks() was called after plugin registration, silently discarding plugin hooks.

The core implementation — the wrapStreamFnWithHooks wrapper, the first-writer-wins / intersection-latch / one-way-block merge semantics in runBeforeLlmCall, and the clearInternalHooks relocation — are all well-structured and correctly implemented. Three issues are worth addressing before merging:

  • Iteration counter off-by-one (attempt.ts:1532): hookIterationRef starts at 0 and is incremented on turn_start, which fires before the LLM call. The first LLM call will see iteration: 1 in production, while the test contract assumes iteration: 0.
  • Fragile instanceof check via dynamic import (attempt.ts:1817): BeforeLlmCallBlockError already carries an isBeforeLlmCallBlock sentinel property. Relying on instanceof via a catch-block dynamic import can silently fail if the module is ever instantiated twice (bundler chunks, dual-package hazard), causing the block error to surface as a prompt failure instead of a graceful termination.
  • tokenEstimate declared but never populated (hook-stream-wrapper.ts:375, types.ts): The field is part of the public event type but is always undefined in practice, which is misleading for plugin authors building token-budget or rate-limiting plugins against it.

Confidence Score: 3/5

  • Mostly safe but has a potential off-by-one in iteration reporting and a fragile error-guard pattern that could cause silent misclassification of block events as run failures.
  • The architecture and merge semantics are sound. The clearInternalHooks fix is correct. However, two issues lower confidence: (1) the iteration counter is likely off-by-one between the test contract and production behavior due to turn_start firing before streamFn; (2) the dynamic-import instanceof check in the catch block is fragile — if the module is ever duplicated by a bundler, BeforeLlmCallBlockError would be mishandled as a real prompt error rather than a graceful block, which is a correctness regression in a security-sensitive path.
  • src/agents/pi-embedded-runner/run/attempt.ts — both flagged issues (off-by-one iteration and dynamic-import instanceof guard) live here.

Comments Outside Diff (1)

  1. src/agents/pi-embedded-runner/run/hook-stream-wrapper.ts, line 375-388 (link)

    tokenEstimate is defined in the event type but never populated

    PluginHookBeforeLlmCallEvent declares tokenEstimate?: number, but the call site here never computes or passes it — handlers will always see undefined. This makes the field misleading for plugin authors who might write token-budget or rate-limiting plugins against it (a natural use case given the field name).

    Either compute an estimate before invoking the hook (even a rough character-count heuristic) or remove the field from the type until the implementation is ready. If it's intentionally deferred to a follow-up PR, a // TODO comment here and in types.ts would help signal that intent.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/agents/pi-embedded-runner/run/hook-stream-wrapper.ts
    Line: 375-388
    
    Comment:
    **`tokenEstimate` is defined in the event type but never populated**
    
    `PluginHookBeforeLlmCallEvent` declares `tokenEstimate?: number`, but the call site here never computes or passes it — handlers will always see `undefined`. This makes the field misleading for plugin authors who might write token-budget or rate-limiting plugins against it (a natural use case given the field name).
    
    Either compute an estimate before invoking the hook (even a rough character-count heuristic) or remove the field from the type until the implementation is ready. If it's intentionally deferred to a follow-up PR, a `// TODO` comment here and in `types.ts` would help signal that intent.
    
    How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: 6fa757a

Comment on lines +1817 to +1818
const { BeforeLlmCallBlockError } = await import("./hook-stream-wrapper.js");
if (err instanceof BeforeLlmCallBlockError) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dynamic instanceof via catch-block import is fragile

BeforeLlmCallBlockError already carries the sentinel property isBeforeLlmCallBlock = true for exactly this situation. Using instanceof via a dynamic import in a catch block has a subtle failure mode: if a bundler or loader ever creates two separate module instances of hook-stream-wrapper.js (e.g., different chunks or a dual-package hazard), the class references won't be the same object and instanceof will return false. The block error would then fall through to the else branch, setting promptError and surfacing a run failure instead of the intended graceful termination.

The sentinel property is already there — prefer checking it directly:

Suggested change
const { BeforeLlmCallBlockError } = await import("./hook-stream-wrapper.js");
if (err instanceof BeforeLlmCallBlockError) {
const isBlockError =
err instanceof Error &&
(err as { isBeforeLlmCallBlock?: boolean }).isBeforeLlmCallBlock === true;
if (isBlockError) {

This removes the dynamic import entirely and is immune to the dual-instance problem.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1817-1818

Comment:
**Dynamic `instanceof` via catch-block import is fragile**

`BeforeLlmCallBlockError` already carries the sentinel property `isBeforeLlmCallBlock = true` for exactly this situation. Using `instanceof` via a dynamic import in a `catch` block has a subtle failure mode: if a bundler or loader ever creates two separate module instances of `hook-stream-wrapper.js` (e.g., different chunks or a dual-package hazard), the class references won't be the same object and `instanceof` will return `false`. The block error would then fall through to the `else` branch, setting `promptError` and surfacing a run failure instead of the intended graceful termination.

The sentinel property is already there — prefer checking it directly:

```suggestion
          const isBlockError =
            err instanceof Error &&
            (err as { isBeforeLlmCallBlock?: boolean }).isBeforeLlmCallBlock === true;
          if (isBlockError) {
```

This removes the dynamic import entirely and is immune to the dual-instance problem.

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +1532 to +1538
const hookIterationUnsub = hookRunner?.hasHooks("before_llm_call")
? activeSession.subscribe((event) => {
if (event.type === "turn_start") {
hookIterationRef.current++;
}
})
: undefined;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Iteration counter is likely off-by-one for the first LLM call

hookIterationRef starts at 0, and the subscription increments it on every turn_start event. In the agent loop, turn_start fires before the LLM call (streamFn) in each turn. This means:

  1. hookIterationRef.current = 0 (initial)
  2. turn_start fires → hookIterationRef.current = 1
  3. Wrapped streamFn is called → event.iteration = 1 ← first call

The tests in hook-stream-wrapper.test.ts manually set iterationRef.current = 0 for the first call and = 1 for the second, establishing the expected contract that the first LLM call maps to iteration: 0. In production the first call will be iteration: 1, and subsequent calls will be iteration: N+1 relative to test expectations.

If turn_start does fire before streamFn, this is an off-by-one. Two straightforward fixes:

  • Option A: Initialise hookIterationRef = { current: -1 } so that after the first turn_start increment the first LLM call sees 0.
  • Option B: Keep { current: 0 } and increment after the LLM call (e.g. on turn_end or llm_response).

It's worth verifying the turn_start / streamFn ordering in pi-agent-core to confirm the direction of the fix.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1532-1538

Comment:
**Iteration counter is likely off-by-one for the first LLM call**

`hookIterationRef` starts at `0`, and the subscription increments it on every `turn_start` event. In the agent loop, `turn_start` fires **before** the LLM call (`streamFn`) in each turn. This means:

1. `hookIterationRef.current = 0` (initial)
2. `turn_start` fires → `hookIterationRef.current = 1`
3. Wrapped `streamFn` is called → `event.iteration = 1` ← first call

The tests in `hook-stream-wrapper.test.ts` manually set `iterationRef.current = 0` for the first call and `= 1` for the second, establishing the expected contract that the first LLM call maps to `iteration: 0`. In production the first call will be `iteration: 1`, and subsequent calls will be `iteration: N+1` relative to test expectations.

If `turn_start` does fire before `streamFn`, this is an off-by-one. Two straightforward fixes:

- **Option A:** Initialise `hookIterationRef = { current: -1 }` so that after the first `turn_start` increment the first LLM call sees `0`.
- **Option B:** Keep `{ current: 0 }` and increment *after* the LLM call (e.g. on `turn_end` or `llm_response`).

It's worth verifying the `turn_start` / `streamFn` ordering in `pi-agent-core` to confirm the direction of the fix.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6fa757a11b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1818 to +1820
if (err instanceof BeforeLlmCallBlockError) {
log.info(`LLM call blocked by before_llm_call hook: ${err.message}`);
} else {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid reusing prior assistant output when hook blocks call

When before_llm_call blocks, this branch only logs and leaves promptError unset, so the attempt is treated as successful. In that path assistantTexts stays empty, and payload building falls back to lastAssistant from session history (src/agents/pi-embedded-runner/run/payloads.ts:195-251), which can resend a stale prior assistant reply if the conversation already had one. That turns a blocked call into incorrect user-visible output instead of a clean no-response outcome.

Useful? React with 👍 / 👎.

Comment thread src/plugins/types.ts
"subagent_ended",
"gateway_start",
"gateway_stop",
"before_llm_call",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Enforce allowPromptInjection for before_llm_call hooks

This introduces before_llm_call as a fully valid typed hook, but it is not covered by the prompt-injection policy gate. registerTypedHook only enforces plugins.entries.<id>.hooks.allowPromptInjection=false for names in isPromptInjectionHookName (src/plugins/registry.ts:519), and that set still excludes before_llm_call (src/plugins/types.ts:380-383), so a plugin can still mutate messages/systemPrompt through this hook even when operators explicitly disable prompt injection for that plugin.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling gateway Gateway runtime size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant