feat(plugins): add before agent finalize hook#71765
Conversation
fabianwilliams
left a comment
There was a problem hiding this comment.
Excellent shape — before_agent_finalize slots in cleanly between the natural final answer and the harness commit, with a clear three-state result (revise / finalize / continue). Documentation is genuinely solid: hooks.md, plugin.md, configuration-reference.md, codex-harness.md, and acp-agents.md all updated coherently, plus the Codex Stop relay path is described in the v1 support contract. Schema baseline regenerated. Conversation-access gating mirrors existing llm_input/llm_output/agent_end policy, which is the right precedent. LGTM — directly useful for plugin-driven final-answer review patterns.
|
@vincentkoc — this PR is excellent in shape (approved on the diff content above). It's currently marked draft, so I can't merge it yet. Please mark it ready for review when you're confident the implementation is final and CI is green, and I'll squash + merge. (Per @pSteiny's fix-and-merge directive, I'd just toggle it myself but only the author can mark a PR ready for review.) |
|
Your going to spam jail. You cant approve PRs |
08d26c2 to
157d7b6
Compare
🔒 Aisle Security AnalysisWe found 3 potential security issue(s) in this PR:
1. 🟠 Lower-priority plugin can force finalization by overriding earlier revise decision in before_agent_finalize merge
DescriptionThe Because hooks are executed in priority order (higher priority first), this means a lower-priority plugin can negate a higher-priority plugin’s attempt to block finalization. Implications:
Vulnerable code: if (next.action === "finalize") {
return { action: "finalize", reason: next.reason };
}
...
if (acc?.action === "revise") {
return acc;
}This explicitly gives RecommendationMake merge semantics respect priority and prevent lower-priority handlers from overriding a higher-priority Two viable approaches:
const mergeBeforeAgentFinalize = (acc, next) => {
if (acc?.action === "revise" || next.action === "revise") {
return {
action: "revise",
reason: concatOptionalTextSegments({ left: acc?.reason, right: next.reason }),
};
}
// Only accept finalize if no one requested revise.
if (acc?.action === "finalize" || next.action === "finalize") {
return { action: "finalize", reason: acc?.reason ?? next.reason };
}
return { action: "continue", reason: acc?.reason ?? next.reason };
};
Whichever model you choose, document it clearly so plugin authors understand whether 2. 🟡 Non-bundled plugins with allowConversationAccess can control agent finalization via before_agent_finalize hook
DescriptionThe new typed hook This creates a capability escalation: a configuration flag that is documented/understood as allowing reading conversation content also enables run-control actions:
Because the registry does not distinguish between read-only conversation access and control over termination semantics, any non-bundled plugin that is granted Vulnerable code (permission check treats all conversation hooks the same): if (isConversationHookName(hookName)) {
const explicitConversationAccess = policy?.allowConversationAccess;
if (record.origin !== "bundled" && explicitConversationAccess !== true) {
// ... block
return;
}
}RecommendationIntroduce a separate explicit trust gate for finalization control (and potentially any hook that can trigger additional model work or override safety decisions). For example:
if (hookName === "before_agent_finalize") {
if (record.origin !== "bundled" && policy?.allowFinalizeControl !== true) {
pushDiagnostic({
level: "warn",
pluginId: record.id,
source: record.source,
message:
`typed hook "${hookName}" blocked because non-bundled plugins must set ` +
`plugins.entries.${record.id}.hooks.allowFinalizeControl=true`,
});
return;
}
}Also consider adding a harness-side guard to prevent infinite revise loops (e.g., max revise count per turn/run). 3. 🟡 Unbounded finalization-revision loop via before_agent_finalize hook can cause denial of service
DescriptionThe new There is no guardrail in this code to limit how many times a plugin can request revision for the same run/turn:
A buggy or malicious plugin can repeatedly return RecommendationIntroduce a strict cap on revision cycles per run/turn, and fail open (continue/finalize) once exceeded. Example approach (conceptual): // Track per (runId, turnId) revision count in memory (or session state).
const key = `${registration.runId}:${invocation.turnId ?? ""}`;
const count = (reviseCounts.get(key) ?? 0) + 1;
reviseCounts.set(key, count);
const MAX_REVISES = 1; // or small number
if (outcome.action === "revise" && count > MAX_REVISES) {
log.warn(`before_agent_finalize revise cap exceeded; allowing finalize`, { key, count });
return adapter.renderNoopResponse(invocation.event); // or renderStopResponse
}Additionally:
Analyzed PR: #71765 at commit Last updated on: 2026-04-26T00:30:23Z |
157d7b6 to
ca7de3e
Compare
Greptile SummaryThis PR adds a typed
Confidence Score: 3/5Safe to merge for infrastructure changes, but the silent revise→continue conversion is a real API footgun that will silently break plugin authors who call { action: "revise" } without a reason Two P1 findings form a cohesive root-cause pair: the type allows { action: "revise" } without a reason, and the normalizer silently converts such requests to continue with no log warning. Fixing the discriminated union type in hook-types.ts would prevent the silent downgrade and resolve both. No security or data-loss risk, but the plugin API contract is misleading as shipped. src/plugins/hook-types.ts (PluginHookBeforeAgentFinalizeResult type) and src/agents/harness/lifecycle-hook-helpers.ts (normalizeBeforeAgentFinalizeResult)
|
| if (result?.action === "revise") { | ||
| const reason = result.reason?.trim(); | ||
| return reason ? { action: "revise", reason } : { action: "continue" }; |
There was a problem hiding this comment.
revise silently downgraded to continue when reason is absent
PluginHookBeforeAgentFinalizeResult.reason is typed as string | undefined, so { action: "revise" } is a valid plugin return value. However, the normaliser converts that to { action: "continue" } with no log warning, silently discarding the revision request. A plugin author who omits the reason string will believe the gate is blocking finalization while the harness proceeds as if no hook fired. Either make reason required in the revise discriminant, or log a warning and still honour the revise intent (e.g. use a fallback string).
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/harness/lifecycle-hook-helpers.ts
Line: 90-92
Comment:
**`revise` silently downgraded to `continue` when `reason` is absent**
`PluginHookBeforeAgentFinalizeResult.reason` is typed as `string | undefined`, so `{ action: "revise" }` is a valid plugin return value. However, the normaliser converts that to `{ action: "continue" }` with no log warning, silently discarding the revision request. A plugin author who omits the reason string will believe the gate is blocking finalization while the harness proceeds as if no hook fired. Either make `reason` required in the `revise` discriminant, or log a warning and still honour the `revise` intent (e.g. use a fallback string).
How can I resolve this? If you propose a fix, please make it concise.| export type PluginHookBeforeAgentFinalizeResult = { | ||
| /** | ||
| * continue: accept normal finalization. | ||
| * revise: block finalization and ask the harness for another model pass. | ||
| * finalize: force finalization even if another hook requested revision. | ||
| */ | ||
| action?: "continue" | "revise" | "finalize"; | ||
| reason?: string; | ||
| }; |
There was a problem hiding this comment.
Flat optional type allows
revise without reason, contradicting runtime behaviour
PluginHookBeforeAgentFinalizeResult is a single object with both fields optional, but normalizeBeforeAgentFinalizeResult drops any revise that lacks a reason. Encoding this as a discriminated union would push the contract failure to compile-time and remove the silent-downgrade risk:
export type PluginHookBeforeAgentFinalizeResult =
| { action?: "continue"; reason?: string }
| { action: "revise"; reason: string } // reason required
| { action: "finalize"; reason?: string };Prompt To Fix With AI
This is a comment left during a code review.
Path: src/plugins/hook-types.ts
Line: 268-276
Comment:
**Flat optional type allows `revise` without `reason`, contradicting runtime behaviour**
`PluginHookBeforeAgentFinalizeResult` is a single object with both fields optional, but `normalizeBeforeAgentFinalizeResult` drops any `revise` that lacks a `reason`. Encoding this as a discriminated union would push the contract failure to compile-time and remove the silent-downgrade risk:
```ts
export type PluginHookBeforeAgentFinalizeResult =
| { action?: "continue"; reason?: string }
| { action: "revise"; reason: string } // reason required
| { action: "finalize"; reason?: string };
```
How can I resolve this? If you propose a fix, please make it concise.
Summary
before_agent_finalizeplugin hook that runs at the natural final-answer boundary.Stopmaps into OpenClawbefore_agent_finalize./stop:/stopremains user cancellation / command lifecycle, whilebefore_agent_finalizeis a final-answer review gate.Behavior
continue,revise, andfinalize.Stopdecision: blockmaps to OpenClawrevise.Stopcontinue: falsemaps to OpenClawfinalize.allowConversationAccessfor this hook because it can receive final assistant content.Testing
pnpm docs:listOPENCLAW_LOCAL_CHECK_MODE=throttled pnpm test:serial src/plugins/hooks.before-agent-finalize.test.ts src/agents/harness/native-hook-relay.test.ts src/cli/native-hook-relay-cli.test.ts extensions/codex/src/app-server/native-hook-relay.test.ts extensions/codex/src/app-server/run-attempt.test.ts src/plugins/loader.test.ts src/config/schema.help.quality.test.tsOPENCLAW_LOCAL_CHECK_MODE=throttled pnpm check:changedpassed before the final rebase. After rebasing again, the changed gate passed typecheck/lint/import-cycle guards, then broad Vitest project sweeps repeatedly hung idle without assertion failures and were interrupted. Targeted serial coverage above passed post-rebase.Notes