Skip to content

Compaction/Safeguard: add summary quality audit retries#25556

Merged
jalehman merged 13 commits intoopenclaw:mainfrom
rodrigouroz:codex/pr20038-04
Mar 5, 2026
Merged

Compaction/Safeguard: add summary quality audit retries#25556
jalehman merged 13 commits intoopenclaw:mainfrom
rodrigouroz:codex/pr20038-04

Conversation

@rodrigouroz
Copy link
Contributor

@rodrigouroz rodrigouroz commented Feb 24, 2026

Summary

Describe the problem and fix in 2–5 bullets:

  • Problem: compaction summary quality could still regress (missing required headings, dropped asks/identifiers), and retries had edge-case behavior gaps.
  • Why it matters: degraded summaries can lose continuity, while unsafe retry feedback formatting can weaken prompt trust boundaries.
  • What changed:
    • Added bounded quality audit + retry flow, then hardened it across follow-up commits.
    • Quality checks now use exact heading-line matching, policy-aware identifier enforcement, and stronger Unicode-aware latest-ask overlap checks.
    • Audit inputs are scoped to model-summarized content only (not verbatim preserved turns), and retry failures now keep the last successful summary instead of canceling compaction.
    • Reused shared prompt sanitization utilities by adding wrapUntrustedPromptDataBlock in src/agents/sanitize-for-prompt.ts and consuming it from compaction safeguard (no local sanitizer duplication).
    • Quality guard is now opt-in by default (qualityGuardEnabled defaults to false unless explicitly configured).
  • What did NOT change (scope boundary): no config-schema surface changes in this PR.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • Compaction quality retries are available but now disabled by default unless runtime enables qualityGuardEnabled.
  • When enabled, retries are bounded and keep the last successful summary if a retry attempt errors.

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: Node 22 + pnpm
  • Model/provider: n/a (unit tests)
  • Integration/channel (if any): n/a
  • Relevant config (redacted): compaction safeguard runtime defaults

Steps

  1. pnpm test src/agents/pi-extensions/compaction-safeguard.test.ts src/agents/sanitize-for-prompt.test.ts
  2. pnpm tsgo
  3. pnpm format

Expected

  • Updated safeguard and prompt-sanitization tests pass.
  • Typecheck passes with strict test fixture casts.
  • Formatting is clean.

Actual

  • Passed locally.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified quality audit semantics for sections, identifier-policy behavior, latest-ask overlap, and preserved-turn bypass protections.
  • Verified retry failure fallback keeps a usable summary.
  • Verified shared sanitizer wrapper is reused in compaction safeguard and covered by unit tests.
  • What you did not verify: production traffic/token cost impact under sustained load.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps:

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: keep qualityGuardEnabled unset/false, or revert this PR.
  • Files/config to restore: src/agents/pi-extensions/compaction-safeguard.ts, src/agents/sanitize-for-prompt.ts
  • Known bad symptoms reviewers should watch for: increased compaction latency only when quality guard is explicitly enabled.

Risks and Mitigations

List only real risks for this PR. Add/remove entries as needed. If none, write None.

  • Risk: when operators enable quality guard, retries can still increase latency/token usage.
    • Mitigation: strict retry clamp + fallback behavior on retry errors.
  • Risk: false-positive ask overlap checks in edge language mixes.
    • Mitigation: Unicode-aware tokenization + stopword filtering + stronger overlap thresholds + tests.

Stack: 4/9, depends on #25555.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

function extractOpaqueIdentifiers(text: string): string[] {
const matches =
text.match(
/([A-Fa-f0-9]{8,}|https?:\/\/\S+|\/[\w./-]+|[A-Za-z]:\\[\w\\.-]+|[A-Za-z0-9._-]+\.[A-Za-z0-9._/-]+:\d{1,5}|\b\d{6,}\b)/g,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

regex pattern \/[\w./-]+ will match single-char paths like /a or /x, which are likely false positives

the pattern intends to match Unix-style paths but lacks a minimum length constraint; consider requiring at least 2 path segments or a minimum length to avoid matching single-letter paths that are unlikely to be meaningful identifiers

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-extensions/compaction-safeguard.ts
Line: 294

Comment:
regex pattern `\/[\w./-]+` will match single-char paths like `/a` or `/x`, which are likely false positives

the pattern intends to match Unix-style paths but lacks a minimum length constraint; consider requiring at least 2 path segments or a minimum length to avoid matching single-letter paths that are unlikely to be meaningful identifiers

How can I resolve this? If you propose a fix, please make it concise.

@rodrigouroz rodrigouroz force-pushed the codex/pr20038-04 branch 2 times, most recently from 6db0640 to 8971441 Compare February 24, 2026 15:38
@openclaw-barnacle openclaw-barnacle bot added the app: web-ui App: web-ui label Feb 24, 2026
@rodrigouroz rodrigouroz force-pushed the codex/pr20038-04 branch 5 times, most recently from f073ec7 to a35475c Compare February 24, 2026 16:39
@rodrigouroz rodrigouroz marked this pull request as draft February 24, 2026 16:59
@openclaw-barnacle openclaw-barnacle bot removed the app: web-ui App: web-ui label Feb 26, 2026
@openclaw-barnacle
Copy link

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle bot added the stale Marked as stale due to inactivity label Mar 4, 2026
@rodrigouroz rodrigouroz marked this pull request as ready for review March 4, 2026 19:04
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b16cd70531

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +594 to +596
const missingIdentifiers = params.identifiers.filter((id) => !params.summary.includes(id));
if (missingIdentifiers.length > 0) {
reasons.push(`missing_identifiers:${missingIdentifiers.slice(0, 3).join(",")}`);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Respect identifier policy during quality audit

auditSummaryQuality always treats omitted identifiers as a failure and the retry path then tells the model to “include every required section with exact identifiers preserved,” which overrides the existing identifierPolicy behavior from resolveExactIdentifierSectionInstruction (including off and redaction-oriented custom policies). In sessions configured to avoid strict identifier retention, this can force sensitive IDs/tokens back into persisted compaction summaries and changes configured behavior rather than enforcing it.

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 183e644. The quality audit is now policy-aware: identifier-presence checks run only when identifierPolicy is strict, and retry feedback no longer forces strict retention when policy is off/custom.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 4, 2026

Greptile Summary

This PR introduces a bounded quality-audit and retry flow for compaction summaries, along with several hardening fixes across follow-up commits. When qualityGuardEnabled is set (opt-in, defaults to false), the summarization loop audits the generated summary for required section headings, identifier retention, and latest-ask overlap; if the audit fails and retries remain, it feeds structured feedback into the next attempt and falls back to the last successful summary on retry errors.

Key changes:

  • Quality guard (auditSummaryQuality, extractOpaqueIdentifiers, hasAskOverlap): exact heading-line matching (consistent with hasRequiredSummarySections), policy-aware identifier enforcement, Unicode-aware latest-ask overlap with stopword filtering and double-match threshold for longer asks.
  • Scoped audit inputs: identifier seeding and latestUserAsk extraction now use only model-summarized content (messagesToSummarize + turnPrefixMessages), excluding preservedRecentMessages — preventing verbatim preserved turns from masking omissions in the generated summary.
  • Shared sanitization: local sanitizeUntrustedInstructionText removed; replaced by the new exported wrapUntrustedPromptDataBlock in src/agents/sanitize-for-prompt.ts.
  • Retry fallback: on retry failure, the last successful summary is kept rather than propagating the error and cancelling compaction.
  • Hex identifier normalization: pure-hex identifiers are normalized to uppercase and matched case-insensitively to prevent false-positive retention failures across case variations.
  • isQueryStopWordToken extracted from extractKeywords in query-expansion.ts for reuse.

All previously raised review concerns (section substring vs. line match, preserved-turn identifier bypass, hex case-sensitivity, unsanitized reasons) have been addressed and verified in this branch.

Confidence Score: 4/5

  • This PR is safe to merge; quality guard is opt-in (defaults off), all critical issues from prior review rounds are verified as fixed, and retry fallback behavior is sound.
  • The PR successfully hardens compaction summary quality with a bounded retry mechanism. All previously raised correctness concerns (section heading matching, identifier/ask extraction scope, hex normalization, prompt sanitization) have been verified as addressed in the code. The feature is gated behind qualityGuardEnabled: false by default, eliminating production risk without explicit opt-in. One minor observation remains: the qualityGuardMaxRetries = 0 edge case creates a silent no-op (audit runs but no retry occurs), which is worth documenting or warning about for operators who enable the feature. This is a configuration/UX concern, not a correctness or safety issue.
  • src/agents/pi-extensions/compaction-safeguard.ts — one configuration edge case noted in feedback; otherwise the complex retry loop is well-implemented and thoroughly hardened.

Last reviewed commit: 268b292

Comment on lines +917 to +932
const canRegenerate =
messagesToSummarize.length > 0 ||
(preparation.isSplitTurn && turnPrefixMessages.length > 0);
if (!qualityGuardEnabled || !canRegenerate) {
summary = summaryWithPreservedTurns;
break;
}
const quality = auditSummaryQuality({
summary: summaryWithoutPreservedTurns,
identifiers,
latestAsk: latestUserAsk,
identifierPolicy,
});
const splitTurnSection = `**Turn Context (split turn):**\n\n${prefixSummary}`;
summary = historySummary.trim()
? `${historySummary}\n\n---\n\n${splitTurnSection}`
: splitTurnSection;
summary = summaryWithPreservedTurns;
if (quality.ok || attempt >= totalAttempts - 1) {
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When qualityGuardEnabled = true but qualityGuardMaxRetries = 0 (reachable via resolveQualityGuardMaxRetries(-1) → 0), totalAttempts = 1. The quality audit runs but the condition attempt >= totalAttempts - 1 evaluates to true on the first iteration (0 >= 0), causing an immediate break without any retry attempt. The quality feedback is generated but discarded, making the retry feature silently inactive.

This is a valid configuration path (the clamping test explicitly checks resolveQualityGuardMaxRetries(-1) === 0), but callers enabling the guard with 0 retries will see no retry behavior without any warning, making the feature a no-op.

Consider either:

  1. Logging a warning when the guard is enabled with 0 retries, or
  2. Documenting that qualityGuardMaxRetries: 0 disables the retry loop while keeping the audit flag semantically enabled
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-extensions/compaction-safeguard.ts
Line: 917-932

Comment:
When `qualityGuardEnabled = true` but `qualityGuardMaxRetries = 0` (reachable via `resolveQualityGuardMaxRetries(-1) → 0`), `totalAttempts = 1`. The quality audit runs but the condition `attempt >= totalAttempts - 1` evaluates to true on the first iteration (`0 >= 0`), causing an immediate break without any retry attempt. The quality feedback is generated but discarded, making the retry feature silently inactive.

This is a valid configuration path (the clamping test explicitly checks `resolveQualityGuardMaxRetries(-1) === 0`), but callers enabling the guard with 0 retries will see no retry behavior without any warning, making the feature a no-op.

Consider either:
1. Logging a warning when the guard is enabled with 0 retries, or
2. Documenting that `qualityGuardMaxRetries: 0` disables the retry loop while keeping the audit flag semantically enabled

How can I resolve this? If you propose a fix, please make it concise.

@rodrigouroz
Copy link
Contributor Author

@jalehman This is my next follow up, Thank you!

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1c8fe6b7b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +900 to +902
summaryWithoutPreservedTurns = historySummary.trim()
? `${historySummary}\n\n---\n\n${splitTurnSection}`
: splitTurnSection;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Audit history summary before appending split-turn section

When isSplitTurn is true, summaryWithoutPreservedTurns is built by concatenating historySummary with splitTurnSection, and the quality audit runs on that combined text. If the main historySummary omits required headings, but the split-turn prefix summary still contains them, auditSummaryQuality can incorrectly pass and skip retries. This makes the new quality guard ineffective for split-turn compactions, because headings from the prefix can mask structural regressions in the actual history summary.

Useful? React with 👍 / 👎.

@jalehman jalehman self-assigned this Mar 5, 2026
jalehman added a commit to rodrigouroz/openclaw that referenced this pull request Mar 5, 2026
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f37e30dc0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +900 to +902
summaryWithoutPreservedTurns = historySummary.trim()
? `${historySummary}\n\n---\n\n${splitTurnSection}`
: splitTurnSection;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Audit history summary before appending split-turn section

In split-turn compactions, summaryWithoutPreservedTurns is built by concatenating historySummary with splitTurnSection before auditSummaryQuality runs, so required-heading checks can be satisfied by headings that exist only in the prefix summary. This means a malformed historySummary (missing ## Decisions, etc.) can still pass quality audit whenever the split-turn prefix contains those headings, causing retries to be skipped and degraded history summaries to persist in split-turn sessions.

Useful? React with 👍 / 👎.

@jalehman jalehman merged commit 036c329 into openclaw:main Mar 5, 2026
12 checks passed
thinstripe pushed a commit to thinstripe/openclaw that referenced this pull request Mar 6, 2026
Merged via squash.

Prepared head SHA: be473ef
Co-authored-by: rodrigouroz <384037+rodrigouroz@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
handsdiff pushed a commit to handsdiff/activeclaw that referenced this pull request Mar 6, 2026
Merged via squash.

Prepared head SHA: be473ef
Co-authored-by: rodrigouroz <384037+rodrigouroz@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
jenawant pushed a commit to jenawant/openclaw that referenced this pull request Mar 10, 2026
Merged via squash.

Prepared head SHA: be473ef
Co-authored-by: rodrigouroz <384037+rodrigouroz@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: XL stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants