fix(openclaw-plugin): enforce token budget and reduce context bloat (#730)#6
fix(openclaw-plugin): enforce token budget and reduce context bloat (#730)#6
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds budget-aware memory injection: new config options (recallMaxContentChars, recallPreferAbstract, recallTokenBudget), token-budgeted sequential memory-line construction with truncation and abstract-preference, leaf detection tightened to level === 2, plus Vitest tests and test tooling. Changes
sequenceDiagram
participant Plugin as Memory Plugin
participant Builder as buildMemoryLinesWithBudget
participant Config as Config
participant Reader as readFn
participant TokenEst as estimateTokenCount
Plugin->>Builder: call(memories[], readFn, options)
Builder->>Config: read recallPreferAbstract, recallTokenBudget, recallMaxContentChars
loop per memory (until budget)
Builder->>Config: check preferAbstract & item.level
alt preferAbstract and item.abstract non-empty
Builder->>Builder: use item.abstract
else item.level == 2
Builder->>Reader: read(item.uri)
Reader-->>Builder: content or error
alt success & non-blank
Builder->>Builder: use content
else
Builder->>Builder: fallback to item.abstract or uri
end
end
Builder->>Builder: truncate to recallMaxContentChars + "..."
Builder->>TokenEst: estimateTokenCount(text)
TokenEst-->>Builder: tokens
alt current + tokens <= budget or first item
Builder->>Builder: add line, update totals
else
Builder->>Builder: stop iteration
end
end
Builder-->>Plugin: { lines, estimatedTokens }
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review Summary by QodoEnforce token budget and reduce context bloat in memory injection
WalkthroughsDescription• Enforce token budget (default 2000) with decrement loop to stop injection when exhausted • Raise recallScoreThreshold default from 0.01 to 0.15 to filter ~70% irrelevant memories • Prefer memory abstract over full content fetch to reduce token usage by 100-300 chars • Add three new config options: recallMaxContentChars (500), recallPreferAbstract (true), recallTokenBudget (2000) • Narrow isLeafLikeMemory boost to level-2 only, removing false-positive .md URI boost • Add comprehensive vitest test suite (10 tests) covering all slices and backward compatibility Diagramflowchart LR
A["Memory Ranking"] -->|"Filter by score >= 0.15"| B["Post-Process"]
B -->|"Prefer abstract"| C["Build Memory Lines"]
C -->|"Truncate to 500 chars"| D["Content Prep"]
D -->|"Estimate tokens"| E["Budget Loop"]
E -->|"Stop at 2000 tokens"| F["Inject Context"]
File Changes1. examples/openclaw-plugin/__tests__/context-bloat-730.test.ts
|
Code Review by Qodo
1.
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical issue of excessive memory context injection in the OpenClaw plugin, which led to high token usage and increased operational costs. The changes introduce several strategies to enforce a token budget, reduce the size of injected memories, and improve the relevance filtering of recalled information. This significantly optimizes the plugin's performance and cost-efficiency without introducing breaking changes. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Failed to generate code suggestions for PR |
There was a problem hiding this comment.
🧹 Nitpick comments (3)
examples/openclaw-plugin/__tests__/context-bloat-730.test.ts (1)
148-153: Consider tightening the token budget assertion.The test allows
estimatedTokensto be up to 120 with a budget of 100, which is a 20% overshoot. This is because the implementation allows the first line to exceed the budget. While functionally correct, consider documenting this behavior in the test comment or adjusting the assertion to be more precise (e.g.,expect(estimatedTokens).toBeLessThanOrEqual(106)for one line at ~53 tokens).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/openclaw-plugin/__tests__/context-bloat-730.test.ts` around lines 148 - 153, Tighten the token-budget assertion in the test by reducing the allowed overshoot or document the acceptable overshoot behavior: change the assertion on estimatedTokens in the test (the variable estimatedTokens used alongside lines) to a stricter bound (e.g., expect(estimatedTokens).toBeLessThanOrEqual(106)) to reflect one-line ~53-token rounding, or add a clarifying comment above the assertions explaining why a small overshoot above the 100-token budget is allowed due to the first-line rounding behavior.examples/openclaw-plugin/index.ts (2)
701-733: Consider extracting shared content-resolution logic to reduce duplication.
buildMemoryLinesandbuildMemoryLinesWithBudgetshare nearly identical logic for resolving content (lines 708-724 vs 753-769) and truncation (lines 726-728 vs 771-773). Consider extracting a helper function likeresolveMemoryContent(item, readFn, options)to reduce duplication.♻️ Proposed refactor to extract shared logic
+async function resolveMemoryContent( + item: FindResultItem, + readFn: (uri: string) => Promise<string>, + options: BuildMemoryLinesOptions, +): Promise<string> { + let content: string; + + if (options.recallPreferAbstract && item.abstract?.trim()) { + content = item.abstract.trim(); + } else if (item.level === 2) { + try { + const fullContent = await readFn(item.uri); + content = + fullContent && typeof fullContent === "string" && fullContent.trim() + ? fullContent.trim() + : (item.abstract ?? item.uri); + } catch { + content = item.abstract ?? item.uri; + } + } else { + content = item.abstract ?? item.uri; + } + + if (content.length > options.recallMaxContentChars) { + content = content.slice(0, options.recallMaxContentChars) + "..."; + } + + return content; +} export async function buildMemoryLines( memories: FindResultItem[], readFn: (uri: string) => Promise<string>, options: BuildMemoryLinesOptions, ): Promise<string[]> { const lines: string[] = []; for (const item of memories) { - let content: string; - - if (options.recallPreferAbstract && item.abstract?.trim()) { - content = item.abstract.trim(); - } else if (item.level === 2) { - try { - const fullContent = await readFn(item.uri); - content = - fullContent && typeof fullContent === "string" && fullContent.trim() - ? fullContent.trim() - : (item.abstract ?? item.uri); - } catch { - content = item.abstract ?? item.uri; - } - } else { - content = item.abstract ?? item.uri; - } - - if (content.length > options.recallMaxContentChars) { - content = content.slice(0, options.recallMaxContentChars) + "..."; - } - + const content = await resolveMemoryContent(item, readFn, options); lines.push(`- [${item.category ?? "memory"}] ${content}`); } return lines; }Apply similar changes to
buildMemoryLinesWithBudget.Also applies to: 739-788
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/openclaw-plugin/index.ts` around lines 701 - 733, buildMemoryLines and buildMemoryLinesWithBudget duplicate the logic that resolves content and truncates it; extract that into a shared helper (e.g., resolveMemoryContent(item: FindResultItem, readFn: (uri:string)=>Promise<string>, options: BuildMemoryLinesOptions): string) and call it from both buildMemoryLines and buildMemoryLinesWithBudget to centralize: implement the preference for abstract (options.recallPreferAbstract), the level===2 read-with-try/catch and fallback to item.abstract or item.uri, and the truncation to options.recallMaxContentChars with ellipsis so both functions simply format lines from the returned content.
776-784: Clarify the budget overshoot behavior.The condition
lineTokens > budgetRemaining && lines.length > 0allows the first memory to exceed the budget, ensuring at least one memory is always injected. This is reasonable behavior but worth documenting in the function's JSDoc to set expectations.📝 Proposed documentation addition
+/** + * Build memory lines with token budget enforcement. + * At least one memory is always included if the input is non-empty, + * even if that single memory exceeds the budget. + */ export async function buildMemoryLinesWithBudget(🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/openclaw-plugin/index.ts` around lines 776 - 784, Update the function's JSDoc that contains this token-budget loop to explicitly document the overshoot behavior: note that the check (lineTokens > budgetRemaining && lines.length > 0) intentionally allows the first memory (when lines.length === 0) to exceed the budget so at least one memory is injected; describe the roles of lineTokens, budgetRemaining, and lines and state that subsequent lines will not be added once they would push over the remaining budget. Reference the variables lineTokens, budgetRemaining, and lines in the JSDoc so callers understand the expected behavior and trade-off.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@examples/openclaw-plugin/__tests__/context-bloat-730.test.ts`:
- Around line 148-153: Tighten the token-budget assertion in the test by
reducing the allowed overshoot or document the acceptable overshoot behavior:
change the assertion on estimatedTokens in the test (the variable
estimatedTokens used alongside lines) to a stricter bound (e.g.,
expect(estimatedTokens).toBeLessThanOrEqual(106)) to reflect one-line ~53-token
rounding, or add a clarifying comment above the assertions explaining why a
small overshoot above the 100-token budget is allowed due to the first-line
rounding behavior.
In `@examples/openclaw-plugin/index.ts`:
- Around line 701-733: buildMemoryLines and buildMemoryLinesWithBudget duplicate
the logic that resolves content and truncates it; extract that into a shared
helper (e.g., resolveMemoryContent(item: FindResultItem, readFn:
(uri:string)=>Promise<string>, options: BuildMemoryLinesOptions): string) and
call it from both buildMemoryLines and buildMemoryLinesWithBudget to centralize:
implement the preference for abstract (options.recallPreferAbstract), the
level===2 read-with-try/catch and fallback to item.abstract or item.uri, and the
truncation to options.recallMaxContentChars with ellipsis so both functions
simply format lines from the returned content.
- Around line 776-784: Update the function's JSDoc that contains this
token-budget loop to explicitly document the overshoot behavior: note that the
check (lineTokens > budgetRemaining && lines.length > 0) intentionally allows
the first memory (when lines.length === 0) to exceed the budget so at least one
memory is injected; describe the roles of lineTokens, budgetRemaining, and lines
and state that subsequent lines will not be added once they would push over the
remaining budget. Reference the variables lineTokens, budgetRemaining, and lines
in the JSDoc so callers understand the expected behavior and trade-off.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 60ca8323-0844-4a1a-8914-1d1d8199a5c3
⛔ Files ignored due to path filters (1)
examples/openclaw-plugin/package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (7)
examples/openclaw-plugin/__tests__/context-bloat-730.test.tsexamples/openclaw-plugin/config.tsexamples/openclaw-plugin/index.tsexamples/openclaw-plugin/memory-ranking.tsexamples/openclaw-plugin/openclaw.plugin.jsonexamples/openclaw-plugin/package.jsonexamples/openclaw-plugin/vitest.config.ts
There was a problem hiding this comment.
Code Review
This pull request is a solid improvement that effectively addresses the token bloat issue by introducing a token budget and other optimizations for memory recall. The changes are well-organized and include a comprehensive suite of new regression tests, which is excellent for ensuring stability. I have one suggestion to improve maintainability by reducing some code duplication in the newly added functions.
| export async function buildMemoryLines( | ||
| memories: FindResultItem[], | ||
| readFn: (uri: string) => Promise<string>, | ||
| options: BuildMemoryLinesOptions, | ||
| ): Promise<string[]> { | ||
| const lines: string[] = []; | ||
| for (const item of memories) { | ||
| let content: string; | ||
|
|
||
| if (options.recallPreferAbstract && item.abstract?.trim()) { | ||
| content = item.abstract.trim(); | ||
| } else if (item.level === 2) { | ||
| try { | ||
| const fullContent = await readFn(item.uri); | ||
| content = | ||
| fullContent && typeof fullContent === "string" && fullContent.trim() | ||
| ? fullContent.trim() | ||
| : (item.abstract ?? item.uri); | ||
| } catch { | ||
| content = item.abstract ?? item.uri; | ||
| } | ||
| } else { | ||
| content = item.abstract ?? item.uri; | ||
| } | ||
|
|
||
| if (content.length > options.recallMaxContentChars) { | ||
| content = content.slice(0, options.recallMaxContentChars) + "..."; | ||
| } | ||
|
|
||
| lines.push(`- [${item.category ?? "memory"}] ${content}`); | ||
| } | ||
| return lines; | ||
| } | ||
|
|
||
| export type BuildMemoryLinesWithBudgetOptions = BuildMemoryLinesOptions & { | ||
| recallTokenBudget: number; | ||
| }; | ||
|
|
||
| export async function buildMemoryLinesWithBudget( | ||
| memories: FindResultItem[], | ||
| readFn: (uri: string) => Promise<string>, | ||
| options: BuildMemoryLinesWithBudgetOptions, | ||
| ): Promise<{ lines: string[]; estimatedTokens: number }> { | ||
| let budgetRemaining = options.recallTokenBudget; | ||
| const lines: string[] = []; | ||
| let totalTokens = 0; | ||
|
|
||
| for (const item of memories) { | ||
| if (budgetRemaining <= 0) { | ||
| break; | ||
| } | ||
|
|
||
| let content: string; | ||
|
|
||
| if (options.recallPreferAbstract && item.abstract?.trim()) { | ||
| content = item.abstract.trim(); | ||
| } else if (item.level === 2) { | ||
| try { | ||
| const fullContent = await readFn(item.uri); | ||
| content = | ||
| fullContent && typeof fullContent === "string" && fullContent.trim() | ||
| ? fullContent.trim() | ||
| : (item.abstract ?? item.uri); | ||
| } catch { | ||
| content = item.abstract ?? item.uri; | ||
| } | ||
| } else { | ||
| content = item.abstract ?? item.uri; | ||
| } | ||
|
|
||
| if (content.length > options.recallMaxContentChars) { | ||
| content = content.slice(0, options.recallMaxContentChars) + "..."; | ||
| } | ||
|
|
||
| const line = `- [${item.category ?? "memory"}] ${content}`; | ||
| const lineTokens = estimateTokenCount(line); | ||
|
|
||
| if (lineTokens > budgetRemaining && lines.length > 0) { | ||
| break; | ||
| } | ||
|
|
||
| lines.push(line); | ||
| totalTokens += lineTokens; | ||
| budgetRemaining -= lineTokens; | ||
| } | ||
|
|
||
| return { lines, estimatedTokens: totalTokens }; | ||
| } |
There was a problem hiding this comment.
There's significant code duplication between buildMemoryLines and buildMemoryLinesWithBudget. The logic for retrieving and truncating memory content is identical in both functions. To improve maintainability and adhere to the DRY (Don't Repeat Yourself) principle, this common logic can be extracted into a private helper function.
async function getMemoryContent(
item: FindResultItem,
readFn: (uri: string) => Promise<string>,
options: BuildMemoryLinesOptions,
): Promise<string> {
let content: string;
if (options.recallPreferAbstract && item.abstract?.trim()) {
content = item.abstract.trim();
} else if (item.level === 2) {
try {
const fullContent = await readFn(item.uri);
content =
fullContent && typeof fullContent === "string" && fullContent.trim()
? fullContent.trim()
: (item.abstract ?? item.uri);
} catch {
content = item.abstract ?? item.uri;
}
} else {
content = item.abstract ?? item.uri;
}
if (content.length > options.recallMaxContentChars) {
content = content.slice(0, options.recallMaxContentChars) + "...";
}
return content;
}
export async function buildMemoryLines(
memories: FindResultItem[],
readFn: (uri: string) => Promise<string>,
options: BuildMemoryLinesOptions,
): Promise<string[]> {
const lines: string[] = [];
for (const item of memories) {
const content = await getMemoryContent(item, readFn, options);
lines.push(`- [${item.category ?? "memory"}] ${content}`);
}
return lines;
}
export type BuildMemoryLinesWithBudgetOptions = BuildMemoryLinesOptions & {
recallTokenBudget: number;
};
export async function buildMemoryLinesWithBudget(
memories: FindResultItem[],
readFn: (uri: string) => Promise<string>,
options: BuildMemoryLinesWithBudgetOptions,
): Promise<{ lines: string[]; estimatedTokens: number }> {
let budgetRemaining = options.recallTokenBudget;
const lines: string[] = [];
let totalTokens = 0;
for (const item of memories) {
if (budgetRemaining <= 0) {
break;
}
const content = await getMemoryContent(item, readFn, options);
const line = `- [${item.category ?? "memory"}] ${content}`;
const lineTokens = estimateTokenCount(line);
if (lineTokens > budgetRemaining && lines.length > 0) {
break;
}
lines.push(line);
totalTokens += lineTokens;
budgetRemaining -= lineTokens;
}
return { lines, estimatedTokens: totalTokens };
}…get behavior (volcengine#730) Extract resolveMemoryContent() helper to eliminate duplicate content-resolution logic between buildMemoryLines and buildMemoryLinesWithBudget. Add JSDoc and inline comment documenting intentional first-line budget overshoot (spec §6.2). Tighten test assertion from <=120 to <=106 tokens.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@examples/openclaw-plugin/index.ts`:
- Around line 714-721: The code uses nullish coalescing (item.abstract ??
item.uri) which does not fallback when abstract is an empty string, so change
these assignments to use a truthy fallback (e.g., item.abstract?.trim() ||
item.uri or item.abstract || item.uri) wherever content is set (inside the try
block assignment, the catch block assignment, and the else branch) so empty
strings fall back to item.uri; update the occurrences in the functions/blocks
that set content (the try block that checks fullContent and the subsequent
catch/else assignments) accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4b66805c-4e11-435e-89f5-e0136cbc69a5
📒 Files selected for processing (2)
examples/openclaw-plugin/__tests__/context-bloat-730.test.tsexamples/openclaw-plugin/index.ts
Resolve conflict in index.ts: keep buildMemoryLinesWithBudget approach inside main's timeout wrapper (AUTO_RECALL_TIMEOUT_MS).
…olcengine#730) Change nullish coalescing (??) to truthy fallback (||) in resolveMemoryContent() so empty-string abstracts fall back to item.uri instead of producing empty content lines.
|
Why 0.15 over 0.10? Short answer: 0.15 is the better choice, but the margin is small. Neither has hard empirical data in the codebase. Evidence from research ┌───────────────┬─────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────┐ Codebase-specific factors
Recommendation 0.15 is correct for now. It's the conservative edge of the industry-standard noise floor range (0.15-0.25). Going to 0.10 would admit If you want to be data-driven later, you could add telemetry to log score distributions and empirically find the optimal cutoff for |
Problem
5 compounding causes inject 16K+ tokens of memory context per LLM call with no budget.
See:
docs/744/category-1-openclaw-plugin/730-token-not-reduced.mdFix (7 commits, each independently revertible)
recallScoreThresholddefault 0.01→0.15 (filters ~70% irrelevant memories)isLeafLikeMemoryboost to level-2 only (reduce false-positive relevance)buildMemoryLines()and preferitem.abstractoverclient.read(uri)(100-300 chars vs full file)recallMaxContentChars(default: 500) andrecallPreferAbstract(default: true) config optionsrecallTokenBudget(default: 2000) with decrement loop — hard stop when budget exhaustedNew Config Options
recallMaxContentCharsrecallPreferAbstractrecallTokenBudgetAll options added to: TypeScript type,
assertAllowedKeys(),openclaw.plugin.jsonschema + uiHints.Testing
10 regression tests (vitest), covering:
recallScoreThreshold: 0.01preserved)isLeafLikeMemoryranking (level-2 only, no.mdURI boost)client.read()skipped when abstract available)recallMaxContentChars)estimateTokenCount(chars/4 heuristic)Impact
Test Plan
Closes volcengine#730
Summary by CodeRabbit
New Features
Improvements
Tests
Chores