-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Defer-load low-value tools to shrink prefix-cache footprint #2076
Copy link
Copy link
Closed
Labels
autonomous-readyGreenlit for the autonomous nightly loop to pick up, implement, PR, and mergeGreenlit for the autonomous nightly loop to pick up, implement, PR, and mergedocumentationImprovements or additions to documentationImprovements or additions to documentationenhancementNew feature or requestNew feature or requestrustPull requests that update rust codePull requests that update rust code
Metadata
Metadata
Assignees
Labels
autonomous-readyGreenlit for the autonomous nightly loop to pick up, implement, PR, and mergeGreenlit for the autonomous nightly loop to pick up, implement, PR, and mergedocumentationImprovements or additions to documentationImprovements or additions to documentationenhancementNew feature or requestNew feature or requestrustPull requests that update rust codePull requests that update rust code
Projects
StatusShow more project fields
Done
Problem.
crates/tui/src/tools/registry.rsregisters 50+ tools alphabetically (sorted for prefix-cache stability at:196-200). The tool catalog alone is ~8–12K tokens on every request that passes tools.defer_loadingalready exists for MCP tools (:936) but not for native ones — so the same model seesautomation_list,revert_turn, andimage_ocrin its context on every turn even though they're called once a session at most.Shape. Pick the core 14:
Everything else (
diagnostics,pandoc_convert,git_log,git_show,git_blame,recall_archive,revert_turn,automation_*,task_*,checklist_*,finance,image_ocr,notify,validate_data, etc.) getsdefer_loading: trueand is materialized on the first ToolSearch hit. Config knob (tools.always_load = ["foo", "bar"]) lets users opt back in.Acceptance.
scripts/; target ≥ 40% reduction.Existing issues this resolves on PR merge
Partially addresses:
When opening a PR for this issue, the PR body must include the
Closes #Nlines for every issue above that this PR fully resolves, so they auto-close on merge.Slate item 3/18. Tracking doc:
.private/issue-slate.md(maintainer-only).