Skip to content

Defer-load low-value tools to shrink prefix-cache footprint #2076

@Hmbown

Description

@Hmbown

Problem. crates/tui/src/tools/registry.rs registers 50+ tools alphabetically (sorted for prefix-cache stability at :196-200). The tool catalog alone is ~8–12K tokens on every request that passes tools. defer_loading already exists for MCP tools (:936) but not for native ones — so the same model sees automation_list, revert_turn, and image_ocr in its context on every turn even though they're called once a session at most.

Shape. Pick the core 14:

read_file, write_file, edit_file, exec_shell, grep_files, file_search,
git_status, git_diff, run_tests, apply_patch, web_search, fetch_url,
list_dir, agent_open

Everything else (diagnostics, pandoc_convert, git_log, git_show, git_blame, recall_archive, revert_turn, automation_*, task_*, checklist_*, finance, image_ocr, notify, validate_data, etc.) gets defer_loading: true and is materialized on the first ToolSearch hit. Config knob (tools.always_load = ["foo", "bar"]) lets users opt back in.

Acceptance.

  • Tool catalog token count measured before/after with a one-shot script under scripts/; target ≥ 40% reduction.
  • Default behavior unchanged for any tool the model already calls in a typical session.
  • Per-tool opt-in via config.

Existing issues this resolves on PR merge

Partially addresses:

When opening a PR for this issue, the PR body must include the Closes #N lines for every issue above that this PR fully resolves, so they auto-close on merge.

Slate item 3/18. Tracking doc: .private/issue-slate.md (maintainer-only).

Metadata

Metadata

Assignees

No one assigned

    Labels

    autonomous-readyGreenlit for the autonomous nightly loop to pick up, implement, PR, and mergedocumentationImprovements or additions to documentationenhancementNew feature or requestrustPull requests that update rust code

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions