[CLI] Refactor query command #2995
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the CLI query command by moving prompt and request logic into a dedicated query subdirectory and updating the documentation. It simplifies token reporting by using usage data directly from the engine instead of local estimation. Feedback was provided regarding incomplete docstrings for several public functions in lmcache/cli/commands/query/prompt.py, which violate the project's style guide requiring descriptions for arguments, return values, and exceptions.
Signed-off-by: deng451e <838677410@qq.com>
…ubpackage - Extract output_tokens as int (like prompt_tokens) to avoid float rendering (9.00 → 9) that mismatched docs examples - Add complete Args/Returns/Raises sections to expand_prompt, resolve_documents, and unknown_documents per project style guide Signed-off-by: deng451e <838677410@qq.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d24787e. Configure here.
| MetricValue = tuple[str, Any] | ||
| MetricMap = dict[str, MetricValue] | ||
| from lmcache.cli.commands.query.prompt import PromptBuilder | ||
| from lmcache.cli.commands.query.request import Request |
There was a problem hiding this comment.
Design doc not updated after query restructuring
Low Severity
The design doc docs/design/cli/commands/query-command.md was not updated to reflect this restructuring. It still references the old file paths (lmcache/cli/commands/query.py, lmcache/cli/prompt.py, lmcache/cli/request.py), shows the old output format with per-document token breakdown ("Prompt documents lmcache", "Prompt query"), and describes the now-removed tokenizer-based token counting behavior. The project rules require design documents to be updated for architectural changes.
Additional Locations (1)
Triggered by project rule: LMCache Code Review Style Guide
Reviewed by Cursor Bugbot for commit d24787e. Configure here.
Description:
Restructures lmcache query into a dedicated subpackage and removes local tokenizer usage for prompt token counting.
Changes:
Note
Medium Risk
Moderate risk because it changes
lmcache querymetric calculation (token counts now sourced from engine-reported usage and TTFT behavior when no tokens stream) and moves modules, which could affect imports/packaging.Overview
Refactors
lmcache queryinto a dedicatedlmcache/cli/commands/query/subpackage, relocating prompt expansion and HTTP request logic and updating docs to match the new layout.Changes
query enginemetrics output to reportprompt_tokens/output_tokensdirectly from the engine’sinclude_usagestream data (removing local tokenizer-based counting and per-document breakdown), and adjusts TTFT calculation to fall back to total round-trip time when no token is observed.Reviewed by Cursor Bugbot for commit d24787e. Bugbot is set up for automated code reviews on this repo. Configure here.