Skip to content

Improve prompts and tool descriptions#35

Merged
ScriptSmith merged 1 commit into
mainfrom
improve-prompts
May 24, 2026
Merged

Improve prompts and tool descriptions#35
ScriptSmith merged 1 commit into
mainfrom
improve-prompts

Conversation

@ScriptSmith

Copy link
Copy Markdown
Member

No description provided.

@greptile-apps

greptile-apps Bot commented May 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR improves the LLM-facing tool descriptions for file_search, web_search, web_fetch, and the Vega-Lite chart tool, refactors the system prompt to drop the model-name identity in favour of instanceLabel, and introduces a generic agentic guidance block (getAgenticGuidance) that is now prepended alongside per-tool systemGuidance whenever any tool is enabled.

  • Tool descriptions: All updated descriptions follow the new "what / when / when-not / caveats" shape and replace all-caps directives with explanatory prose; the Rust backend and TypeScript frontend descriptions were kept in sync.
  • System prompt / agentic guidance: getDefaultSystemPrompt drops the modelId parameter (intentional — bare model names can mislead cross-provider routing), and getAppendedSystemGuidance is the new single entry point that combines agentic and per-tool guidance, ensuring the settings modal preview matches what is actually sent.
  • Reranker prompt: Compacted from six explicit score bands to three approximate ranges; deliberate prompt-engineering trade-off.

Confidence Score: 4/5

Safe to merge; all changes are prompt-text and light refactoring with no data-path mutations.

The changes are confined to string descriptions and a thin wrapper function. The getDefaultSystemPrompt signature change has no remaining callers passing modelId, and both call sites were updated correctly. The one structural concern is that file_search_tool.rs still duplicates the description literal rather than delegating to Self::function_description(), leaving a future drift risk — but both strings were kept in sync in this PR.

src/services/file_search_tool.rs — the description literal duplication between function_tool_definition() and function_description() is worth tidying before the next description update.

Important Files Changed

Filename Overview
src/services/file_search_tool.rs Improved LLM-facing description for file_search; both the JSON definition and function_description() strings were updated in sync, but they remain independent literals (description still duplicated rather than referenced from function_description).
src/services/reranker.rs Reranker system prompt compacted from 6 explicit bands to 3 approximate ranges and trimmed by ~9 lines; deliberate tradeoff that slightly shifts score distribution expectations at the high and low ends.
src/services/web_search_tool.rs Updated function_description() to a richer, more actionable LLM-facing description; no duplication since get_tool() already calls Self::function_description().
ui/src/utils/defaultSystemPrompt.ts Removed modelId parameter from getDefaultSystemPrompt (deliberate: model name provides no behavioral steering), updated math delimiter guidance, and added getAgenticGuidance helper that appends tool-use instructions when tools are present.
ui/src/pages/chat/utils/toolExecutors.ts Added getAppendedSystemGuidance (wraps getAgenticGuidance + getEnabledToolsSystemGuidance); getEnabledToolsSystemGuidance is now only called internally but remains exported.
ui/src/pages/chat/useChat.ts Switched to getAppendedSystemGuidance, dropped modelId from getDefaultSystemPrompt call, and updated vega_lite/web_search/web_fetch tool descriptions to be more informative and less directive in tone.
ui/src/components/ConversationSettingsModal/ConversationSettingsModal.tsx Import and call site updated from getEnabledToolsSystemGuidance to getAppendedSystemGuidance so the preview panel reflects the full effective prompt (including agentic guidance).
agent_instructions/adding_frontend_tool.md Added authoring guidance for tool descriptions: two-description sync warning, recommended shape, positive constraint framing, and em-dash avoidance.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[useChat: build request] --> B{hasSystemMessage?}
    B -- No --> C[getDefaultSystemPrompt]
    C --> D[getAppendedSystemGuidance]
    D --> E[getAgenticGuidance]
    D --> F[getEnabledToolsSystemGuidance]
    E --> G[Combine: basePrompt + guidance]
    F --> G
    G --> H[system message added to input]
    B -- Yes --> H
    I[ConversationSettingsModal] --> J[getDefaultSystemPrompt]
    I --> K[getAppendedSystemGuidance]
    J --> L[Read-only effective prompt preview]
    K --> L
Loading

Comments Outside Diff (1)

  1. ui/src/pages/chat/utils/toolExecutors.ts, line 2738 (link)

    P2 getEnabledToolsSystemGuidance is now only called by getAppendedSystemGuidance within the same file — no external callers remain after this PR. Keeping it exported widens the public API surface unnecessarily and may invite callers that bypass the agentic-guidance block. Removing the export clarifies that getAppendedSystemGuidance is the intended entry point.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: ui/src/pages/chat/utils/toolExecutors.ts
    Line: 2738
    
    Comment:
    `getEnabledToolsSystemGuidance` is now only called by `getAppendedSystemGuidance` within the same file — no external callers remain after this PR. Keeping it `export`ed widens the public API surface unnecessarily and may invite callers that bypass the agentic-guidance block. Removing the export clarifies that `getAppendedSystemGuidance` is the intended entry point.
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
src/services/file_search_tool.rs:200-205
The description string is duplicated as a literal in both `function_tool_definition()` and `function_description()`. If one is updated without the other they silently diverge — exactly the drift risk the new documentation section warns about. `web_search_tool.rs` already avoids this by writing `"description": Self::function_description()` in its tool definition; the same pattern works here.

```suggestion
        serde_json::json!({
            "type": "function",
            "name": "file_search",
            "description": Self::function_description(),
            "parameters": Self::function_parameters_schema()
        })
```

### Issue 2 of 2
ui/src/pages/chat/utils/toolExecutors.ts:2738
`getEnabledToolsSystemGuidance` is now only called by `getAppendedSystemGuidance` within the same file — no external callers remain after this PR. Keeping it `export`ed widens the public API surface unnecessarily and may invite callers that bypass the agentic-guidance block. Removing the export clarifies that `getAppendedSystemGuidance` is the intended entry point.

```suggestion
function getEnabledToolsSystemGuidance(enabledToolIds: string[]): string {
```

Reviews (1): Last reviewed commit: "Improve prompts and tool descriptions" | Re-trigger Greptile

Comment on lines 200 to 205
serde_json::json!({
"type": "function",
"name": "file_search",
"description": "Search for relevant information in the configured knowledge base. Use this tool when you need to find specific information, answer questions about documents, or look up facts from the available files.",
"description": "Search the knowledge base attached to this conversation (the user's uploaded files and configured vector stores) using semantic search. Returns the most relevant passages with relevance scores, not whole documents, so synthesize your answer from these snippets. Use this for questions about the user's own documents or domain data. Don't use it for general knowledge you already have; for current web information, use a web search tool if one is available.",
"parameters": Self::function_parameters_schema()
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The description string is duplicated as a literal in both function_tool_definition() and function_description(). If one is updated without the other they silently diverge — exactly the drift risk the new documentation section warns about. web_search_tool.rs already avoids this by writing "description": Self::function_description() in its tool definition; the same pattern works here.

Suggested change
serde_json::json!({
"type": "function",
"name": "file_search",
"description": "Search for relevant information in the configured knowledge base. Use this tool when you need to find specific information, answer questions about documents, or look up facts from the available files.",
"description": "Search the knowledge base attached to this conversation (the user's uploaded files and configured vector stores) using semantic search. Returns the most relevant passages with relevance scores, not whole documents, so synthesize your answer from these snippets. Use this for questions about the user's own documents or domain data. Don't use it for general knowledge you already have; for current web information, use a web search tool if one is available.",
"parameters": Self::function_parameters_schema()
})
serde_json::json!({
"type": "function",
"name": "file_search",
"description": Self::function_description(),
"parameters": Self::function_parameters_schema()
})
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/services/file_search_tool.rs
Line: 200-205

Comment:
The description string is duplicated as a literal in both `function_tool_definition()` and `function_description()`. If one is updated without the other they silently diverge — exactly the drift risk the new documentation section warns about. `web_search_tool.rs` already avoids this by writing `"description": Self::function_description()` in its tool definition; the same pattern works here.

```suggestion
        serde_json::json!({
            "type": "function",
            "name": "file_search",
            "description": Self::function_description(),
            "parameters": Self::function_parameters_schema()
        })
```

How can I resolve this? If you propose a fix, please make it concise.

@ScriptSmith ScriptSmith merged commit 137084d into main May 24, 2026
21 checks passed
@ScriptSmith ScriptSmith deleted the improve-prompts branch May 30, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant