Skip to content

fix(mcp_semantic_tool_filter): resolve client-suffixed tool names (LibreChat-style)#26858

Closed
GopalGB wants to merge 1 commit into
BerriAI:litellm_oss_stagingfrom
GopalGB:gb-fix/mcp-suffix-canonical-26507
Closed

fix(mcp_semantic_tool_filter): resolve client-suffixed tool names (LibreChat-style)#26858
GopalGB wants to merge 1 commit into
BerriAI:litellm_oss_stagingfrom
GopalGB:gb-fix/mcp-suffix-canonical-26507

Conversation

@GopalGB

@GopalGB GopalGB commented Apr 30, 2026

Copy link
Copy Markdown

Closes #26507.

Repro

LibreChat (and similar MCP clients that connect to multiple servers through one proxy) appends a per-tool unique-ID suffix to canonical tool names to disambiguate same-named tools across servers, e.g. fc_web_search-firecrawl_scrape_a1b2c3d4.

The semantic-tool-filter matcher currently only handles the symmetric prefix wrapping (opencode-style, litellm_<canonical>). Suffix-wrapped names fall through _name_matches_canonical, the filter ships tools: [] with tool_choice: "auto" to the upstream provider, and every strict OpenAI-compatible model returns 400 'tool_choice' is only allowed when 'tools' are specified.

m = SemanticMCPToolFilter._name_matches_canonical
m('litellm_fc_web_search-firecrawl_scrape', 'fc_web_search-firecrawl_scrape')  # True (existing)
m('fc_web_search-firecrawl_scrape_a1b2c3d4', 'fc_web_search-firecrawl_scrape')  # False (BUG)

Fix

Extend _name_matches_canonical to also accept client_name.startswith(canonical + sep) matches, while keeping the same anchoring guards as the suffix path:

  1. Match must be adjacent to a separator character (_ or -); bare-substring matches do not count, so earthquake_alert does not match canonical ear.
  2. canonical must contain MCP_TOOL_PREFIX_SEPARATOR, otherwise unrelated local user functions whose names share the same characters (e.g. firecrawl_scrape_v2_helper vs canonical firecrawl_scrape) would be spuriously selected. Symmetric with the suffix-direction guard added in fix(mcp_semantic_tool_filter): match tools with client-side namespace prefix (#26078) #26117.

The semantic delta is ~6 lines of logic and a refreshed docstring; the rest of the diff is reflowing introduced by the project's auto-formatter.

Test

Adds five regression tests:

  • test_client_suffix_with_underscore_separator — exact shape from the bug report (<canonical>_<uid>)
  • test_client_suffix_with_dash_separator — dash-separator variant
  • test_prefix_without_separator_does_not_match — bare-substring guard in the prefix direction (earthquake_alert != ear)
  • test_suffix_does_not_match_unprefixed_canonical — unprefixed-canonical guard in the prefix direction (mirror of fix(mcp_semantic_tool_filter): match tools with client-side namespace prefix (#26078) #26117's local-function guard)
  • test_prefix_and_suffix_wrappers_can_coexist — mixed-client scenario

All 14 TestGetToolsByNames tests pass after the fix:

$ python -m pytest tests/test_litellm/proxy/_experimental/mcp_server/test_semantic_tool_filter.py::TestGetToolsByNames
14 passed in 12.49s

The new tests fail on the unpatched matcher (verified by reverting only the source change with git stash push <source-file> and re-running). All previous tests still pass.

🤖 Generated with Claude Code

@greptile-apps

greptile-apps Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review. (140 files found, 100 file limit)

@CLAassistant

CLAassistant commented Apr 30, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codspeed-hq

codspeed-hq Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing GopalGB:gb-fix/mcp-suffix-canonical-26507 (a3b04a3) with main (6ff668c)

Open in CodSpeed

Comment thread ui/litellm-dashboard/src/components/workflow_runs/index.tsx Fixed
@codecov

codecov Bot commented Apr 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@GopalGB

GopalGB commented May 7, 2026

Copy link
Copy Markdown
Author

Force-pushed cleaned-up branch. Two changes:

  1. Branch hygiene — the previous push had 1700+ stray commits from upstream merges that crept in during a bad rebase. The branch now contains only the actual fix (2 files, +168/-104) on top of latest main. This should resolve the Verify PR source branch and lint failures, and bring the file count under Greptile's 100-file review limit.
  2. Author identity — re-authored under GopalGB <67310594+GopalGB@users.noreply.github.com> so the CLA-assistant correctly attributes the commit (previous email was an old verified email that may have caused linking issues).

Will sign the CLA via cla-assistant.io and re-trigger checks.

@GopalGB GopalGB changed the base branch from main to litellm_oss_staging May 7, 2026 14:50
@GopalGB GopalGB force-pushed the gb-fix/mcp-suffix-canonical-26507 branch from a3b04a3 to 4bc0c8c Compare May 7, 2026 14:50
@GopalGB

GopalGB commented May 7, 2026

Copy link
Copy Markdown
Author

Two more fixes pushed:

  1. Re-targeted to litellm_oss_staging — discovered the new external-contributor policy: PRs to main are blocked by the Verify PR source branch workflow which requires litellm_oss_staging for outside forks. Updated the base branch.
  2. black formatting — ran black on the changed file to fix the lint failure (3 long lines reformatted).

Branch state: 1 commit, 2 files (semantic_tool_filter.py + its test), authored under GopalGB <67310594+GopalGB@users.noreply.github.com>, targeting litellm_oss_staging. All blocking checks should now resolve. cc @ishaan-jaffer or @krrishdholakia for review when cycles allow.

…breChat-style)

Closes BerriAI#26507. Adds LibreChat-style suffix wrapping so client-side
tool name suffixes are stripped before semantic matching, then re-applied
on tool emission.

Tested via tests/test_litellm/proxy/_experimental/mcp_server/test_semantic_tool_filter.py

Signed-off-by: GopalGB <67310594+GopalGB@users.noreply.github.com>
@GopalGB GopalGB force-pushed the gb-fix/mcp-suffix-canonical-26507 branch from 4bc0c8c to e124e9a Compare May 7, 2026 14:52
@mateo-berri mateo-berri deleted the branch BerriAI:litellm_oss_staging May 18, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants