Skip to content

Add optimizer composite tool E2E test with fake embedding server#4059

Merged
jerm-dro merged 1 commit intomainfrom
jerm/2026-03-09-testing-optimizer-composite-tools
Mar 10, 2026
Merged

Add optimizer composite tool E2E test with fake embedding server#4059
jerm-dro merged 1 commit intomainfrom
jerm/2026-03-09-testing-optimizer-composite-tools

Conversation

@jerm-dro
Copy link
Copy Markdown
Contributor

@jerm-dro jerm-dro commented Mar 9, 2026

Summary

I was paranoid that the optimizer and composite tools is something that would break during the migration to session v2, since composite tools are handled differently than MCPServers. This PR adds an integration test that ensures the two components are wired up correctly.

  • Adds a new Ginkgo test suite ("Optimizer Composite Tools") that exercises composite tool discovery and invocation through the optimizer's find_tool and call_tool meta-tools using the fake embedding server.
  • Adds DeployFakeEmbeddingServer / CleanupFakeEmbeddingServer helpers to helpers.go — a Python HTTP server mimicking the TEI /info and /embed endpoints, reusable by any test needing optimizer mode without TEI.

Type of change

  • New feature

Test plan

  • Manual testing (describe below)

Ran the new test suite against a local Kind cluster with task operator-deploy-local:

KUBECONFIG="$(pwd)/kconfig.yaml" ginkgo -v --fail-fast \
  --focus "Optimizer Composite" ./test/e2e/thv-operator/...

All 5 specs passed:

  • should only expose find_tool and call_tool
  • should discover backend tool via find_tool
  • should discover composite tool via find_tool
  • should invoke backend tool via call_tool
  • should invoke composite tool via call_tool

Changes

File Change
test/e2e/thv-operator/virtualmcp/helpers.go Add DeployFakeEmbeddingServer, CleanupFakeEmbeddingServer helpers and fakeEmbeddingServerScript constant
test/e2e/thv-operator/virtualmcp/virtualmcp_optimizer_composite_test.go New Ginkgo test suite for optimizer composite tool discovery and invocation

Does this introduce a user-facing change?

No

The existing optimizer E2E test uses the heavyweight TEI container image
for its embedding server, which is slow to pull and start. This adds a
parallel test that uses a lightweight fake embedding server (Python HTTP
server mimicking the TEI /info and /embed endpoints) while exercising
composite tool discovery and invocation through the optimizer's find_tool
and call_tool meta-tools.

The new DeployFakeEmbeddingServer helper can be reused by other tests
that need optimizer mode without the TEI dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added the size/M Medium PR: 300-599 lines changed label Mar 9, 2026
@jerm-dro jerm-dro marked this pull request as ready for review March 9, 2026 21:39
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.70%. Comparing base (c25562f) to head (c0761bb).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4059      +/-   ##
==========================================
+ Coverage   68.61%   68.70%   +0.08%     
==========================================
  Files         446      446              
  Lines       45435    45435              
==========================================
+ Hits        31176    31215      +39     
+ Misses      11850    11811      -39     
  Partials     2409     2409              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jerm-dro jerm-dro requested a review from aponcedeleonch March 9, 2026 23:17
Copy link
Copy Markdown
Member

@aponcedeleonch aponcedeleonch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@jerm-dro jerm-dro merged commit a2824e0 into main Mar 10, 2026
42 of 45 checks passed
@jerm-dro jerm-dro deleted the jerm/2026-03-09-testing-optimizer-composite-tools branch March 10, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Medium PR: 300-599 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants