Skip to content

Acceptance testing — circuit breaker excludes unhealthy backends from the optimizer #3760

@jerm-dro

Description

@jerm-dro

Parent Epic

stacklok/stacklok-epics#201

Depends on: #3733

Context

The epic acceptance criteria requires that the optimizer integrates with the circuit breaker functionality so that unhealthy backends don't cause complete failures. vMCP already has a health monitoring system (pkg/vmcp/health/) that tracks backend health. When a backend is unhealthy at session creation time, its tools should be excluded from that session's optimizer.

This issue does not introduce within-session health transitions or capability refresh logic. Health status is evaluated once at session creation — if a backend is unhealthy when the session starts, its tools are excluded for the lifetime of that session.

Requirements

  1. When a backend is unhealthy at session creation time, its tools must be excluded from that session's FindTool results.
  2. When a backend is healthy at session creation time, its tools are included normally.
  3. Create acceptance tests that validate this behavior.

High-Level Implementation

  • Investigate how the health monitor (pkg/vmcp/health/) currently communicates backend health status
  • Ensure the optimizer's per-session tool scoping (from Back the dummyOptimizer with a shared SQLite instance and basic similarity search #3731) respects backend health at session creation time — unhealthy backends' tools are simply not passed to the optimizer factory
  • Write acceptance tests that:
    • Start with multiple backends, some healthy and some unhealthy
    • Create a session and verify FindTool only returns tools from healthy backends
    • Create another session after a previously unhealthy backend has recovered and verify its tools are now included

Acceptance Tests

  • Unhealthy backend excluded: When a backend is unhealthy at session creation, its tools are not returned by FindTool for that session
  • Healthy backend tools unaffected: Tools from healthy backends are returned normally
  • New session sees recovered backend: A session created after a backend recovers includes that backend's tools
  • No crashes: The optimizer does not crash or hang when backends are unhealthy at session creation time

Metadata

Metadata

Labels

enhancementNew feature or requestgoPull requests that update go codevmcpVirtual MCP Server related issues

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions