🔄 Feature: Bypass DB/Cache Lookup Option for Gateways

Goal

Add a configuration option that allows gateways to operate in "direct proxy" mode, bypassing database caching for tools/resources/prompts. This enables dynamic tool discovery where tools change based on user context or runtime signals without requiring database synchronization.

Why Now?

1. Dynamic Tool Discovery: MCP servers that expose different tools based on user context (permissions, workspace, session) need real-time tool lists
2. Stateless Deployments: Some MCP servers are stateless and don't benefit from caching
3. Reduced Latency: Skip DB lookup for frequently changing tool sets
4. Simplified Operations: Eliminate need to refresh/sync tool registrations

📖 User Stories

Scenario: Direct proxy returns server's current tool list
  Given a gateway with direct_proxy mode enabled
  And an MCP server that returns tools based on user role
  When user with "admin" role calls tools/list
  Then they should see admin-specific tools
  When user with "viewer" role calls tools/list  
  Then they should see viewer-specific tools
  And no tools should be cached in the database

Scenario: Tool changes reflected immediately
  Given a gateway in direct_proxy mode
  And an MCP server that added a new tool
  When the client calls tools/list
  Then the new tool should appear immediately
  And no gateway restart or refresh required

Scenario: Enable direct proxy via API
  Given an existing gateway "dynamic-tools"
  When I update it with direct_proxy=true
  Then all RPC calls should bypass DB lookup
  And tools/list should query upstream directly
  And resources/list should query upstream directly
  And prompts/list should query upstream directly

Scenario: Mixed deployment with both modes
  Given gateway "static-tools" with direct_proxy=false
  And gateway "dynamic-tools" with direct_proxy=true
  Then static-tools should use DB caching
  And dynamic-tools should bypass DB caching
  And both should work simultaneously

Scenario: _meta passed through to upstream server
  Given a gateway in direct_proxy mode
  And a client sends tools/list with _meta containing user info
  When the request reaches the upstream server
  Then _meta should be preserved exactly
  And my server can use _meta.user_id to filter tools

Scenario: _meta preserved in tool invocations
  Given a gateway in direct_proxy mode
  When client calls tools/call with _meta
  Then _meta should be passed to the upstream server
  And the tool can access user context

🏗 Architecture

Request Flow Comparison

flowchart TD
    subgraph "Standard Mode (direct_proxy=false)"
        A1[Client Request] --> B1[Gateway]
        B1 --> C1[DB Lookup]
        C1 --> D1[Return Cached Tools]
    end
    
    subgraph "Direct Proxy Mode (direct_proxy=true)"
        A2[Client Request] --> B2[Gateway]
        B2 --> C2[Forward to MCP Server]
        C2 --> D2[MCP Server]
        D2 --> E2[Return Live Tools]
    end

Database Schema Change

-- Add to gateways table
ALTER TABLE gateways ADD COLUMN direct_proxy BOOLEAN DEFAULT FALSE;

Code Changes

# mcpgateway/db.py - Gateway model
class Gateway(Base):
    # ... existing fields ...
    direct_proxy: Mapped[bool] = mapped_column(Boolean, default=False)

# mcpgateway/services/tool_service.py
async def list_tools(gateway_id: str, request_meta: dict = None):
    gateway = get_gateway(gateway_id)
    
    if gateway.direct_proxy:
        # Bypass DB, forward directly to MCP server
        return await forward_tools_list(gateway, request_meta)
    else:
        # Standard DB lookup
        return get_cached_tools(gateway_id)

📋 Implementation Tasks

Phase 1: Database Schema

• Add direct_proxy boolean column to Gateway model
• Create Alembic migration
• Default to False for existing gateways

Phase 2: Gateway API Updates

• Add direct_proxy field to GatewayCreate schema
• Add direct_proxy field to GatewayUpdate schema
• Add direct_proxy to GatewayResponse schema
• Update API endpoints to handle field

Phase 3: Direct Proxy Logic

• Modify list_tools() to check gateway.direct_proxy
• Modify list_resources() for direct proxy
• Modify list_prompts() for direct proxy
• Ensure _meta is passed through

Phase 4: Tool Invocation

• Ensure tool calls work with direct proxy
• Pass _meta through tool invocations
• Handle errors from unavailable upstream

Phase 5: Admin UI

• Add toggle in gateway create/edit form
• Add indicator in gateway list view
• Add tooltip explaining the option

Phase 6: Testing

• Unit tests for direct proxy flag
• Integration tests for tools/list bypass
• Integration tests for resources/list bypass
• Integration tests for mixed mode deployment
• Test _meta preservation

⚙️ Configuration Example

{
  "name": "Dynamic Tools Server",
  "url": "http://dynamic-server:9000/sse",
  "transport": "sse",
  "direct_proxy": true
}

Environment variable for default (optional):

# Default direct_proxy value for new gateways
GATEWAY_DEFAULT_DIRECT_PROXY=false

✅ Success Criteria

• direct_proxy field added to Gateway model
• Gateways with direct_proxy=true bypass DB lookup
• tools/list returns live data from upstream
• resources/list returns live data from upstream
• prompts/list returns live data from upstream
• _meta is preserved and passed through
• Standard gateways continue using DB cache
• Admin UI shows toggle for the setting
• All tests passing

🏁 Definition of Done

• Database migration created
• Gateway model updated
• API schemas updated
• Direct proxy logic implemented
• Admin UI updated
• Unit tests written and passing
• Integration tests written and passing
• Code passes make verify
• PR reviewed and approved

📝 Additional Notes

Performance Considerations

When to Use Direct Proxy

• Dynamic tools based on user context
• Rapidly changing tool sets
• Stateless MCP servers
• Development/testing

When to Use Standard Mode

• Static tool sets
• High-traffic deployments
• Tools that rarely change
• When upstream may be slow

🔗 Related Issues

• Issue [FEATURE]: Dynamic tools/resources based on user context and server-side signals #2171 - Dynamic tools/resources based on user context
• Related to _meta propagation work