Skip to content

Wire session-aware backend routing and LRU cache-size in proxy transports (RC-12, RC-13) #4212

@yrobla

Description

@yrobla

Description

Switch all three proxy transport constructors (streamable, HTTP-SSE, and transparent) to use session.NewManagerWithRedis, enabling session-sticky routing across multiple proxyrunner replicas (RC-12).

Note: RC-13 (LRU cache-size wiring) has been dropped. RedisStorage.Load uses GETEX, which atomically returns session data and refreshes the TTL in a single round-trip, making a local LRU cache layer unnecessary. See #4294 for the rationale.

Parent epic: stacklok/stacklok-epics#263

Dependencies: #4236 (RC-7 — NewManagerWithRedis constructor), #4239 (TASK-001 — RunConfig fields)
Blocks: none (leaf node in the DAG)

Acceptance Criteria

RC-12 — Session-aware routing

  • NewHTTPProxy (streamable): constructs session manager via session.NewManagerWithRedis; accepts a Redis storage override via WithSessionStorage option (defaults to session.NewLocalStorage() when not provided)
  • NewHTTPSSEProxy: same
  • NewTransparentProxyWithOptions: same
  • Transparent proxy Rewrite reads "backend_url" metadata key from session and overrides pr.Out.URL; falls back to static targetURL when absent or unparseable
  • Non-initialize requests with unknown session ID return HTTP 400 in transparent proxy handler
  • Existing single-replica behavior preserved when no WithSessionStorage is provided (defaults to LocalStorage)

General

  • All existing tests in pkg/transport/proxy/streamable/, pkg/transport/proxy/httpsse/, and pkg/transport/proxy/transparent/ pass
  • go vet ./pkg/transport/proxy/... reports no issues
  • Tests for transparent proxy Rewrite: (a) backend_url in metadata overrides target; (b) no backend_url falls back to static target; (c) unknown session ID on non-initialize returns HTTP 400
  • Code reviewed and approved

Technical Approach

RC-12 — Storage wiring

Add WithSessionStorage(storage session.Storage) Option to each proxy constructor. When not provided, default to session.NewLocalStorage(). Pass the storage to session.NewManagerWithRedis:

// At call sites in pkg/transport/http.go, when Redis is configured:
mgr := session.NewManagerWithRedis(redisClient)
// Otherwise fall back to local storage (single-replica default)

Extend the transparent proxy Rewrite:

if sid := pr.In.Header.Get("Mcp-Session-Id"); sid != "" {
    if sess, ok := p.sessionManager.Get(sid); ok {
        if backendURL, _ := sess.GetMetadataValue("backend_url"); backendURL != "" {
            if parsed, err := url.Parse(backendURL); err == nil {
                pr.Out.URL = parsed
            }
        }
    }
}

Code Pointers

  • pkg/transport/proxy/streamable/streamable_proxy.go:72–98NewHTTPProxy; session.NewManagersession.NewManagerWithRedis
  • pkg/transport/proxy/streamable/streamable_proxy.go:617–621resolveSessionForBatch; pattern for HTTP 400 on unknown session
  • pkg/transport/proxy/httpsse/http_proxy.go:93–120NewHTTPSSEProxy; same replacement
  • pkg/transport/proxy/transparent/transparent_proxy.go:253–310NewTransparentProxyWithOptions; session manager wiring
  • pkg/transport/proxy/transparent/transparent_proxy.go:487–513Rewrite closure; backend_url override injected here
  • pkg/transport/http.go — call sites for both proxy constructors

Out of Scope

  • RoutingStorage / LRU cache layer (dropped — see #4294)
  • RunConfig.SessionCacheSize wiring (no longer needed)
  • Backend pod selection / round-robin on initialize
  • StatefulSet replica count wiring (TASK-002) or graceful shutdown (TASK-003)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestscalabilityItems related to scalabilityvmcpVirtual MCP Server related issues

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions