Enable sticky sessions on operator-created Services#3986
Merged
Conversation
MCP servers use stateful session protocols (SSE, streamable-http). When replicas > 1, Kubernetes round-robin routing breaks sessions. Set SessionAffinity: ClientIP on all operator-created Services (MCPServer, MCPRemoteProxy, VirtualMCPServer) so requests from the same client consistently reach the same backend pod. Also add drift detection in serviceNeedsUpdate() and copy SessionAffinity in the ensureService update paths so existing Services get reconciled. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document that all operator-created Services use SessionAffinity: ClientIP to support stateful MCP sessions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3986 +/- ##
==========================================
- Coverage 68.56% 68.51% -0.05%
==========================================
Files 437 437
Lines 44662 44674 +12
==========================================
- Hits 30621 30609 -12
- Misses 11657 11682 +25
+ Partials 2384 2383 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Member
|
Should we make this configurable by the user? |
Collaborator
Author
|
@eleftherias we could, though I don't know in what case you wouldn't want this. |
Member
|
If the server is stateless then the stickiness isn't needed and isn't the best way to distribute the load. Also if they use an external load balancer this could cause conflicts with the policy they've configured. |
Collaborator
Author
|
@eleftherias gotcha. In most cases the MCP server is stateful, though. And this is the case for our proxy as well. Should we then default to havin this enabled and have a flag to optionally disable this? |
Member
|
Yes that makes sense |
Collaborator
Author
|
@eleftherias should I add the configurability to a separate PR or to this one? |
eleftherias
approved these changes
Mar 4, 2026
Member
|
Separate PR works |
JAORMX
added a commit
that referenced
this pull request
Mar 4, 2026
PR #3986 hardcoded SessionAffinity: ClientIP on all operator-created Services. Review feedback identified this should be configurable: stateless MCP servers don't need stickiness and external load balancers may conflict. Add a sessionAffinity field (enum: ClientIP/None, default: ClientIP) to MCPServerSpec, MCPRemoteProxySpec, and VirtualMCPServerSpec. Wire the field through service creation and drift detection in all three controllers. Also fix a pre-existing bug in the MCPServer controller where the service update path did not copy Labels/Annotations, which could cause infinite reconciliation when those fields drifted. EmbeddingServer is intentionally excluded — embedding servers are stateless inference endpoints, not MCP protocol services. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
6 tasks
reyortiz3
pushed a commit
that referenced
this pull request
Mar 4, 2026
* Enable sticky sessions on operator-created Services MCP servers use stateful session protocols (SSE, streamable-http). When replicas > 1, Kubernetes round-robin routing breaks sessions. Set SessionAffinity: ClientIP on all operator-created Services (MCPServer, MCPRemoteProxy, VirtualMCPServer) so requests from the same client consistently reach the same backend pod. Also add drift detection in serviceNeedsUpdate() and copy SessionAffinity in the ensureService update paths so existing Services get reconciled. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update operator architecture docs for SessionAffinity Document that all operator-created Services use SessionAffinity: ClientIP to support stateful MCP sessions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
eleftherias
pushed a commit
that referenced
this pull request
Mar 4, 2026
PR #3986 hardcoded SessionAffinity: ClientIP on all operator-created Services. Review feedback identified this should be configurable: stateless MCP servers don't need stickiness and external load balancers may conflict. Add a sessionAffinity field (enum: ClientIP/None, default: ClientIP) to MCPServerSpec, MCPRemoteProxySpec, and VirtualMCPServerSpec. Wire the field through service creation and drift detection in all three controllers. Also fix a pre-existing bug in the MCPServer controller where the service update path did not copy Labels/Annotations, which could cause infinite reconciliation when those fields drifted. EmbeddingServer is intentionally excluded — embedding servers are stateless inference endpoints, not MCP protocol services. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SessionAffinity: ClientIPon all three operator-created Service types (MCPServer, MCPRemoteProxy, VirtualMCPServer) so stateful MCP session protocols (SSE, streamable-http) work correctly when replicas > 1serviceNeedsUpdate()for each controller so existing Services withoutClientIPaffinity get reconciledSessionAffinityin theensureServiceupdate paths alongside existingPorts/TypecopiesTest plan
task lint-fix— 0 issuesgo test -race ./cmd/thv-operator/controllers/...— all passtask test— all pass (one pre-existing flaky test unrelated: Flaky test: TestServer_HealthMonitoring_Enabled #3985)kubectl get svc <name> -o jsonpath='{.spec.sessionAffinity}'returnsClientIP🤖 Generated with Claude Code