Description

Make the StatefulSet replica count configurable by wiring RunConfig.BackendReplicas from pkg/runner/config.go through to the Kubernetes client in pkg/container/kubernetes/client.go, where WithReplicas is currently hardcoded to 1. When BackendReplicas is nil, the field is omitted entirely from the server-side apply spec so that HPA or kubectl retains control of replica scaling; when it is set, the specified value is applied. This change is RC-11 from the proxyrunner horizontal scaling epic.

Context

Part of epic #263 — Horizontal Scaling for Proxyrunner (THV-0047). Today all Kubernetes StatefulSets are deployed with WithReplicas(1) hardcoded in pkg/container/kubernetes/client.go:404. To allow the proxyrunner to run as multiple replicas, or to leave replica management to an HPA, this hardcoded call must be made conditional on whether RunConfig.BackendReplicas is set. TASK-001 (#265) added the BackendReplicas *int32 field to RunConfig; this task threads that value through the call chain: RunConfig → runtime.Setup → DeployWorkloadOptions → client.DeployWorkload → StatefulSetSpec.

Dependencies: #265 (Extend RunConfig with replica and cache-size fields)
Blocks: None (TASK-003, TASK-004 are independent; TASK-005, TASK-006 depend on TASK-004)

Acceptance Criteria

• DeployWorkloadOptions in pkg/container/runtime/types.go has a new BackendReplicas *int32 field
• pkg/runtime/setup.go's Setup function accepts a backendReplicas *int32 parameter and assigns it to containerOptions.BackendReplicas
• pkg/runner/runner.go passes r.Config.BackendReplicas to runtime.Setup
• pkg/container/kubernetes/client.go's DeployWorkload uses options.BackendReplicas: when non-nil, calls appsv1apply.StatefulSetSpec().WithReplicas(*options.BackendReplicas); when nil, omits WithReplicas entirely
• When BackendReplicas is nil, the applied StatefulSet spec does not include a replicas field (verified via server-side apply field inspection or assertion that the spec's .Replicas remains controlled by the previous field manager)
• When BackendReplicas is set to 3, the applied StatefulSet spec includes replicas: 3
• The Docker runtime (pkg/container/docker) ignores BackendReplicas (it has no replica concept); no behavior change for non-Kubernetes deployments
• All existing tests in pkg/container/kubernetes/client_test.go pass without modification
• New unit test: BackendReplicas == nil → WithReplicas absent from applied spec
• New unit test: BackendReplicas == &3 → WithReplicas(3) present in applied spec
• All tests pass

Technical Approach

Recommended Implementation

The change threads a single new pointer field through four layers of the call stack. The key invariant is: pass WithReplicas to the StatefulSetSpec apply config only when BackendReplicas != nil. The applyconfigurations/apps/v1 builder is already used for the StatefulSet spec at client.go:401-410; the change is a conditional wrapping of WithReplicas.

Step 1 — Add BackendReplicas to DeployWorkloadOptions (pkg/container/runtime/types.go): Add a BackendReplicas *int32 field alongside the existing options. This is the transport layer for the value between runner.go and the Kubernetes client.

Step 2 — Thread through runtime.Setup (pkg/runtime/setup.go): Add a backendReplicas *int32 parameter to Setup and assign containerOptions.BackendReplicas = backendReplicas. Update the single call site in pkg/runner/runner.go to pass r.Config.BackendReplicas.

Step 3 — Conditional WithReplicas in the Kubernetes client (pkg/container/kubernetes/client.go): Replace the current unconditional .WithReplicas(1) in the StatefulSetSpec builder with a conditional: build spec := appsv1apply.StatefulSetSpec(), then if options != nil && options.BackendReplicas != nil { spec = spec.WithReplicas(*options.BackendReplicas) }, then chain the remaining .WithSelector(...) etc. onto spec.

Patterns & Frameworks

• Follow the nil-safe replica omission principle from architecture.md Core Principle 1: "Never pass WithReplicas when BackendReplicas is nil — omitting the field lets the field manager leave replicas unchanged, preserving HPA/kubectl control."
• Use k8s.io/client-go/applyconfigurations/apps/v1 builder pattern already established in the file; appsv1apply.StatefulSetSpec() already returns a chainable configuration object.
• The *int32 pointer convention for optional Kubernetes integer fields is established in the Kubernetes API itself (e.g., appsv1.StatefulSetSpec.Replicas) and in RunConfig.BackendReplicas introduced by TASK-001.
• The k8s.io/utils/ptr package is already imported in client.go and can be used for convenience pointer helpers in tests.
• Existing test setup in client_test.go uses fake.NewClientset with a pre-existing mock StatefulSet and client.waitForStatefulSetReadyFunc = mockWaitForStatefulSetReady; follow this pattern for new test cases.

Code Pointers

• pkg/container/kubernetes/client.go:400-410 — The StatefulSetApply builder block where WithReplicas(1) is hardcoded; this is the primary change site
• pkg/container/runtime/types.go:237-287 — DeployWorkloadOptions struct and NewDeployWorkloadOptions constructor; add BackendReplicas *int32 here following the same field comment style
• pkg/runtime/setup.go:38-53 — Setup function signature and body; add backendReplicas *int32 parameter and wire it to containerOptions.BackendReplicas
• pkg/runner/runner.go:291-313 — The runtime.Setup(...) call site; pass r.Config.BackendReplicas as the new argument
• pkg/container/kubernetes/client_test.go:140-238 — Representative test pattern using fake.NewClientset, NewClientWithConfigAndPlatformDetector, and mockWaitForStatefulSetReady; new replica tests should follow this structure
• pkg/container/kubernetes/client_test.go:1096-1235 — Test_isStatefulSetReady test showing how Replicas *int32 is set on the mock StatefulSet spec; useful reference for asserting replica values in new tests

Component Interfaces

The key modification to the StatefulSet apply configuration in pkg/container/kubernetes/client.go:

// Build the StatefulSet spec conditionally including replica count.
statefulSetSpec := appsv1apply.StatefulSetSpec().
    WithSelector(metav1apply.LabelSelector().
        WithMatchLabels(map[string]string{
            "app": containerName,
        })).
    WithServiceName(containerName).
    WithTemplate(podTemplateSpec)

// Only set replicas when explicitly configured; omitting the field lets
// HPA or kubectl retain control of the replica count.
if options != nil && options.BackendReplicas != nil {
    statefulSetSpec = statefulSetSpec.WithReplicas(*options.BackendReplicas)
}

statefulSetApply := appsv1apply.StatefulSet(containerName, namespace).
    WithLabels(containerLabels).
    WithSpec(statefulSetSpec)

New field in DeployWorkloadOptions (pkg/container/runtime/types.go):

// BackendReplicas is the desired StatefulSet replica count.
// When nil, the replicas field is omitted from the apply spec so that
// HPA or kubectl retains control. Only applicable to the Kubernetes runtime.
BackendReplicas *int32

Updated Setup signature in pkg/runtime/setup.go:

func Setup(
    ctx context.Context,
    transportType types.TransportType,
    runtime rt.Deployer,
    containerName string,
    image string,
    cmdArgs []string,
    envVars, labels map[string]string,
    permissionProfile *permissions.Profile,
    k8sPodTemplatePatch string,
    isolateNetwork bool,
    ignoreConfig *ignore.Config,
    host string,
    targetPort int,
    targetHost string,
    backendReplicas *int32,  // new parameter
) (*SetupResult, error)

Testing Strategy

Unit Tests (add to pkg/container/kubernetes/client_test.go)

• TestDeployWorkload_NilBackendReplicas: deploy with options.BackendReplicas == nil; assert the resulting StatefulSet's Spec.Replicas is not set to a specific value by this code (the fake clientset may return a default; verify the applied spec did not include replicas by checking the fake's received object)
• TestDeployWorkload_ExplicitBackendReplicas: deploy with options.BackendReplicas = ptr.To(int32(3)); retrieve the StatefulSet via clientset.AppsV1().StatefulSets("default").Get(...) and assert *statefulSet.Spec.Replicas == 3
• TestDeployWorkload_BackendReplicasOne: deploy with options.BackendReplicas = ptr.To(int32(1)); assert *statefulSet.Spec.Replicas == 1 (explicit 1 is different from nil/omitted)

Integration Tests

• No end-to-end integration tests required for this task; behavior is fully exercised by the unit tests using the fake Kubernetes clientset

Edge Cases

• BackendReplicas pointing to int32(0) — document that passing 0 explicitly is valid (Kubernetes will interpret it as scaling to zero); the code should not special-case 0 vs any other positive value
• options == nil passed to DeployWorkload — the existing nil-guard attachStdio := options == nil || options.AttachStdio pattern at line 349 provides a model; ensure the new conditional also guards options != nil before dereferencing

Out of Scope

• Changes to the RoutingStorage LRU cache or Redis-backed session storage (TASK-004, TASK-005, TASK-006)
• Graceful shutdown / SIGTERM handling (TASK-003)
• CLI flags or CRD fields to expose BackendReplicas to end-users — those are operator/CLI concerns
• Docker or other non-Kubernetes runtimes implementing replica semantics
• HPA creation or management — the operator's responsibility
• Validation that BackendReplicas is within a reasonable range — deferred to consuming operator tooling

References

• Epic #263: Horizontal Scaling for Proxyrunner (THV-0047)
• TASK-001 #265: Extend RunConfig with replica and cache-size fields
• RFC THV-0047
• pkg/container/kubernetes/client.go:400-410 — StatefulSet apply builder, primary change site
• pkg/container/runtime/types.go:237-287 — DeployWorkloadOptions definition
• pkg/runtime/setup.go — Setup function threading RunConfig fields to container options
• pkg/runner/runner.go:291-313 — call site for runtime.Setup
• pkg/container/kubernetes/client_test.go — existing test patterns to follow