Skip to content

Bug: defer span.End() inside for loop in readInputStream() — spans never end until session closes #2549

@marlonbarreto-git

Description

@marlonbarreto-git

Bug Description

readInputStream() in internal/server/mcp.go places defer span.End() inside a for loop. In Go, defer schedules execution at function return, not at the end of the enclosing block. This means every loop iteration creates a new span that is never ended until the entire function returns.

Affected Code

File: internal/server/mcp.go, readInputStream() function

func (s *stdioSession) readInputStream(ctx context.Context) error {
    for {
        // ...
        msgCtx, span := s.server.instrumentation.Tracer.Start(msgCtx, "toolbox/server/mcp/stdio",
            trace.WithSpanKind(trace.SpanKindServer),
        )
        defer span.End()   // BUG: defers accumulate, never run until function exits
        // ...
    }
}

Impact

For long-running STDIO MCP sessions processing hundreds or thousands of messages:

  1. Memory leak: Every span object accumulates in the tracer's active-span registry for the entire session lifetime
  2. Wrong span durations: All spans show duration = (session end time - message receive time) instead of actual processing time
  3. OTel exporter flood: On session disconnect, all accumulated spans are ended simultaneously, potentially overwhelming the exporter
  4. Deferred function stack growth: Go accumulates all deferred calls on the goroutine's stack, growing linearly with message count

Comparison with Correct Usage

The same file uses defer span.End() correctly outside loops:

  • sseHandler() line ~325 — defer at function scope (correct)
  • processMcpMessage() line ~575 — defer at function scope (correct)

The cleanupRoutine() (lines ~100-109) correctly uses a closure pattern to scope defers per iteration.

Proposed Solution

Option A (recommended): Call span.End() explicitly at the end of the loop body

for {
    msgCtx, span := s.server.instrumentation.Tracer.Start(...)
    // ... process message ...
    span.End()   // End span immediately after processing
}

Option B: Wrap the loop body in a closure to scope the defer

for {
    func() {
        msgCtx, span := s.server.instrumentation.Tracer.Start(...)
        defer span.End()
        // ... process message ...
    }()
}
Aspect Option A Option B
Clarity More explicit, easier to reason about Familiar defer pattern but adds nesting
Error handling Must ensure span.End() is called on all paths defer handles all paths automatically
Performance No closure overhead Small closure allocation per iteration

Happy to submit a PR for this fix.

Metadata

Metadata

Assignees

Labels

priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions