Skip to content

perf: add tracing span for main-turn chat_stream LLM call #3619

@bug-ops

Description

@bug-ops

Description

The primary agent turn LLM call — call_chat_with_toolschat_stream — has no dedicated tracing span. Its latency is invisible in local Chrome traces: the gap between `agent.prepare_context` and `agent.process_response` captures the entire LLM streaming call, but there is no named span for it.

All auxiliary LLM calls are instrumented (`llm.chat` via `#[cfg_attr(feature="profiling", tracing::instrument)]`, `llm.embed`, `llm_call` manual span in `native.rs`). The streaming path has no equivalent.

Observed during CI-677 trace analysis of `.local/traces/f9677ab45e154f80ac0dde3b38b070d1_20260505T205611.json`.

Reproduction Steps

  1. Run a session with `telemetry.backend = "local"` and `features = "full"`
  2. Open the trace in Perfetto / chrome://tracing
  3. Observe: no `llm.chat_stream` span anywhere in the trace
  4. The entire LLM network time is a silent gap inside `agent.turn`

Expected Behavior

A `llm.chat_stream` (or `llm.turn_call`) span should appear in the trace covering the full streaming HTTP round-trip: from the first byte sent to the last chunk received and aggregated.

Actual Behavior

No span is emitted. The streaming LLM call is invisible to the local trace system.

Environment

  • Version: v0.20.1 (HEAD: 64dc395)
  • Features: full (profiling enabled)
  • Affected file: `crates/zeph-core/src/agent/tool_execution/native.rs` — `call_chat_with_tools` / `call_chat_no_tools`

Logs / Evidence

Span names present in CI-676 local trace:

  • `llm.chat` (12 B/E events, short durations: 470–4823 µs — auxiliary calls only)
  • `llm.embed` (412 B/E events)
  • `llm_call` (10 B/E events, in non-streaming path)
  • No `llm.chat_stream` entries

The main turn in the same session took ~15 seconds end-to-end (visible in `agent.turn` span), but the network time to the LLM is unattributed.

Metadata

Metadata

Assignees

Labels

P3Research — medium-high complexityllmzeph-llm crate (Ollama, Claude)

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions