feat: add LLM trace diagnostics to session exports#457
Conversation
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (11)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request implements a comprehensive LLM tracing system that records request metadata, stream events, and token usage, integrating these traces into session processing and exports. The SDK is also updated with new turn-change endpoints and enhanced question-handling capabilities. Feedback was provided to refine the isEmptyCompletion check in the recorder to include all stored part types for better accuracy.
Summary
Adds lightweight LLM trace summaries for assistant model runs and includes them in local session exports.
Why
Session exports currently preserve final messages, parts, and token metadata, but they do not show whether an output anomaly happened at the request boundary, AI SDK normalized stream, PawWork processor, or persisted message. This PR adds count-only diagnostics so issue #454 can be diagnosed from the export without raw prompts, raw model output, headers, API keys, or provider chunks.
Related Issue
Closes #454
Closes #214
Human Review Status
Pending. A human should make the final merge decision after reviewing the final diff and verification evidence.
Review Focus
Please focus on the trace contract and privacy boundary: request summary allowlist, normalized stream event counts, final stored part counts, and whether assistant-message retention is the right persistence layer.
Risk Notes
Additive assistant message diagnostics widen the v2 SDK schema and persist one count-only trace per assistant model run. The trace intentionally omits prompts, messages, headers, API keys, raw provider chunks, raw output text, and tool bodies. Generated SDK files changed after
bun --cwd packages/sdk/js build. No desktop, packaging, updater, signing, path, shell, or permission behavior is changed.How To Verify
Commands run:
Screenshots or Recordings
Not applicable. No visible UI changes.
Checklist
dev, and my PR title and commit messages use Conventional Commits in EnglishRequested maintainer labeling for type, scope, and priority.