fix(responsecache): split stream cache entries and validate SSE#202
fix(responsecache): split stream cache entries and validate SSE#202SantiagoDePolonia merged 3 commits intomainfrom
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR separates streaming and non-streaming cache entries, stores validated raw SSE bytes for streaming misses, replays those bytes verbatim on streaming hits, prevents caching of malformed SSE, and extends usage extraction to parse cached SSE bodies for token/usage metrics. Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/responsecache/sse_validation_test.go`:
- Around line 12-64: Add a new table-driven test case to the existing test cases
slice in sse_validation_test.go that verifies empty input is rejected: add an
entry with name "rejects empty input", raw: []byte("") and want: false so the
SSE validation logic (the same test harness used for the other cases) explicitly
asserts that an empty payload returns false.
In `@internal/responsecache/sse_validation.go`:
- Around line 28-40: The loop accepts events after a seen `[DONE]` because the
sawDone check runs after early-continue paths (comments/keepalives and
empty-data), so move the sawDone check so it runs immediately after calling
sseEventPayload(event) (i.e., right after "payload, hasData :=
sseEventPayload(event)") and before any continue; this ensures that once sawDone
is set (when payload equals cacheDonePayload) any subsequent event — including
comment/keepalive-only or another `[DONE]` — triggers an immediate false return.
- Around line 43-46: The current validation unmarshals into a map using
json.Unmarshal(payload, &decoded), allocating an unused map and incorrectly
rejecting valid non-object JSON; replace that logic with a syntax-only check
using json.Valid(payload) and remove the unused decoded variable so the function
simply returns false when json.Valid(payload) is false, otherwise proceeds as
valid. Ensure you reference the existing payload variable and remove the
json.Unmarshal call and decoded declaration, using json.Valid(payload) instead.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 41a69a42-06bf-431a-b7d8-14e1417b240c
📒 Files selected for processing (11)
internal/responsecache/handle_request_test.gointernal/responsecache/middleware_test.gointernal/responsecache/responsecache.gointernal/responsecache/semantic.gointernal/responsecache/semantic_test.gointernal/responsecache/simple.gointernal/responsecache/sse_validation.gointernal/responsecache/sse_validation_test.gointernal/responsecache/stream_cache.gointernal/usage/extractor.gointernal/usage/extractor_test.go
Summary
stream=trueandstream=falseas separate exact and semantic cache entriesTesting
go test ./internal/responsecache ./internal/usage ./internal/serverSummary by CodeRabbit
New Features
Bug Fixes
Tests