Delete dead rematch path in SessionAwareCache.release_session#22735
Merged
Delete dead rematch path in SessionAwareCache.release_session#22735
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Collaborator
Author
|
/rerun-test test_streaming_session.py |
Contributor
|
✅ |
pyc96
pushed a commit
to pyc96/sglang
that referenced
this pull request
Apr 14, 2026
yhyang201
pushed a commit
to yhyang201/sglang
that referenced
this pull request
Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Delete the unreachable rematch logic in
SessionAwareCache._resolve_release_state, inline the trivial remainder intorelease_session, and drop the two unit tests that exercised only the dead path.Why this is dead code
The rematch block was introduced in #21875 together with an early-return TODO that disabled it in the same commit:
So the ~40-line block under the early return has never executed in production. It was kept as a placeholder for "future rematch if split-safety assumption breaks."
Why rematch isn't needed
RadixCache._split_nodemutatesTreeNodeobjects in place:childnode stays alive; only itskey/valueget shortened to the post-split suffix.new_nodeis inserted above it with the pre-split prefix;lock_refpropagates.So
slot.last_node = childstays a valid reference across splits, andslot.cache_protected_len(locked token count) is unchanged. The(last_node, cache_protected_len)pair remains correct fordec_lock_refwithout any rematch.Running
match_prefixinrelease_sessionwould actively cause further splits -- exactly the side effect the TODO comment flags.What the PR does
_resolve_release_stateentirely.release_sessioncall site; rewrite the docstring to capture the split-safety reasoning.reqparameter fromrelease_session(single production caller and one unit-test caller updated).test_release_session_recomputes_current_tree_owned_prefix,test_release_session_never_grows_tree_owned_prefix) -- they set up_FakeInnerCachematch_results and asserted rematch outputs that were never produced.Test plan
test_streaming_session_unit.pypassSGLANG_ENABLE_STRICT_MEM_CHECK_DURING_BUSY=2):TestStreamingSession+TestStreamingSessionMixedChunk: 10/10TestStreamingSessionRetract+TestStreamingSessionRetractMixedChunk: 10/10TestStreamingSessionEagle+TestStreamingSessionEagleV2: 10/10 (1 expected skip for spec-v2 overshoot)TestStreamingSessionAbortLeakRepro: 1/1