Skip to content

Delete dead rematch path in SessionAwareCache.release_session#22735

Merged
hnyls2002 merged 1 commit intomainfrom
lsyin/session-cache-dead-code
Apr 14, 2026
Merged

Delete dead rematch path in SessionAwareCache.release_session#22735
hnyls2002 merged 1 commit intomainfrom
lsyin/session-cache-dead-code

Conversation

@hnyls2002
Copy link
Copy Markdown
Collaborator

Summary

Delete the unreachable rematch logic in SessionAwareCache._resolve_release_state, inline the trivial remainder into release_session, and drop the two unit tests that exercised only the dead path.

Why this is dead code

The rematch block was introduced in #21875 together with an early-return TODO that disabled it in the same commit:

# TODO: re-match logic disabled -- match_prefix has side effects
# (splits) that disturb tree accounting. Directly using
# slot.last_node + cache_protected_len is safe after split analysis.
return protected_len, lock_node

if (...):  # unreachable from here on
    ...
    rematch = self.inner.match_prefix(...)

So the ~40-line block under the early return has never executed in production. It was kept as a placeholder for "future rematch if split-safety assumption breaks."

Why rematch isn't needed

RadixCache._split_node mutates TreeNode objects in place:

  • child node stays alive; only its key / value get shortened to the post-split suffix.
  • new_node is inserted above it with the pre-split prefix; lock_ref propagates.

So slot.last_node = child stays a valid reference across splits, and slot.cache_protected_len (locked token count) is unchanged. The (last_node, cache_protected_len) pair remains correct for dec_lock_ref without any rematch.

Running match_prefix in release_session would actively cause further splits -- exactly the side effect the TODO comment flags.

What the PR does

  1. Delete _resolve_release_state entirely.
  2. Inline the trivial assignment at the release_session call site; rewrite the docstring to capture the split-safety reasoning.
  3. Remove the now-unused req parameter from release_session (single production caller and one unit-test caller updated).
  4. Delete the two unit tests (test_release_session_recomputes_current_tree_owned_prefix, test_release_session_never_grows_tree_owned_prefix) -- they set up _FakeInnerCache match_results and asserted rematch outputs that were never produced.

Test plan

  • Syntax check all touched files
  • Remaining 5 unit tests in test_streaming_session_unit.py pass
  • Full streaming session integration suite passes on H200 with strict busy mem check (SGLANG_ENABLE_STRICT_MEM_CHECK_DURING_BUSY=2):
    • TestStreamingSession + TestStreamingSessionMixedChunk: 10/10
    • TestStreamingSessionRetract + TestStreamingSessionRetractMixedChunk: 10/10
    • TestStreamingSessionEagle + TestStreamingSessionEagleV2: 10/10 (1 expected skip for spec-v2 overshoot)
    • TestStreamingSessionAbortLeakRepro: 1/1
    • Zero leak assertions across all runs.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_streaming_session.py

@github-actions
Copy link
Copy Markdown
Contributor

1-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/sessions/test_streaming_session.py

@hnyls2002 hnyls2002 merged commit 33a3ba2 into main Apr 14, 2026
62 of 70 checks passed
@hnyls2002 hnyls2002 deleted the lsyin/session-cache-dead-code branch April 14, 2026 00:02
yhyang201 pushed a commit to yhyang201/sglang that referenced this pull request Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant