Motivation

Numerical logprobs are highly sensitive to GPU type, kernel backend, PyTorch version, and even kernel launch order. This makes CI-based validation unreliable and noisy.

This PR introduces a local, developer-run logprobs test that ensures:

• bitwise consistency of logprobs under deterministic kernels
• reproducible diffs before & after code changes

Updated Test Logic

1. Baseline Generation

   • Fetch 1000 samples from the ShareGPT dataset
   • Randomly choose prompt start positions
   • Run one‑step generation with deterministic kernels
   • Save per‑sample metadata into sglang_baseline_local.pkl

2. Comparison Phase

   • Load the local baseline and re‑run inference on the same prompts
   • For each sample:
     • Compare per‑token logprobs for overlapping top‑k tokens
     • Assert numerical deltas within 1e‑5 tolerance
     • Verify that return_logprob flags behave correctly
   • The test fails fast with detailed per‑sample diffs if thresholds are exceeded

Key Changes

• 🔄 Refactored test_logprobs.py into a standalone test runner with two modes:

  • gen: Generate a local baseline from 1000 ShareGPT samples
  • test: Compare current outputs against the local baseline

• Enabled deterministic execution via enable_deterministic_inference=True

• Enforced strict logprobs thresholds:

  • max Δ ≤ 1e-5

• CLI usage:

# Step 1: generate baseline
python test/srt/test_logprobs.py gen

# Step 2: test after code changes
python test/srt/test_logprobs.py test

Next Steps

We recommend all logprobs-related PRs include test results using this local testing suite to ensure stability across future kernel changes.