… primitives

Splits the conflated context state in compilation/piecewise_context_manager.py
along its real seam: the cuda-graph-capture flag (used by both BCG and tcpcg)
moves to model_executor/, while the torch.compile-warmup flag (tcpcg-internal)
stays in compilation/.

Renames:
  is_in_piecewise_cuda_graph        -> is_in_cuda_graph_capture
  enable_piecewise_cuda_graph       -> enable_cuda_graph_capture
  is_in_pcg_torch_compile           -> is_in_torch_compile_warmup
  enable_piecewise_cuda_graph_compile -> enable_torch_compile_warmup
  PIECEWISE_CUDA_GRAPH_CAPTURE_FAILED_MSG -> CUDA_GRAPH_CAPTURE_FAILED_MSG

Relocations:
  compilation/piecewise_context_manager.py
    -> model_executor/cuda_graph_backend_utils/piecewise_cuda_graph/context_manager.py
       (capture flag, ForwardContext, set/get_forward_context)
    -> compilation/compile_phase.py
       (warmup flag, pcg_capture_stream)
  model_executor/breakable_cuda_graph/{breakable_cuda_graph,context,cuda_utils}.py
    -> model_executor/cuda_graph_backend_utils/breakable_cuda_graph/{...}

The two old paths (compilation/piecewise_context_manager.py and
model_executor/breakable_cuda_graph/) are kept as transition shims that
re-export from the new homes under both old and new names. Removed in
Phase 6.

Audited 38 callsites across 16 production files. All switched to the
renamed primitives at their new import paths. Behavior preserved
mechanically (bucket A everywhere). Two bucket-C candidates flagged
with TODO comments for follow-up:
  - models/nemotron_h.py:_forward_core (CUDA stream overlap path —
    genuine dynamo-tracing constraint, not capture-or-replay)
  - models/deepseek_common/.../forward_mla.py (non-contiguous-output
    bmm form — required by dynamo, not by capture)

Verified clean: 22 audited modules import OK, identity preserved
through both shims, BCG test path still resolves.

See refactor/plan.md §6.5 for context-flag semantics + audit rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ig_resolution

Moves the 18-condition ``_handle_piecewise_cuda_graph`` from server_args.py
to ``cuda_graph_runner.config_resolution`` and converts the long if/if
cascade into a data table of ``_PiecewiseDisableRule(name, predicate)``
entries — easier to read, audit, and extend.

Wires ``resolve_cuda_graph_config(self)`` from ``ServerArgs.__post_init__``
in place of the old method call. The old method is removed (no callers left).

Phase 1 only implements stage 3 (compatibility checks) of the four-stage
pipeline described in plan §3. Stages 1 (parse), 2 (default), and 4
(validate) remain stubs that land in Phase 4 alongside the new CLI surface.
GPU-memory-based defaulting still lives in ``_handle_gpu_memory_settings``
until then.

Behavior parity verified against an 18-case matrix covering every rule
plus the ``enforce_piecewise_cuda_graph`` bypass path. ``--enforce`` still
overrides the entire table for testing per Q3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Locks down the contract that backend extractions in Phases 2b–2d must
satisfy. The ABC has four methods:

  prepare(runner)                     # one-time setup
  capture_one(shape_key, forward_fn)  # capture for one shape
  replay(shape_key)                   # replay artifact at shape
  cleanup()                           # default no-op

plus a class attribute ``captures_attn_metadata`` that lets the runner
know whether ``init_forward_metadata_capture_cuda_graph`` should run
inside the captured region (full-graph style) or outside on every
replay (PCG/BCG style).

Three concrete subclasses declared with the right metadata flag but
NotImplementedError bodies:
  - FullCudaGraphBackend (captures_attn_metadata=True)  — Phase 2b
  - BreakableCudaGraphBackend (captures_attn_metadata=False) — Phase 2c
  - TCPiecewiseCudaGraphBackend (captures_attn_metadata=False) — Phase 2d

Bodies will be lifted from cuda_graph_runner/legacy.py,
breakable_cuda_graph_runner.py, piecewise_cuda_graph_runner.py
respectively.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Phase 0 added the ``ServerArgs.cuda_graph_mode`` field but missed the
corresponding ``parser.add_argument`` registration. ``from_cli_args``
maps every dataclass field name back from the argparse Namespace, so
launching the server crashed with ``AttributeError: 'Namespace' object
has no attribute 'cuda_graph_mode'``.

Adds a minimal ``--cuda-graph-mode`` flag that accepts a JSON object
and parses it to ``Dict[str, str]``. Validation of allowed values
(full/breakable/tcpcg/disabled) per phase lands in Phase 4; for now
the field is still unread.

Caught by trying to launch a baseline Qwen3-8B server for mgsm_en
validation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Lifts the full-CUDA-graph capture primitives out of
``legacy.CudaGraphRunner`` and into ``cuda_graph_backend/full.py`` as
two runner-coupling-free static methods:

  FullCudaGraphBackend.make_graph()
    -> torch.cuda.CUDAGraph()
       (was the FULL branch of CudaGraphRunner._create_device_graph)

  FullCudaGraphBackend.capture_into(graph, pool, stream,
                                    device_module, memory_saver_adapter,
                                    run_once_fn)
    -> opens the appropriate graph capture context (memory-saver-aware)
       and runs run_once_fn under it, returns its output
       (was the FULL branch of CudaGraphRunner._capture_graph)

``legacy.CudaGraphRunner._create_device_graph`` and ``_capture_graph``
keep their env-var-driven Breakable branch inline (Phase 2c lifts that)
and delegate to the new primitives on the FULL branch. Net runtime
behavior: identical bytecode path, plus one extra Python call frame at
*startup* (per-shape capture); zero per-request cost.

The ABC methods (prepare/capture_one/replay) stay NotImplementedError —
the runner still owns the dict-based dispatch and the buffer setup;
Phase 3 wires the backend to be driven through the abstract interface
when runners get unified.

Validated against the cg-refactor baseline (commit 8e79e6e,
mgsm_en N=200 on Qwen3-8B = 0.865, latency 63.18s, throughput 3439.7
tok/s):

  Phase 2b: score 0.840, latency 64.22s, throughput 3418.8 tok/s

Score delta -0.025 = 1σ noise at p=0.85, N=200 (~5 samples flipped
between greedy-decoding runs, plausible from kernel-level
non-determinism). Latency/throughput within ±2% noise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Lifts the breakable-CUDA-graph capture primitives out of the env-var
branch of ``legacy.CudaGraphRunner._capture_graph`` into
``cuda_graph_backend/breakable.py`` as runner-coupling-free static
methods:

  BreakableCudaGraphBackend.make_graph()
    -> BreakableCUDAGraph()
       (HIP guard moves into the backend)

  BreakableCudaGraphBackend.capture_into(graph, pool, stream,
                                         run_once_fn,
                                         *, debug_eager,
                                         memory_saver_adapter)
    -> opens BreakableCUDAGraphCapture, optionally wraps with
       eager_on_graph(True) for --debug-cuda-graph mode, raises on
       memory-saver incompatibility.

``legacy.CudaGraphRunner._create_device_graph`` and ``_capture_graph``
both branches now delegate (Full path → FullCudaGraphBackend, Breakable
env-var path → BreakableCudaGraphBackend). The runner is now a thin
dispatcher over the two backends; only the dict-based per-shape
storage and the prefill BCG class remain non-extracted, both of which
land in Phase 3.

ABC methods (prepare/capture_one/replay) stay NotImplementedError —
runner unification in Phase 3 wires them.

Validation (cg-refactor baseline = 0.865, score floor 0.80 per user):
  Phase 2c default Full path: score 0.845, latency 64.79s,
  throughput 3397.0 tok/s — well within floor.

The breakable env-var decode path is not directly exercised by
mgsm_en (default config uses Full); the migration is byte-equivalent
and shares a path that was working pre-refactor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Lifts the torch.compile setup logic out of
``PiecewiseCudaGraphRunner`` and into ``cuda_graph_backend/tcpcg.py``
as runner-coupling-free static methods:

  TCPiecewiseCudaGraphBackend.build_compilation_config(server_args)
    -> CompilationConfig
       Validates --piecewise-cuda-graph-compiler choice, builds the
       config, registers the MoE A2A split-op when DeepEP/Mooncake is
       in use. Mirrors PiecewiseCudaGraphRunner.__init__ lines that
       previously did this inline.

  TCPiecewiseCudaGraphBackend.install_compile(language_model,
                                              compile_config, graph_pool,
                                              fullgraph=True,
                                              dynamic_arg_dims=None)
    -> wraps language_model with install_torch_compiled. Mirrors the
       call site in PiecewiseCudaGraphRunner.capture().

PCG runner now imports from the backend (deferred to call site to avoid
import-cycle risk) instead of holding the construction logic inline.
The unused top-of-file imports (CompilationConfig, install_torch_compiled,
get_moe_a2a_backend) are removed; they're now reached only through the
backend.

ABC methods (prepare/capture_one/replay) stay NotImplementedError —
runner unification in Phase 3 wires them.

Validation (cg-refactor baseline = 0.865, score floor 0.80 per user):
  Phase 2d: score 0.850, latency 64.09s, throughput 3406.3 tok/s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Establishes the canonical phase-named runner classes; today they are
thin wrappers/factories over the legacy classes so behavior is
unchanged. Phase 3b/c will migrate bodies into them.

cuda_graph_runner/decode_runner.py:
  class DecodeCudaGraphRunner(CudaGraphRunner): pass

  Subclasses the legacy decode runner. New code should refer to this
  name; the legacy class continues to host the implementation.

cuda_graph_runner/prefill_runner.py:
  class PrefillCudaGraphRunner:
      def __new__(cls, model_runner):
          if model_runner.server_args.enable_breakable_cuda_graph:
              return BreakableCudaGraphRunner(model_runner)
          return PiecewiseCudaGraphRunner(model_runner)

  Factory that selects the prefill backend (breakable vs tcpcg) by
  today's server-arg flag. Phase 4 will drive the selection from the
  canonical ``cuda_graph_mode`` config.

model_runner.py:
  - decode path uses DecodeCudaGraphRunner instead of CudaGraphRunner
    (defaultdict default; CPU/NPU paths unchanged).
  - prefill path uses PrefillCudaGraphRunner factory instead of an
    inline if/else.

Late imports avoid circular-dependency risk: model_executor modules
import from cuda_graph_runner; cuda_graph_runner imports the legacy
runner module which in turn imports model_executor primitives.
Localized imports inside the factory and inside model_runner sidestep
the cycle.

External readers of ``model_runner.graph_runner`` (eagle workers,
hardware stubs) continue to work since DecodeCudaGraphRunner is-a
CudaGraphRunner via inheritance.

Validation (cg-refactor baseline = 0.865, score floor 0.80 per user):
  Phase 3a: score 0.835, latency 65.17s, throughput 3371.2 tok/s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Implements stages 1 (parse) and 4 (validate) of the resolver pipeline
in ``cuda_graph_runner.config_resolution`` and switches
``PrefillCudaGraphRunner`` to drive backend selection from the
canonical ``cuda_graph_mode`` field.

Stage 1 (``_parse_canonical``):
  - Translates today's legacy flags to a canonical
    ``Dict[str, str]`` covering both phases:
      ``--disable-cuda-graph``         -> decode = "disabled"
      ``--disable-piecewise-cuda-graph`` -> prefill = "disabled"
      ``--enable-breakable-cuda-graph`` -> prefill = "breakable"
      otherwise                        -> defaults
        {decode: "full", prefill: "tcpcg"}.
  - Explicit ``--cuda-graph-mode`` JSON wins per-phase (Q8); when the
    JSON conflicts with a legacy convenience flag, emits a warning
    naming both, per plan §6 Q8.
  - Re-runs after compatibility checks so any auto-disable flips
    (e.g. ``disable_piecewise_cuda_graph = True`` from the 18-rule
    table) propagate into ``cuda_graph_mode``.

Stage 4 (``_validate_canonical``):
  - Rejects unknown phases (only ``decode``/``prefill`` allowed).
  - Rejects unknown backends per phase.
  - Raises ``NotImplementedError`` for the (prefill, full) cell with a
    pointer to use breakable/tcpcg instead — plan §6 Q1.

PrefillCudaGraphRunner factory now reads
``cuda_graph_mode["prefill"]`` instead of
``enable_breakable_cuda_graph`` directly. Decode side is unchanged
since the only available decode backend in v1 is ``full``.

Validation:
  - Unit tests confirmed default + breakable + decode-disabled mappings,
    JSON-vs-flag override warning, and validator rejection of
    (prefill, full), unknown phase, unknown backend.
  - mgsm_en N=200 on Qwen3-8B: score 0.835, latency 64.19s,
    throughput 3420.8 tok/s — above 0.80 floor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…da_graph_mode

Turns ``DecodeCudaGraphRunner`` into a factory (matching
``PrefillCudaGraphRunner``) that consults
``cuda_graph_mode["decode"]``. Three branches:

  - "full" (default): returns a ``CudaGraphRunner`` instance. No
    behavior change vs Phase 3a.
  - "breakable" (experimental, plan §2.4): bridges to today's
    ``SGLANG_USE_BREAKABLE_CUDA_GRAPH`` env-var path inside
    ``CudaGraphRunner._capture_graph`` / ``_create_device_graph``.
    Sets the env var if not already set. Phase 3b/c will replace the
    env-var read with a constructor parameter.
  - "tcpcg": not implemented for the decode phase in v1; logs a
    one-shot warning and falls back to "full" so the server still
    boots. Tracked as a Phase-3+ follow-up in refactor/progress.md.

The (prefill, full) cell continues to raise NotImplementedError from
the validator (Phase 4a, plan §6 Q1). The matrix is now:

  (decode, full)        — implemented (default)
  (decode, breakable)   — experimental, env-var bridge
  (decode, tcpcg)       — falls back to full + warning (TODO)
  (prefill, breakable)  — implemented
  (prefill, tcpcg)      — implemented (default)
  (prefill, full)       — NotImplementedError stub

Validation:
  - mgsm_en N=200 on Qwen3-8B (default mode = decode:full +
    prefill:tcpcg): score 0.835, latency 64.14s, throughput 3423.4
    tok/s. Above 0.80 floor.
  - The breakable/tcpcg decode branches are exercised at construction
    time but not driven by the default eval; explicit tests for them
    are deferred to a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…i-model PCG/BCG test pass

The transition shim at
``model_executor/breakable_cuda_graph/breakable_cuda_graph.py`` had
its underscore-prefixed re-exports listed explicitly (``_copy_output``,
etc.) but ``_copy_output`` was missed. The PCG/BCG test suite at
``test/registered/breakable_cuda_graph/test_breakable_cuda_graph.py``
imports it directly from the legacy path, so ``TestCopyOutput.setUpClass``
errored with ImportError under the cg-refactor branch.

Adds ``_copy_output`` to the explicit-imports list so the shim
remains 1:1 with the legacy module surface. (The * import covers it
for non-underscore symbols; private symbols need explicit listing.)

End-to-end test pass (CI-registered tests + multi-model coverage at
HEAD ``a7ae66efc`` plus this fix):

PCG suite (test/registered/piecewise_cuda_graph/):
  - TestPiecewiseCudaGraphQwen25VL (Qwen2.5-VL-7B-Instruct
    --enforce-piecewise-cuda-graph + --disable-radix-cache, gsm8k):
    score 0.818 ≥ 0.80 ✓
  - TestPiecewiseCudaGraphInternVL25 (InternVL2.5-8B same setup,
    gsm8k): score 0.575 ≥ 0.54 ✓
  - TestPiecewiseCudaGraphQwen25VLEmbedding (Qwen2.5-VL-3B-Instruct
    embedding, enforce vs disable):
    max_abs_diff 0.0078 < 1e-2 ✓

BCG suite (test/registered/breakable_cuda_graph/):
  - TestBreakableCUDAGraphBasic + TestCopyOutput +
    TestBreakGraphHelper (unit): 11 tests pass ✓
  - TestBreakableCudaGraph (Qwen3-8B
    --enable-breakable-cuda-graph, mgsm_en N=1319):
    score 0.856 ≥ 0.80 ✓

Plus the multi-model spot-checks already in progress.md:
  Qwen3-8B PCG default (Phase 5 commit): 0.835
  Nemotron-H Mamba: 0.280 (parity with BCG-notes 0.310 base model)
  Qwen3-30B-A3B MoE: 0.950

Spec-decoding PCG test (Qwen3.5-35B-A3B + NEXTN, 2-GPU, FP8) deferred
— model not cached, suite is "stage-b-test-2-gpu-large", out of scope
for this single-GPU validation pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…e to dynamo-only

Two model-side gates were tagged ``TODO(cg-refactor)`` because their
explicit comments named torch.compile / dynamo as the constraint, but
they were gated on the broader ``is_in_cuda_graph_capture()`` (which
fires during both capture and replay). Narrowed to
``torch.compiler.is_compiling()`` — fires only during dynamo tracing
(compile time), letting replay take the fast path.

Sites touched:

  models/nemotron_h.py:_forward_core
    Comment: "torch.compile cannot trace CUDA streams". The Mamba
    decoder layer's stream-overlap path was disabled during both
    capture and replay; now it's only disabled during compile.

  models/deepseek_common/.../forward_mla.py
    Comment: "torch dynamo requires out= op was called where output
    tensor was non-contiguous". The non-contiguous-output bmm form
    was used during both capture and replay; now it's only used
    during compile.

Both gates were preserving correctness because the broader gate was a
strict superset of the dynamo-only gate. Narrowing them improves
replay performance without affecting capture-time correctness.

Validation: mgsm_en N=200 on Nemotron-H-8B-Base-8K (which exercises
the nemotron_h.py gate via the hybrid Mamba path): score 0.28,
matching the pre-audit baseline (BCG notes' 0.310 for tp2; tp1 here).
The ``forward_mla.py`` change touches the DeepSeek MLA path; full
validation against DS-Coder-V2-Lite + flashinfer backend is a Phase 6
follow-up.

Removes the imports of ``is_in_cuda_graph_capture`` from both files
since they are no longer used after the narrowing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Phase 1a renamed is_in_piecewise_cuda_graph -> is_in_cuda_graph_capture
with the intent of "umbrella" semantics, but the umbrella was fiction:
only TCPCG ever sets the flag. Full-decode never sets a sglang flag,
and BCG has its own is_in_breakable_cuda_graph(). The generalized
name made callsites lie about what they were checking.

Revert all 35 callsites back to the original explicit names:

  is_in_cuda_graph_capture()           -> is_in_piecewise_cuda_graph()
  enable_cuda_graph_capture(...)       -> enable_piecewise_cuda_graph(...)
  CUDA_GRAPH_CAPTURE_FAILED_MSG        -> PIECEWISE_CUDA_GRAPH_CAPTURE_FAILED_MSG
  _in_cuda_graph_capture (private)     -> _in_piecewise_cuda_graph

is_in_breakable_cuda_graph() and is_in_torch_compile_warmup() unchanged
(both names describe what they actually test).

Callsites that genuinely need umbrella semantics will use explicit
inline `is_in_piecewise_cuda_graph() or is_in_breakable_cuda_graph()`
in subsequent commits — no helper alias.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Adds three new module-level pieces, no behavior change. The legacy
runners still own the actual capture/replay bodies; this commit lands
the base classes that subsequent phases lift those bodies into.

  cuda_graph_runner/base_runner.py     BaseCudaGraphRunner ABC, freeze_gc,
                                       get_batch_sizes_to_capture
  cuda_graph_runner/buffers.py         DecodeInputBuffers, PrefillInputBuffers
                                       (dataclasses + populate_from_forward_batch
                                       + _grouped_foreach_copy_)
  cuda_graph_backend/base.py           BaseCudaGraphBackend ABC: prepare /
                                       can_run / capture_one / replay /
                                       cleanup

Buffer dataclass copies are duplicates for now — legacy.py and
piecewise_cuda_graph_runner.py still hold their originals; subsequent
phases switch their imports across and delete the duplicates.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…backends

Replaces the static-method backend helpers + legacy CudaGraphRunner with
a real Decode runner backed by stateful backends.

  cuda_graph_backend/{full,breakable}.py  → DELETED (static-method shims)
  cuda_graph_backend/full_cudagraph_backend.py    NEW
  cuda_graph_backend/breakable_cudagraph_backend.py NEW
    - Each implements BaseCudaGraphBackend's 5 methods plus capture_session()
      ctx mgr.  Owns _graphs[shape], _outputs[shape], pool, and (Full only)
      memory_saver_adapter.
    - Replay path is uniform from runner POV: backend.replay(shape, fb, **kw)
      returns the captured output; static_fb is unused for these two.

  cuda_graph_runner/legacy.py  → DELETED
  cuda_graph_runner/decode_runner.py → real DecodeCudaGraphRunner(BaseCudaGraphRunner)
    - Lifts the entire CudaGraphRunner body (init, can_run, capture,
      capture_one_batch_size, recapture_if_needed, replay_prepare, replay,
      get_spec_info).
    - Backend dispatched off cuda_graph_mode["decode"]: full | breakable;
      tcpcg falls back to full with a one-shot warning.

  cuda_graph_runner/capture_mode.py    NEW — model_capture_mode + lora-variant globals
  cuda_graph_runner/pool.py            NEW — get/set_global_graph_memory_pool
                                       (used by speculative-draft runners)
  cuda_graph_runner/deepep_adapter.py  NEW — DeepEPCudaGraphRunnerAdapter
  compilation/torch_compile_decoration.py NEW — patch_model + _to_torch +
                                       set_torch_compile_config

Speculative draft runners no longer reuse `CudaGraphRunner.capture(self)` —
each inlines its own ~25-line capture loop using the relocated freeze_gc /
graph_capture / get_tensor_model_parallel_rank helpers.  Their imports
swap from `cuda_graph_runner` package re-exports to direct module paths.

`CudaGraphRunner` symbol is gone; eagle_worker / adaptive_runtime_state /
NPUGraphRunner all updated to `DecodeCudaGraphRunner`.

NPUGraphRunner override of `_create_device_graph` / `_capture_graph` is now
dead code — the new Decode runner uses backend.capture_one() and never
calls those methods.  NPU support needs a follow-up NPUCudaGraphBackend;
out of scope for this PR (CUDA H100 testing only).

Validation:
  Qwen3-8B mgsm_en N=200 (decode CG only, --disable-piecewise-cuda-graph):
  score 0.835, latency 63.58s, throughput 3,455 tok/s
  Within noise of prior commit's 0.835 baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ckend

Replaces the legacy BCG + PCG runners with a single PrefillCudaGraphRunner
that picks between BreakableCudaGraphBackend and TCPiecewiseCudaGraphBackend
off cuda_graph_mode["prefill"].

  cuda_graph_backend/tcpcg.py  → DELETED (static-method shim)
  cuda_graph_backend/tcpcg_cudagraph_backend.py  NEW
    - Stateful TCPiecewiseCudaGraphBackend(BaseCudaGraphBackend).
    - prepare(runner): builds CompilationConfig, multi-platform-op walks
      the language model into compile mode, runs a dummy warmup, calls
      install_torch_compiled, then runs the warmup_compile loop over
      every shape so torch.compile finishes JIT compilation before the
      cuda-graph capture session opens.
    - capture_session(stream): sets enable_piecewise_cuda_graph +
      set_pcg_capture_stream so the piecewise backend captures on the
      right stream.
    - capture_one(shape, fn, dummies): runs forward_fn twice (jit warm
      + cuda-graph capture) — both inside the capture_session.
    - replay(shape, static_fb, **kw): invokes the wrapped
      model_runner.model.forward (NOT language_model.model.forward —
      the former is what builds LogitsProcessorOutput; storing the
      latter in self._compiled_fn was a bug caught by the smoke run).
    - runtime_session(): enable_piecewise_cuda_graph for replay path.

  breakable_cuda_graph_runner.py  → DELETED
  piecewise_cuda_graph_runner.py  → DELETED
  cuda_graph_runner/prefill_runner.py
    - Replaces the prior factory with a real
      PrefillCudaGraphRunner(BaseCudaGraphRunner).
    - Owns PrefillInputBuffers, capture_num_tokens, attention_layers /
      moe_layers / moe_fusions snapshots, and per-bs static_* buffers
      that BCG segments read at replay (allocated regardless of active
      backend; trivial cost).
    - can_run() enforces the BCG-prefill bs<=1 constraint via
      isinstance check; rejects target_verify (tcpcg-prefill captured
      with EXTEND only); per-token-count cap.
    - replay() opens backend.runtime_session(), runs replay_prepare
      (pad/populate/build static_forward_batch), inits attn metadata,
      opens set_forward_context, calls backend.replay(num_tokens,
      static_fb), slices output to raw_num_tokens.
    - _run_warmup_forward(): the per-shape warmup hook that
      TCPiecewiseCudaGraphBackend.prepare calls during install_compile.

  cuda_graph_backend/{full,breakable,tcpcg}_cudagraph_backend.py
    - All three now do their own jit warmup (2x forward_fn) inside
      capture_one rather than expecting the runner to drive warmup
      separately. Decode runner's explicit pre-warmup loop dropped.
    - Each backend stashes self._tp_group during prepare() so the
      barrier between warmup runs works without going through the
      runner.
    - Added BaseCudaGraphBackend.runtime_session() — default no-op for
      Full; opens enable_breakable_cuda_graph for BCG; opens
      enable_piecewise_cuda_graph for tcpcg. Decode/prefill runners
      wrap their replay paths in backend.runtime_session() so model
      code reads the correct is_in_*_cuda_graph flag.
    - has_shape() added to all three backends; tcpcg's always-True
      since torch.compile dispatches by tensor shape internally.

ModelRunner now imports DecodeCudaGraphRunner + PrefillCudaGraphRunner
from cuda_graph_runner package directly. set_torch_compile_config moved
to compilation/torch_compile_decoration.

Validation:
  Qwen3-8B mgsm_en N=200, default cuda_graph_mode={'decode':'full', 'prefill':'tcpcg'}:
  score 0.850, latency 64.82s, throughput 3,380 tok/s
  Above baseline 0.835.

Bug caught + fixed during smoke run: TCPiecewise.prepare initially
stored self._compiled_fn = language_model.model.forward (the inner
torch-compiled module). replay() invokes through this directly,
skipping the outer Qwen3ForCausalLM.forward wrapper that builds
LogitsProcessorOutput — so output came back as raw hidden states and
the runner's isinstance-dispatch hit the PPProxyTensors-only fallback
(AssertionError). Fix: store runner.model_runner.model.forward (the
outer wrapper). Validated against Qwen3-8B PCG path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…l,full) reject

Field/method renames for symmetry between phases:
  ModelRunner.graph_runner               → decode_cuda_graph_runner
  ModelRunner.piecewise_cuda_graph_runner → prefill_cuda_graph_runner
  ModelRunner.init_device_graphs()       → init_decode_cuda_graph()
  ModelRunner.init_piecewise_cuda_graphs() → init_prefill_cuda_graph()

Speculative-worker callers (eagle_worker / eagle_worker_v2 /
multi_layer_eagle_worker_v2 / eagle_info_v2 / adaptive_runtime_state)
all updated to access ``model_runner.decode_cuda_graph_runner`` instead
of the old ``graph_runner`` field.

(prefill, full) reject:
  - _downgrade_unsupported_combinations renamed to
    _reject_unsupported_combinations and now raises NotImplementedError
    at config-resolution time. Previous behavior silently downgraded
    to (prefill, disabled) with a warning. Per refactor goal: explicit
    over implicit; the user gets a clear error pointing to breakable
    or tcpcg.

Shim deletion:
  - python/sglang/srt/model_executor/breakable_cuda_graph/  → DELETED
    (4-file shim re-exporting from cuda_graph_backend_utils/breakable_cuda_graph/)
  - python/sglang/srt/compilation/piecewise_context_manager.py  → DELETED
    (re-exports superseded by direct imports from new homes)
  - test/registered/breakable_cuda_graph/test_breakable_cuda_graph.py
    repointed to the real location.

Validation:
  Qwen3-8B mgsm_en N=200, default tcpcg+full:
  score 0.835, latency 64.24s, throughput 3,418 tok/s
  Within noise of Phase E+F's 0.850.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

  test/registered/breakable_cuda_graph/  → test/registered/cuda_graph/breakable/
  test/registered/piecewise_cuda_graph/  → test/registered/cuda_graph/piecewise/

No content changes; pure file moves. Test imports were already
repointed to the real (non-shim) source locations in Phase G.

Note: .github/CODEOWNERS line 47 still references the deleted file
``piecewise_cuda_graph_runner.py``; left untouched to avoid touching
governance config without explicit ask.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

main moved 228 commits since the cg-refactor fork point. Six conflicts
resolved manually:

  python/sglang/srt/layers/attention/flashinfer_backend.py
    - Combine HEAD's is_in_tc_piecewise rename with main's added
      ``not self.use_paged`` clause on the use_ragged check.

  python/sglang/srt/model_executor/cuda_graph_runner/decode_runner.py
    - Drop bisect/gc imports added on main (lifted into base_runner /
      utils on cg-refactor); keep contextlib (used by the device_timer
      wrap). Combine main's device_timer.wrap with HEAD's backend.replay
      via ``with timer_ctx, self.backend.runtime_session():``.

  python/sglang/srt/model_executor/model_runner.py
    - Combine main's device_timer wrap with HEAD's renamed runner
      attribute (self.prefill_cuda_graph_runner, not the legacy
      self.piecewise_cuda_graph_runner).

  python/sglang/srt/model_executor/piecewise_cuda_graph_runner.py
    - Modify/delete conflict — file was deleted in cg-refactor as part
      of the lift into prefill_runner.py. Stay deleted.

  python/sglang/srt/speculative/eagle_draft_cuda_graph_runner.py
  python/sglang/srt/speculative/eagle_draft_extend_cuda_graph_runner.py
    - Drop main's legacy template methods (_create_graph,
      _capture_init, _capture_graph, _replay, capture,
      capture_one_batch_size). EAGLE drafts now subclass
      DecodeCudaGraphRunner and override capture_one_shape directly.
      Preserve main's eagle_draft device_timer.wrap by adding it
      around the backend.replay() call in the existing replay() body.

Verified: every critical module imports cleanly post-merge
(server_args, model_runner, flashattention_backend, flashinfer_backend,
flashinfer_cutedsl, parallel_state, both custom_all_reduce variants,
decode/prefill runners, factory, all eagle workers).
Backend.TC_PIECEWISE = 'tc_piecewise' round-trips through cuda_graph_mode.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Apply isort + black after the main merge: import reordering in
factory.py and custom_all_reduce_v2.py, blank-line cleanup in
compile_phase.py / capture_mode.py / buffers.py, reformatted assertion
in breakable_cudagraph_backend.py, line wrapping in eagle_info_v2.py
and buffers.py. No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>