🚀 The feature, motivation and pitch

We are going to migrate from model runner v1 to model runner v2 gradually, here is the roadmap:

• Start with dense model, namely "Qwen/Qwen3-0.6B" and "facebook/opt-125m" as they covered most of the CI tests
• Then, with moe model, like "deepseek-ai/DeepSeek-V2-lite"
• Finally, test with popular model, like "deepseek-ai/DeepSeek-V4-Pro"

Tasks:

• [Model Runner v2] Oracle for model runner v2 - qwen3 dense model by default [1/N] #39337
  • [Model Runner V2] Fix flex attention kv blocks calculation issue #39353
  • [Model Runner V2] Multiple prompt logprobs support #39937
  • [Model Runner V2] Add logprob_token_ids support #40559
  • [Model Runner v2] Fix block table IMA issue #40648
  • [Model Runner v2] Fix v2 compile counter num_gpu_runner_capture_triggers and num_cudagraph_captured #41285
  • [Model Runner V2] Support stock torch compile for v2 #41667 We temporally block this as there would be a big refactor for torch compile stuff recently
  • [Model Runner V2] Bug fix: logprob dtype int64/int32 issue #41761
  • [ModelRunner V2] Share identical MTP weights #42538 @njhill
• [MRV2] Also enable MRV2 for Llama and Mistral dense models  #43458 @njhill
  • [Model Runner v2] Support reload weights (sleep mode) #42673
  • [Model Runner V2] Fix kv_connector pre_forward order #42676
  • [Model Runner V2] Fix prompt logprobs calculation Sizes of tensors must match error #42778
  • [Model Runner v2] Support update_config #42783 @mgoin
  • [ModelRunnerV2] Support prompt embeds #42963 @gcanlin
  • [MRV2][BugFix] Fix default-stream CG capture in P/W LoRA case #43160 @njhill
  • [Model Runner V2] Fix lora Triton Error [CUDA]: device-side assert triggered #43139
  • [Model Runner v2] Force v1 runner for tests #43233
  • [MRV2][BugFix] Fix KV connector handling in spec decode case #43719 @njhill
• [Model Runner v2] Migration from v1 to v2, with Qwen and DSv2 MOE models [3/N] #42667
  • [Model Runner V2] Feature: Support ElasticEPScalingExecutor for MRv2 #43915
• [Model Runner V2] Enable all dense models for mrv2 [4/N] #44443
  • [Model Runner V2] Fix mrv2 mm lora issue #44450
  • [Model Runner V2] Fix v2 AttributeError: 'CohereASRDecoder' object has no attribute 'embed_input_ids' #44568
  • [Model Runner V2] Fix openai.InternalServerError: Error code: 500 - 'list index out of range' #45467
• [Model Runner V2] Migration to support quantized model by default [5/N] #44446
• [Model Runner V2] Enable GraniteMOE for MRv2 by default #45461
• with popular model
• Switch to model runner v2 by default

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.