Skip to content

Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 (and nvidia/C-RADIOv2-H)#12277

Merged
Fridge003 merged 13 commits intosgl-project:mainfrom
netanel-haber:support-nemotronh-nano-vl-v2
Nov 26, 2025
Merged

Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 (and nvidia/C-RADIOv2-H)#12277
Fridge003 merged 13 commits intosgl-project:mainfrom
netanel-haber:support-nemotronh-nano-vl-v2

Conversation

@netanel-haber
Copy link
Copy Markdown
Contributor

@netanel-haber netanel-haber commented Oct 28, 2025 •
edited
Loading

Motivation

Support Multimodal nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16.

NVIDIA Nemotron Nano v2 12B VL model enables multi-image reasoning and video understanding, along with strong document intelligence, visual Q&A and summarization capabilities.

Support its vision encoder: nvidia/CRadioV2-H.

Modifications

  • Added python/sglang/srt/models/nano_nemotron_vl.py
  • Added python/sglang/srt/configs/radio.py

Accuracy Tests

Reference was VLLM, with EVS turned off, temperature 0.
Both VideoMME and DocVQA run via VLMEvalKit.

Checklist

@netanel-haber netanel-haber force-pushed the support-nemotronh-nano-vl-v2 branch 2 times, most recently from fe75b56 to 998988a Compare November 2, 2025 14:31
@github-actions github-actions Bot added performance Multi-modal multi-modal language model deepseek labels Nov 6, 2025
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Nov 16, 2025
@netanel-haber netanel-haber marked this pull request as ready for review November 16, 2025 17:21
@netanel-haber netanel-haber changed the title Support nemotronh nano vl v2 Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 Nov 16, 2025
@netanel-haber netanel-haber changed the title Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 (and nvidia/C-RADIOv2-H) Nov 16, 2025
@yhyang201
Copy link
Copy Markdown
Collaborator

yhyang201 commented Nov 18, 2025

Do you think it would be possible to open a separate PR for the changes made to the following files?

  • python/sglang/test/gsm8k_mixin.py

  • python/sglang/test/mmmu_vlm_mixin.py

  • test/srt/models/test_vlm_models.py

This would help us streamline the review and move forward more quickly. Thanks!

@yhyang201 yhyang201 self-assigned this Nov 18, 2025
@netanel-haber
Copy link
Copy Markdown
Contributor Author

netanel-haber commented Nov 18, 2025 •
edited
Loading

Do you think it would be possible to open a separate PR for the changes made to the following files?

Sure, I'll do so promptly.

@netanel-haber netanel-haber mentioned this pull request Nov 18, 2025
Merged
@netanel-haber netanel-haber mentioned this pull request Nov 18, 2025
Merged
2 tasks
@netanel-haber
Copy link
Copy Markdown
Contributor Author

netanel-haber commented Nov 18, 2025

#13506 @yhyang201

@netanel-haber netanel-haber force-pushed the support-nemotronh-nano-vl-v2 branch from a0c18eb to f973053 Compare November 18, 2025 10:13
@yhyang201
Copy link
Copy Markdown
Collaborator

yhyang201 commented Nov 24, 2025

Could you rebase and resolve the conflicts? Then we can run the CI tests — I believe we’ll be able to merge it soon.

@netanel-haber
Copy link
Copy Markdown
Contributor Author

netanel-haber commented Nov 24, 2025

Could you rebase and resolve the conflicts? Then we can run the CI tests — I believe we’ll be able to merge it soon.

Done

@yhyang201
Copy link
Copy Markdown
Collaborator

yhyang201 commented Nov 24, 2025

Can you check the cause of the CI error?

config = config_class.from_pretrained(model, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 640, in from_pretrained
    return cls.from_dict(config_dict, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 808, in from_dict
    config = cls(**config_dict)
  File "/public_sglang_ci/runner-l3a-gpu-23/_work/sglang/sglang/python/sglang/srt/configs/nano_nemotron_vl.py", line 56, in __init__
    vision_auto_config = get_class_from_dynamic_module(
  File "/usr/local/lib/python3.10/dist-packages/transformers/dynamic_module_utils.py", line 604, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/dynamic_module_utils.py", line 467, in get_cached_module_file
    get_cached_module_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/dynamic_module_utils.py", line 427, in get_cached_module_file
    modules_needed = check_imports(resolved_module_file)
  File "/usr/local/lib/python3.10/dist-packages/transformers/dynamic_module_utils.py", line 260, in check_imports
    raise ImportError(
ImportError: This modeling file requires the following packages that were not found in your environment: open_clip. Run `pip install open_clip`

@netanel-haber
Copy link
Copy Markdown
Contributor Author

netanel-haber commented Nov 24, 2025

Can you check the cause of the CI error?
...
ImportError: This modeling file requires the following packages that were not found in your environment: open_clip. Run pip install open_clip

The commit I just pushed should entirely prevent that error from occuring.

@Fridge003 Fridge003 merged commit 082b54c into sgl-project:main Nov 26, 2025
155 of 180 checks passed
harvenstar pushed a commit to harvenstar/sglang that referenced this pull request Dec 4, 2025
@netanel-haber
Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 (and nvidia/C-RADI…
0e954a0 
…Ov2-H) (sgl-project#12277)
@netanel-haber netanel-haber deleted the support-nemotronh-nano-vl-v2 branch May 5, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yhyang201 yhyang201 yhyang201 approved these changes

@mickqian mickqian Awaiting requested review from mickqian mickqian is a code owner

@JustinTong0323 JustinTong0323 Awaiting requested review from JustinTong0323 JustinTong0323 is a code owner

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner

@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

Assignees

@yhyang201 yhyang201

Labels

deepseek documentation Improvements or additions to documentation Multi-modal multi-modal language model performance run-ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@netanel-haber @yhyang201 @Kangyan-Zhou @Fridge003