Skip to content

model: Minicpmo#3023

Merged
zhyncs merged 2 commits intosgl-project:mainfrom
mickqian:minicpmo
Mar 25, 2025
Merged

model: Minicpmo#3023
zhyncs merged 2 commits intosgl-project:mainfrom
mickqian:minicpmo

Conversation

@mickqian
Copy link
Copy Markdown
Collaborator

@mickqian mickqian commented Jan 21, 2025 •
edited
Loading

Motivation

Support openbmb/MiniCPM-o-2_6

Modifications

Checklist

@mickqian mickqian marked this pull request as ready for review January 21, 2025 03:19
@mickqian mickqian marked this pull request as draft January 23, 2025 12:26
@mickqian mickqian force-pushed the minicpmo branch 4 times, most recently from 1764685 to de11d37 Compare January 29, 2025 07:58
@mickqian mickqian marked this pull request as ready for review January 29, 2025 08:56
mickqian
mickqian commented Jan 29, 2025
View reviewed changes
Comment thread test/srt/test_vision_openai_server.py Outdated
@yizhang2077 yizhang2077 self-assigned this Feb 7, 2025
@mickqian mickqian force-pushed the minicpmo branch 5 times, most recently from bc33318 to 78f5175 Compare February 26, 2025 02:11
@merrymercy merrymercy requested a review from HaiShaw as a code owner March 3, 2025 08:12
@zhaochenyang20 zhaochenyang20 mentioned this pull request Mar 4, 2025
Closed
67 tasks
yizhang2077
yizhang2077 reviewed Mar 4, 2025
View reviewed changes
Comment thread python/sglang/srt/model_executor/model_runner.py Outdated
Comment thread python/sglang/srt/models/minicpmv.py Outdated
Comment thread python/sglang/srt/models/minicpmv.py Outdated
Comment thread python/sglang/srt/models/minicpmo.py Outdated
@yizhang2077
Copy link
Copy Markdown
Collaborator

yizhang2077 commented Mar 4, 2025

I think we should add test .mp3 files to other places instead of adding to project directly.

@mickqian mickqian changed the title Minicpmo model: Minicpmo Mar 23, 2025
@mickqian mickqian force-pushed the minicpmo branch 4 times, most recently from 87f83d4 to d00dc4d Compare March 24, 2025 05:02
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
Comment thread python/sglang/srt/models/minicpmo.py Outdated
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
Comment thread python/sglang/srt/models/minicpmo.py Outdated
Comment thread python/sglang/srt/models/minicpmo.py Outdated
Comment thread python/sglang/srt/models/minicpmo.py Outdated
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
Comment thread test/srt/test_vlm_accuracy.py Outdated
Comment thread test/srt/test_vlm_accuracy.py Outdated
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
Comment thread test/srt/test_vision_openai_server.py Outdated
Copy link
Copy Markdown
Collaborator

@yizhang2077 yizhang2077 Mar 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we directly move audio related test into TestMinicpmoServer?

@yizhang2077
Copy link
Copy Markdown
Collaborator

yizhang2077 commented Mar 24, 2025 •
edited
Loading

LGTM，after ci passes and unit test related issues have been solved, this PR can be merged. cc @zhaochenyang20 and plz paste MMMU benchmark result here~ @mickqian

@yizhang2077
Copy link
Copy Markdown
Collaborator

yizhang2077 commented Mar 24, 2025

FYI, tts model in current Minicpmo is not used, as well as streaming audio embedding

@mickqian
Copy link
Copy Markdown
Collaborator Author

mickqian commented Mar 25, 2025

LGTM，after ci passes and unit test related issues have been solve, this PR can be merged. cc @zhaochenyang20 and plz paste MMMU benchmark result here~   @mickqian

accuracy
sglang 0.481
hf 0.49

@zhaochenyang20
Copy link
Copy Markdown
Collaborator

zhaochenyang20 commented Mar 25, 2025

@yizhang2077 wait for yi's approval!

@zhaochenyang20
Copy link
Copy Markdown
Collaborator

zhaochenyang20 commented Mar 25, 2025

great！let me rerun the CI and merge it asap

@zhyncs zhyncs merged commit 1e86457 into sgl-project:main Mar 25, 2025
@yizhang2077
Copy link
Copy Markdown
Collaborator

yizhang2077 commented Mar 25, 2025

@zhaochenyang20 we may need contributor to support audio benchmark

@zhaochenyang20
Copy link
Copy Markdown
Collaborator

zhaochenyang20 commented Mar 25, 2025

great work1!

merrymercy
merrymercy reviewed Mar 28, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@merrymercy merrymercy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move all model-specific files under python/sglang/srt/managers/multimodal_processors to python/sglang/srt/models/

@mickqian mickqian mentioned this pull request May 21, 2025
Closed
6 tasks
@yanbing-j
Copy link
Copy Markdown
Contributor

yanbing-j commented Nov 12, 2025

Hi @mickqian , since you have added WhisperAttention and WhisperEncoder in the current PR, I am wondering that do you have any plan to support Whisper model in SGLang? Thanks!

@zhaochenyang20
Copy link
Copy Markdown
Collaborator

zhaochenyang20 commented Nov 12, 2025

Hi @mickqian , since you have added WhisperAttention and WhisperEncoder in the current PR, I am wondering that do you have any plan to support Whisper model in SGLang? Thanks!

For new model support, could you add my wechat 18015766633

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@merrymercy merrymercy merrymercy left review comments

@yizhang2077 yizhang2077 yizhang2077 approved these changes

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@zhyncs zhyncs Awaiting requested review from zhyncs

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

@ByronHsu ByronHsu Awaiting requested review from ByronHsu

@HandH1998 HandH1998 Awaiting requested review from HandH1998

@BBuf BBuf Awaiting requested review from BBuf

@HaiShaw HaiShaw Awaiting requested review from HaiShaw

@zhaochenyang20 zhaochenyang20 Awaiting requested review from zhaochenyang20

@xiezhq-hermann xiezhq-hermann Awaiting requested review from xiezhq-hermann xiezhq-hermann is a code owner

Assignees

@yizhang2077 yizhang2077

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@mickqian @yizhang2077 @zhaochenyang20 @WangVertex @yanbing-j @merrymercy @zhyncs