Upstream merge 25 01 13 by gshtras · Pull Request #358 · ROCm/vllm

gshtras · 2025-01-13T23:08:11Z

Upstream merge

reverting memory profiling changes due to it showing excessive memory usage

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

…e with Llava-OneVision (vllm-project#11685) Signed-off-by: Roger Wang <ywang@roblox.com> Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

…-project#11772) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…11776) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…#11648) Signed-off-by: yisheng <yi.sheng@intel.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: Abatom <abzhonghua@gmail.com> Co-authored-by: Zhonghua Deng <abatom@163.com>

Signed-off-by: Roger Wang <ywang@roblox.com>

Signed-off-by: Liangfu Chen <liangfc@amazon.com>

…ject#11768)

Signed-off-by: Russell Bryant <rbryant@redhat.com>

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Signed-off-by: Roger Wang <ywang@roblox.com>

…tion (vllm-project#11800) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Signed-off-by: Roger Wang <ywang@roblox.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…t#11750) Signed-off-by: youkaichao <youkaichao@gmail.com>

Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>

Signed-off-by: Sourashis Roy <sroy@roblox.com>

…eps > 1 (vllm-project#11794)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

vllm-project#11824)

Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>

Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Roger Wang <ywang@roblox.com>

Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>

Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

…roject#11100) Signed-off-by: Akshat Tripathi <akshat@krai.ai> Signed-off-by: Oleg Mosalov <oleg@krai.ai> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Oleg Mosalov <oleg@krai.ai> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Isotr0py <2037008807@qq.com>

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

…m-project#9685) Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

…project#11973) Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

…project#11979)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>

Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>

Signed-off-by: Chenguang Li <757486878@qq.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

…ect#11998) Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

…llm-project#11935) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

…ct#11516) Signed-off-by: Shanshan Shen <467638484@qq.com>

vllm-project#11982) Signed-off-by: elijah <f1renze.142857@gmail.com>

This reverts commit 889e662.

jeejeelee and others added 30 commits January 6, 2025 15:22

[Bugfix][V1] Fix molmo text-only inputs (vllm-project#11676)

32c9eff

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

[Kernel] Move attn_type to Attention.__init__() (vllm-project#11690)

e20c92b

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

[V1] Extend beyond image modality and support mixed-modality inferenc…

91b361a

…e with Llava-OneVision (vllm-project#11685) Signed-off-by: Roger Wang <ywang@roblox.com> Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (vllm…

08fb75c

…-project#11772) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Model] Future-proof Qwen2-Audio multi-modal processor (vllm-project#…

d0169e1

…11776) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[XPU] Make pp group initilized for pipeline-parallelism (vllm-project…

d93d2d7

…#11648) Signed-off-by: yisheng <yi.sheng@intel.com>

[Doc][3/N] Reorganize Serving section (vllm-project#11766)

8ceffbf

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Kernel][LoRA]Punica prefill kernels fusion (vllm-project#11234)

b278557

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: Abatom <abzhonghua@gmail.com> Co-authored-by: Zhonghua Deng <abatom@163.com>

[Bugfix] Update attention interface in Whisper (vllm-project#11784)

0f3f3c8

Signed-off-by: Roger Wang <ywang@roblox.com>

[CI] Fix neuron CI and run offline tests (vllm-project#11779)

898cdf0

Signed-off-by: Liangfu Chen <liangfc@amazon.com>

fix init error for MessageQueue when n_local_reader is zero (vllm-pro…

e512f76

…ject#11768)

[Doc] Create a vulnerability management team (vllm-project#9925)

ce1917f

Signed-off-by: Russell Bryant <rbryant@redhat.com>

[CI][CPU] adding build number to docker image name (vllm-project#11788)

1e4ce29

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

[V1][Doc] Update V1 support for LLaVa-NeXT-Video (vllm-project#11798)

8082ad7

Signed-off-by: Roger Wang <ywang@roblox.com>

[Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calcula…

8f37be3

…tion (vllm-project#11800) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[doc] add doc to explain how to use uv (vllm-project#11773)

869e829

Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

[V1] Support audio language models on V1 (vllm-project#11733)

2de197b

Signed-off-by: Roger Wang <ywang@roblox.com>

[doc] update how pip can install nightly wheels (vllm-project#11806)

d9fa1c0

Signed-off-by: youkaichao <youkaichao@gmail.com>

[Doc] Add note to gte-Qwen2 models (vllm-project#11808)

c0efe92

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[optimization] remove python function call for custom op (vllm-projec…

869579a

…t#11750) Signed-off-by: youkaichao <youkaichao@gmail.com>

[Bugfix] update the prefix for qwen2 (vllm-project#11795)

c994223

Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>

[Doc]Add documentation for using EAGLE in vLLM (vllm-project#11417)

973f5dc

Signed-off-by: Sourashis Roy <sroy@roblox.com>

[Bugfix] Significant performance drop on CPUs with --num-scheduler-st…

a4e2b26

…eps > 1 (vllm-project#11794)

[Doc] Group examples into categories (vllm-project#11782)

5950f55

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

[Bugfix] Fix image input for Pixtral-HF (vllm-project#11741)

91445c7

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Misc] sort torch profiler table by kernel timing (vllm-project#11813)

4d29e91

Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (

dc71af0

vllm-project#11824)

Fixed docker build for ppc64le (vllm-project#11518)

b640b19

Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>

[OpenVINO] Fixed Docker.openvino build (vllm-project#11732)

f4923cb

Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

[Bugfix] Add checks for LoRA and CPU offload (vllm-project#11810)

f645eb6

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

ywang96 and others added 27 commits January 12, 2025 06:36

[V1] Avoid sending text prompt to core engine (vllm-project#11963)

b25cfab

Signed-off-by: Roger Wang <ywang@roblox.com>

[CI/Build] Add markdown linter (vllm-project#11857)

43f3d9e

Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>

[Model] Initialize support for Deepseek-VL2 models (vllm-project#11578)

f967e51

Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

[Hardware][TPU] workaround fix for MoE on TPU (vllm-project#11764)

263a870

[V1][Core][1/n] Logging and Metrics (vllm-project#11962)

9597a09

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

[Model] Support GGUF models newly added in transformers 4.46.0 (vll…

d14e98d

…m-project#9685) Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

[V1] [2/n] Logging and Metrics - OutputProcessor Abstraction (vllm-…

619ae26

…project#11973) Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

[MISC] fix typo in kv transfer send recv test (vllm-project#11983)

f7b3ba8

[Bug] Fix usage of .transpose() and .view() consecutively. (vllm-…

9dd02d8

…project#11979)

[CI][Spec Decode] fix: broken test for EAGLE model (vllm-project#11972)

80ea3af

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>

[Misc] Fix Deepseek V2 fp8 kv-scale remapping (vllm-project#11947)

cf6bbcb

Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>

[Misc]Minor Changes about Worker (vllm-project#11555)

c3f05b0

Signed-off-by: Chenguang Li <757486878@qq.com>

[platform] add ray_device_key (vllm-project#11948)

89ce62a

Signed-off-by: youkaichao <youkaichao@gmail.com>

Fix Max Token ID for Qwen-VL-Chat (vllm-project#11980)

5340a30

Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>

[Kernel] unified_attention for Attention.forward (vllm-project#11967)

0f8cafe

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

[Doc][V1] Update model implementation guide for V1 support (vllm-proj…

cd82499

…ect#11998) Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

[Doc] Organise installation documentation into categories and tabs (v…

e8c23ff

…llm-project#11935) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

[platform] add device_control env var (vllm-project#12009)

458e63a

Signed-off-by: youkaichao <youkaichao@gmail.com>

[Platform] Move get_punica_wrapper() function to Platform (vllm-proje…

a7d5968

…ct#11516) Signed-off-by: Shanshan Shen <467638484@qq.com>

bugfix: Fix signature mismatch in benchmark's get_tokenizer function (

c6db213

vllm-project#11982) Signed-off-by: elijah <f1renze.142857@gmail.com>

Merge remote-tracking branch 'upstream/main'

ce53f46

Using list

5a51290

Revert "[misc] improve memory profiling (vllm-project#11809)"

079750e

This reverts commit 889e662.

Trying to make scales work with compileable attention

043c93d

Docs lint

16f8680

Merge remote-tracking branch 'origin/main' into upstream_merge_25_01_13

eb4abfd

shajrawi approved these changes Jan 14, 2025

View reviewed changes

gshtras merged commit 5976f48 into main Jan 14, 2025
12 checks passed

gshtras deleted the upstream_merge_25_01_13 branch January 14, 2025 20:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upstream merge 25 01 13#358

Upstream merge 25 01 13#358
gshtras merged 116 commits intomainfrom
upstream_merge_25_01_13

gshtras commented Jan 13, 2025 •

edited by github-actions bot

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

gshtras commented Jan 13, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

gshtras commented Jan 13, 2025 •

edited by github-actions bot

Loading