[Model] Let more models to support the score template. by noooop · Pull Request #31335 · vllm-project/vllm

noooop · 2025-12-25T07:52:48Z

TL;DR

examples/pooling/score/qwen3_reranker_offline.py
examples/pooling/score/qwen3_reranker_online.py
- Qwen/Qwen3-Reranker-0.6B: qwen3_reranker.jinja
- Qwen/Qwen3-Reranker-4B: qwen3_reranker.jinja
- Qwen/Qwen3-Reranker-BB: qwen3_reranker.jinja
examples/pooling/score/using_template_offline.py
examples/pooling/score/using_template_online.py
- BAAI/bge-reranker-v2-gemma: bge-reranker-v2-gemma.jinja
- tomaarsen/Qwen3-Reranker-0.6B-seq-cls: qwen3_reranker.jinja
- tomaarsen/Qwen3-Reranker-4B-seq-cls: qwen3_reranker.jinja
- tomaarsen/Qwen3-Reranker-8B-seq-cls: qwen3_reranker.jinja
- mixedbread-ai/mxbai-rerank-base-v2: mxbai_rerank_v2.jinja
- mixedbread-ai/mxbai-rerank-large-v2: mxbai_rerank_v2.jinja
- nvidia/llama-nemotron-rerank-1b-v2: nemotron-rerank.jinja
- Qwen/Qwen3-Reranker-0.6B: qwen3_reranker.jinja
- Qwen/Qwen3-Reranker-4B: qwen3_reranker.jinja
- Qwen/Qwen3-Reranker-BB: qwen3_reranker.jinja

Purpose

Following #30550
Let more models to support the score template.

Add more models to Testing and Examples

bge_reranker_v2_gemma √
mxbai_rerank √
qwen3_reranker √
nemotron_rerank √

Test Plan

tests/models/language/pooling_mteb_test/

Test Result

pass

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

mergify · 2025-12-25T07:53:26Z

Documentation preview: https://vllm--31335.org.readthedocs.build/en/31335/

gemini-code-assist

Code Review

This pull request enhances vLLM's support for reranker models by introducing score templates for several new models. The changes include adding new, well-documented example scripts for both offline and online scoring, creating specific Jinja templates for different models, and refactoring existing examples to be more flexible and user-friendly. The overall quality of the changes is high, with clear documentation and good coding practices. I have reviewed the changes for high and critical severity issues and found none. The PR is a valuable addition that improves the usability of vLLM for reranking tasks.

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

tests/models/language/pooling_mteb_test/mteb_score_utils.py

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

mergify · 2025-12-30T06:39:49Z

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

docs/models/supported_models.md

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop · 2025-12-31T07:11:56Z

Thanks for the review and Happy New Year.

mergify · 2025-12-31T07:43:24Z

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

noooop · 2025-12-31T08:02:28Z

Perhaps we should upgrade sentence-transformers in another PR

Let's do this next year.

mergify · 2026-01-05T06:51:38Z

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

…31335) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

…31335) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>

…31335) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

mergify bot added documentation Improvements or additions to documentation qwen Related to Qwen models labels Dec 25, 2025

gemini-code-assist bot reviewed Dec 25, 2025

View reviewed changes

improve example

9cf0a7c

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop force-pushed the more_score_template branch from 4b3cd5d to 9cf0a7c Compare December 25, 2025 10:35

noooop added 7 commits December 25, 2025 18:36

Merge branch 'main' into more_score_template

cd6daba

refine examples

bdb50ee

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Merge branch 'main' into more_score_template

7256dd9

+ fix test_cross_encoder.py

8616bde

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Merge branch 'main' into more_score_template

c4e85a7

+ refine

6d036cd

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

+ fix tests

9540469

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop added the ready ONLY add when PR is ready to merge/full CI is needed label Dec 29, 2025

Merge branch 'main' into more_score_template

75199fc

noooop marked this pull request as ready for review December 29, 2025 09:09

chatgpt-codex-connector bot reviewed Dec 29, 2025

View reviewed changes

tests/models/language/pooling_mteb_test/mteb_score_utils.py Outdated Show resolved Hide resolved

noooop added 3 commits December 30, 2025 10:59

refine

e2dc26b

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Merge branch 'main' into more_score_template

3d2dab3

sentence-transformers==5.2.0

f3e85ab

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

mergify bot added the ci/build label Dec 30, 2025

vllm-project deleted a comment from mergify bot Dec 30, 2025

noooop removed the ready ONLY add when PR is ready to merge/full CI is needed label Dec 30, 2025

noooop added 2 commits December 30, 2025 14:09

refine

0aaaaa6

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

refine

5eb6a87

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop requested a review from DarkLight1337 December 30, 2025 06:46

ruff

e166849

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

DarkLight1337 reviewed Dec 31, 2025

View reviewed changes

docs/models/supported_models.md Outdated Show resolved Hide resolved

fix

147923e

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop added the rl Related to RL workflows label Dec 31, 2025

noooop added ready ONLY add when PR is ready to merge/full CI is needed and removed rl Related to RL workflows labels Dec 31, 2025

Merge branch 'main' into more_score_template

9c02ec0

noooop removed the ready ONLY add when PR is ready to merge/full CI is needed label Dec 31, 2025

Merge branch 'main' into more_score_template

7a0adeb

fix

a3db23b

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop force-pushed the more_score_template branch from de308fb to a3db23b Compare January 5, 2026 07:49

noooop enabled auto-merge (squash) January 5, 2026 09:50

noooop added the ready ONLY add when PR is ready to merge/full CI is needed label Jan 5, 2026

noooop merged commit 911d38e into vllm-project:main Jan 5, 2026
59 checks passed

noooop deleted the more_score_template branch January 5, 2026 11:57

noooop mentioned this pull request Jan 7, 2026

[Models] Allow converting Qwen3-VL into Reranker model #31890

Merged

5 tasks

noooop mentioned this pull request Feb 3, 2026

feat: add max tokens per doc in rerank request #33315

Open

hmellor mentioned this pull request Mar 4, 2026

[Frontend][Model] Qwen3Rerank API Server backward compatibility #20239

Closed

4 tasks

This was referenced Mar 31, 2026

[RFC]: Add max_tokens_per_doc support for rerank and scoring endpoints #38651

Open

feat: add max_tokens_per_doc in rerank request (rebase of #33315) #38827

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Model] Let more models to support the score template.#31335

[Model] Let more models to support the score template.#31335
noooop merged 28 commits intovllm-project:mainfrom
noooop:more_score_template

noooop commented Dec 25, 2025 •

edited by github-actions bot

Loading

Uh oh!

mergify bot commented Dec 25, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

mergify bot commented Dec 30, 2025

Uh oh!

Uh oh!

noooop commented Dec 31, 2025

Uh oh!

mergify bot commented Dec 31, 2025

Uh oh!

noooop commented Dec 31, 2025

Uh oh!

mergify bot commented Jan 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

noooop commented Dec 25, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

Purpose

Test Plan

Test Result

Uh oh!

mergify bot commented Dec 25, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mergify bot commented Dec 30, 2025

Uh oh!

Uh oh!

noooop commented Dec 31, 2025

Uh oh!

mergify bot commented Dec 31, 2025

Uh oh!

noooop commented Dec 31, 2025

Uh oh!

mergify bot commented Jan 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

noooop commented Dec 25, 2025 •

edited by github-actions bot

Loading