Skip to content

Extend ColBERT support to non-standard BERT backbones#34170

Merged
noooop merged 7 commits intovllm-project:mainfrom
ieBoytsov:extend-colbert-support
Feb 13, 2026
Merged

Extend ColBERT support to non-standard BERT backbones#34170
noooop merged 7 commits intovllm-project:mainfrom
ieBoytsov:extend-colbert-support

Conversation

@ieBoytsov
Copy link
Copy Markdown
Contributor

@ieBoytsov ieBoytsov commented Feb 9, 2026

Purpose

Follow-up to the initial ColBERT support (#33686). Extends vLLM's ColBERT late-interaction support beyond standard BERT to multiple encoder backbones, enabling models like lightonai/GTE-ModernColBERT-v1 (ModernBERT) and jinaai/jina-colbert-v2 (Jina XLM-RoBERTa).

What changed:

  • ColBERTMixin — new mixin that extracts all shared ColBERT logic (projection layer init, weight loading, pooler building, colbert_dim auto-detection from config or weight shape). Adding a new backbone is now ~40 lines of code.

  • ColBERTModernBertModel — ColBERT with ModernBERT backbone

  • ColBERTJinaRobertaModel — ColBERT with Jina XLM-RoBERTa backbone (uses JinaRobertaModel)

  • Refactored the existing HF_ColBERT class to use ColBERTMixin internally, keeping backward compatibility

  • Minor config plumbing so Jina XLM-RoBERTa and pylate-trained models (e.g. lightonai/GTE-ModernColBERT-v1) are recognized correctly

  • Tests are now parametrized across all three backbones (BERT, ModernBERT, Jina) with per-backbone HF comparison tests

  • Updated docs with architecture table and usage examples for each backbone

Architecture Backbone Example Model
HF_ColBERT BERT answerdotai/answerai-colbert-small-v1
ColBERTModernBertModel ModernBERT lightonai/GTE-ModernColBERT-v1
ColBERTJinaRobertaModel Jina XLM-RoBERTa jinaai/jina-colbert-v2

Test Plan

# Unit tests (parametrized across all 3 backbones)
python -m pytest tests/models/language/pooling/test_colbert.py -x -v

# Online endpoint tests (rerank, score, token_embed, embed_not_supported × 3 models)
python -m pytest tests/entrypoints/pooling/score/test_online_colbert.py -x -v


## Test Result

All tests passed

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Feb 9, 2026

Documentation preview: https://vllm--34170.org.readthedocs.build/en/34170/

@mergify mergify bot added documentation Improvements or additions to documentation new-model Requests to new models labels Feb 9, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a significant improvement, extending ColBERT support to multiple backbones like ModernBERT and Jina XLM-RoBERTa. The introduction of ColBERTMixin is a great design choice for code reuse and makes adding new backbones much cleaner. The tests have also been nicely refactored to be parametrized across all supported backbones, which improves test coverage and maintainability. I've found one issue related to loading weights for ColBERT models that have a bias in their projection layer, which could affect models like colbert-ir/colbertv2.0.

@ieBoytsov
Copy link
Copy Markdown
Contributor Author

Hi guys @noooop @DarkLight1337 a follow up PR from me to extend ColBERT support beyond standard BERT backbones. Check this out and let me know what you think. Thanks in advance!

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
@ieBoytsov ieBoytsov force-pushed the extend-colbert-support branch from 66f8e79 to 8dc9ce8 Compare February 12, 2026 14:09
@noooop noooop enabled auto-merge (squash) February 12, 2026 14:32
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Feb 12, 2026
Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
auto-merge was automatically disabled February 12, 2026 16:17

Head branch was pushed to by a user without write access

@noooop noooop enabled auto-merge (squash) February 12, 2026 16:22
@noooop noooop merged commit 071d863 into vllm-project:main Feb 13, 2026
60 checks passed
wzhao18 pushed a commit to wzhao18/vllm that referenced this pull request Feb 18, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
eldarkurtic pushed a commit to eldarkurtic/vllm that referenced this pull request Feb 19, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
Signed-off-by: Eldar Kurtic <research@neuralmagic.com>
ZJY0516 pushed a commit to ZJY0516/vllm that referenced this pull request Feb 23, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
llsj14 pushed a commit to llsj14/vllm that referenced this pull request Mar 1, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
tunglinwood pushed a commit to tunglinwood/vllm that referenced this pull request Mar 4, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
EricccYang pushed a commit to EricccYang/vllm that referenced this pull request Apr 1, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
Signed-off-by: EricccYang <yangyang4991@gmail.com>
liuchenbing2026 pushed a commit to liuchenbing2026/vllm that referenced this pull request Apr 4, 2026
…4170)

Signed-off-by: Ilya Boytsov <ilya.boytsov@aleph-alpha.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation new-model Requests to new models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants