Skip to content

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task#24872

Merged
DarkLight1337 merged 17 commits intovllm-project:mainfrom
noooop:fix_bert_seq_cls
Sep 18, 2025
Merged

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task#24872
DarkLight1337 merged 17 commits intovllm-project:mainfrom
noooop:fix_bert_seq_cls

Conversation

@noooop
Copy link
Copy Markdown
Collaborator

@noooop noooop commented Sep 15, 2025 •
edited by github-actions Bot
Loading

Purpose

Support BertForTokenClassification / Named Entity Recognition (NER) task

fix bert + Flex Attention + torch.comple by @Isotr0py
(see: #24872 (comment)

Fix #24752
Fix #25060

Test Plan

tests/models/language/pooling/test_token_classification.py

Test Result

pass

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify mergify Bot added documentation Improvements or additions to documentation new-model Requests to new models labels Sep 15, 2025
@noooop
+ ner
d0078ec 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
+ ner online
ed3dba4 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Sep 15, 2025 •
edited
Loading

@mlinmg

try: examples/online_serving/ner.py

But it's definitely not very user-friendly

Do you ok with that. or We still need to further optimize the API.

@mlinmg
Copy link
Copy Markdown
Contributor

mlinmg commented Sep 15, 2025

That's great! The implementation really depends on the popularity of the token classification API. For now, a basic implementation should be more than enough imo. If more people start using it, it might be worthwhile to create a complete end-to-end flow.

@mergify mergify Bot added the frontend label Sep 16, 2025
@noooop
Polishing
c6d4cb7 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
Polishing
5a5ef91 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
dd1d36f 
Signed-off-by: wang.yuqi <noooop@126.com>
noooop
noooop commented Sep 16, 2025
View reviewed changes
Comment thread tests/models/language/pooling/test_token_classification.py Outdated
@noooop
fix
9775e5d 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
cda6dea 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
4e79662 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
0a61127 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop noooop marked this pull request as ready for review September 16, 2025 06:41
@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Sep 16, 2025

@DarkLight1337

Ready for review

DarkLight1337
DarkLight1337 reviewed Sep 16, 2025
View reviewed changes
Comment thread examples/offline_inference/pooling/README.md Outdated
DarkLight1337
DarkLight1337 reviewed Sep 16, 2025
View reviewed changes
Comment thread examples/online_serving/pooling/README.md Outdated
@noooop
fix
690aca0 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop noooop mentioned this pull request Sep 16, 2025
Merged
5 tasks
@Isotr0py
fix flex attn compile
3d86837 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@mergify mergify Bot added the v1 label Sep 16, 2025
@DarkLight1337 DarkLight1337 mentioned this pull request Sep 17, 2025
Closed
1 task
@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Sep 18, 2025

@DarkLight1337
Could you please review this pr.

DarkLight1337
DarkLight1337 approved these changes Sep 18, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if the test can pass

@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 18, 2025
@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Sep 18, 2025 •
edited
Loading

@DarkLight1337

You didn't using auto-merge..

@DarkLight1337 DarkLight1337 merged commit 5f696c3 into vllm-project:main Sep 18, 2025
66 checks passed
@noooop noooop deleted the fix_bert_seq_cls branch September 18, 2025 15:26
debroy-rh pushed a commit to debroy-rh/vllm that referenced this pull request Sep 19, 2025
@noooop @Isotr0py @debroy-rh
[New Model] Support BertForTokenClassification / Named Entity Recogni…
8ce7e1f 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@noooop noooop mentioned this pull request Sep 22, 2025
Merged
5 tasks
ABC12345anouys pushed a commit to ABC12345anouys/vllm that referenced this pull request Sep 25, 2025
@noooop @Isotr0py @ABC12345anouys
[New Model] Support BertForTokenClassification / Named Entity Recogni…
364d46b 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
@noooop @Isotr0py @charlifu
[New Model] Support BertForTokenClassification / Named Entity Recogni…
aebda1b 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: charlifu <charlifu@amd.com>
@DarkLight1337 DarkLight1337 mentioned this pull request Sep 29, 2025
Closed
1 task
@antrec antrec mentioned this pull request Oct 7, 2025
Merged
5 tasks
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
@noooop @Isotr0py @choprahetarth
[New Model] Support BertForTokenClassification / Named Entity Recogni…
323a34b 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@noooop @Isotr0py @lywa1998
[New Model] Support BertForTokenClassification / Named Entity Recogni…
37461a4 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py left review comments

@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes

@hmellor hmellor Awaiting requested review from hmellor

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

@aarnphm aarnphm Awaiting requested review from aarnphm

@chaunceyjiang chaunceyjiang Awaiting requested review from chaunceyjiang

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat

@njhill njhill Awaiting requested review from njhill

@comaniac comaniac Awaiting requested review from comaniac

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation frontend new-model Requests to new models ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: Does not run embedding model sergeyzh/rubert-tiny-turbo [Bug]: Some Bert models no longer compatible for BertForSequenceClassification

4 participants

@noooop @mlinmg @Isotr0py @DarkLight1337