Skip to content

Add VultronRetrieverPrime-Qwen3.5-8B ModelMeta#4833

Merged
Samoed merged 1 commit into
embeddings-benchmark:mainfrom
athrael-soju:add-vultron-prime-qwen35-8b
Jun 18, 2026
Merged

Add VultronRetrieverPrime-Qwen3.5-8B ModelMeta#4833
Samoed merged 1 commit into
embeddings-benchmark:mainfrom
athrael-soju:add-vultron-prime-qwen35-8b

Conversation

@athrael-soju

@athrael-soju athrael-soju commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Adds a ModelMeta for athrael-soju/VultronRetrieverPrime-Qwen3.5-8B, a late-interaction (ColBERT-style MaxSim) visual document retriever.

  • Architecture: ColQwen3.5 (Qwen3.5-VL backbone), multi-vector, embed_dim 320, image+text
  • Size: 8.4B params, Apache-2.0, 6 languages (eng/fra/deu/spa/ita/por)
  • Loader: reuses the existing ColQwen3_5Wrapper (no wrapper change); extra_requirements_groups=["colpali_engine"]
  • Revision: e8f3104b743a04b0d5f715b67117d687ae99ce51 (public, open weights on HF)

Official ViDoRe scores (ndcg@5 / ndcg@5 / ndcg@10): V1 0.9208 / V2 0.6818 / V3 0.6472.

The matching results PR to embeddings-benchmark/results will follow once this is merged (the results name needs the registered ModelMeta to resolve).

@athrael-soju athrael-soju mentioned this pull request Jun 18, 2026
Merged
@athrael-soju athrael-soju force-pushed the add-vultron-prime-qwen35-8b branch from 9035b1e to 604ae0d Compare June 18, 2026 18:32
@athrael-soju @claude
Add VultronRetrieverPrime-Qwen3.5-8B ModelMeta
9b56d5e 
Late-interaction (ColBERT MaxSim) visual document retriever: ColQwen3.5, dim 320,
8.4B params, Apache-2.0, 6 languages. Reuses the existing ColQwen3_5Wrapper.
Official ViDoRe scores V1 0.9208 / V2 0.6818 / V3 0.6472 (results PR to follow).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@athrael-soju athrael-soju force-pushed the add-vultron-prime-qwen35-8b branch from 604ae0d to 9b56d5e Compare June 18, 2026 19:02
@athrael-soju

athrael-soju commented Jun 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

CI is green except the leaderboard check and some of the OS tests, which seem unrelated to this PR:

  • test: the earlier failure was a transient HF-CDN SSLError downloading
    sentence-transformers/all-MiniLM-L6-v2 (an unrelated model). It cleared on
    re-run; test is now green and the ModelMeta validation passes.
  • leaderboard: fails with InvalidVersion: '2.12.16-2.15.4'
    (test_leaderboard_app_does_not_crash). This is the known multiversion-results
    bug tracked in Results with multiversion coudn't be loaded #4830, and it fails identically on main (14cf6d61), not
    introduced here. ci: revert leaderboard refresh trusted publishing #4831 was merged through the same red earlier today.

This PR adds only the ModelMeta for athrael-soju/VultronRetrieverPrime-Qwen3.5-8B,
reusing the existing ColQwen3_5Wrapper. The matching results are in
embeddings-benchmark/results#575, gated on this PR.

@Samoed I'd appreciate a quick review.

Samoed
Samoed approved these changes Jun 18, 2026
View reviewed changes

@Samoed Samoed left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Congratulations with release!

@Samoed Samoed enabled auto-merge (squash) June 18, 2026 19:39
@Samoed Samoed merged commit 99a830c into embeddings-benchmark:main Jun 18, 2026
15 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Samoed Samoed Samoed approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@athrael-soju @Samoed