Model description

Some BERT-like models:

• NomicBert
• GTE
• Snowflake GTE
• Jinaai embeddings

Are well used custom models which implement BERT with RoPE.

This is a tracking issue to add support for them in Transformers.

One motivation for this is so that the Transformers modeling backend for vLLM can run these models, allowing us to completely remove all embedding model implementations from vLLM.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

No response