Skip to content

add giga-embedding-instruct model#77

Closed
ekolodin wants to merge 4 commits into
embeddings-benchmark:mainfrom
ekolodin:add-giga-embeddings-instruct
Closed

add giga-embedding-instruct model#77
ekolodin wants to merge 4 commits into
embeddings-benchmark:mainfrom
ekolodin:add-giga-embeddings-instruct

Conversation

@ekolodin

Copy link
Copy Markdown
Contributor

Checklist

  • Run tests locally to make sure nothing is broken using make test.
  • Run the results files checker make pre-push.

Adding a model checklist

Ref #

  • I have added model implementation to mteb/models/ directory. Instruction to add a model can be found here

@Samoed

Samoed commented Dec 20, 2024

Copy link
Copy Markdown
Member

Could you run MiraclReranking with the current MTEB version since its score format has changed? Also, could you add the implementation to the MTEB repository?

@KennethEnevoldsen KennethEnevoldsen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay on this

The result format is good.

Just to make you aware you will have to submit a model meta to the mteb repo for it to appear on the leaderboard.

The model meta file is also unfilled

@Samoed

Samoed commented Feb 5, 2025

Copy link
Copy Markdown
Member

@KennethEnevoldsen I was trying to add this model in embeddings-benchmark/mteb#1741

@KennethEnevoldsen

Copy link
Copy Markdown
Contributor

Ah yes I see - I could have seen that from the reference

@KennethEnevoldsen

Copy link
Copy Markdown
Contributor

Re-reviewed this PR. Seems like the revision is missing. I don't think we can merge it without it. However @Samoed's implementation has been merged so it should be easy to rerun.

Will close this PR for now, but do open a new one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants