Skip to content

fix: remove NaN handling#1444

Closed
KennethEnevoldsen wants to merge 1 commit into
v2.0.0from
fix--remove-NaN-handling
Closed

fix: remove NaN handling#1444
KennethEnevoldsen wants to merge 1 commit into
v2.0.0from
fix--remove-NaN-handling

Conversation

@KennethEnevoldsen

@KennethEnevoldsen KennethEnevoldsen commented Nov 13, 2024 •
edited by isaac-chung
Loading

Copy link
Copy Markdown
Contributor

As discussed in #1398 NaN should never occur. Removing handling of them.

Fixes #1389

Checklist

  • Run tests locally to make sure nothing is broken using make test.
  • Run the formatter to format the code using make lint.

@KennethEnevoldsen
fix: remove NaN handling
a92963a 
As discussed in #1398 NaN should never occur. Removing handling of them.

Fixes #1389
@KennethEnevoldsen KennethEnevoldsen requested review from isaac-chung and orionw and removed request for isaac-chung November 13, 2024 12:57
@KennethEnevoldsen KennethEnevoldsen mentioned this pull request Nov 13, 2024
Merged
2 tasks
@KennethEnevoldsen

KennethEnevoldsen commented Nov 14, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@orionw not quite sure about the merge conflict here - seems like there is no conflict, but I might be missing something ? (code might have been deleted and moved elsewhere)

@Samoed

Samoed commented Nov 14, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Code was moved to

mteb/mteb/evaluation/evaluators/model_classes.py

Lines 341 to 351 in 9c58518

with torch.inference_mode():
cos_scores = self.score_functions[score_function](
query_embeddings, sub_corpus_embeddings
)
is_nan = torch.isnan(cos_scores)
if is_nan.sum() > 0:
raise ValueError(
f"NaN values detected in the similarity scores: {is_nan.sum()}"
)

@KennethEnevoldsen

KennethEnevoldsen commented Nov 14, 2024

Copy link
Copy Markdown
Contributor Author

Perfect - redid the fix directly on v2.0.0

@Samoed Samoed deleted the fix--remove-NaN-handling branch October 19, 2025 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@isaac-chung isaac-chung isaac-chung approved these changes
@orionw orionw Awaiting requested review from orionw

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@KennethEnevoldsen @Samoed @isaac-chung