[feat] Add triplets/n-tuple support to AnglE#3609
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds experimental triplet and n-tuple support to AnglELoss. Previously, AnglELoss only supported pairs of texts with float similarity scores. Now it can also process triplets (anchor, positive, negative) and n-tuples (anchor, positive, negative_1, ..., negative_n) by internally converting them to pairwise comparisons.
Changes:
- Extended AnglELoss to handle triplet and n-tuple inputs by converting them to pairs internally
- Added a training example script demonstrating the new functionality with the AllNLI dataset
- Updated documentation to reflect the new input formats supported
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| sentence_transformers/losses/AnglELoss.py | Adds compute_loss_from_embeddings method to handle triplet/n-tuple inputs by converting them to pairwise comparisons with appropriate labels |
| examples/sentence_transformer/training/nli/training_nli_angle.py | New training script demonstrating AnglELoss with triplet data from AllNLI dataset |
| examples/sentence_transformer/training/nli/README.md | Documents the new training_nli_angle.py example |
| docs/sentence_transformer/loss_overview.md | Updates loss function table to include AnglELoss for triplet and n-tuple inputs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
feat] Add triplets/n-tuple support to AnglE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello!
Pull Request overview
Details
This is a draft to add triplets/n-tuple support to AnglELoss, at which point the inputs are converted to pairs again, with labels set to 1 for positive pairs and 0 for negative pairs. I copied a simple training script and updated it to use AnglE, resulting in this model:
Which can be fairly compared to e.g. these: