Skip to content

[Liger] Integrate Liger CPO & SimPO#2506

Closed
Mecoli1219 wants to merge 10 commits into
huggingface:mainfrom
Mecoli1219:liger-cpo
Closed

[Liger] Integrate Liger CPO & SimPO#2506
Mecoli1219 wants to merge 10 commits into
huggingface:mainfrom
Mecoli1219:liger-cpo

Conversation

@Mecoli1219

@Mecoli1219 Mecoli1219 commented Dec 20, 2024 •
edited
Loading

Copy link
Copy Markdown

What does this PR do?

Integrating Liger-kernel's CPO and SimPO losses into cpo_trainer.

Related to #2495

TODO:

  • Update Liger-kernel's aux-output to include chosen_rewards andrejected_rewards

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Mecoli1219
Integrate Liger CPO & SimPO
703272f 
Signed-off-by: Mecoli1219 <michaellai901026@gmail.com>
@Mecoli1219

Mecoli1219 commented Dec 20, 2024

Copy link
Copy Markdown
Author

Wait for linkedin/Liger-Kernel#492

@qgallouedec qgallouedec mentioned this pull request Dec 20, 2024
Closed
7 tasks
@kashif kashif marked this pull request as ready for review January 3, 2025 15:56
@kashif kashif changed the title [WIP] Integrate Liger CPO & SimPO [Liger] Integrate Liger CPO & SimPO Jan 3, 2025
@HuggingFaceDocBuilderDev

HuggingFaceDocBuilderDev commented Jan 3, 2025

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@kashif

kashif commented Jan 3, 2025

Copy link
Copy Markdown
Collaborator

needs: linkedin/Liger-Kernel#510

@Mecoli1219
merge upstream
eb69366 
Signed-off-by: Mecoli1219 <michaellai901026@gmail.com>
@austin362667 austin362667 mentioned this pull request Jan 8, 2025
Merged
4 tasks
@shimizust

shimizust commented Jun 9, 2025

Copy link
Copy Markdown

Hi @Mecoli1219 @kashif I was wondering the status of this integration? Can we merge these changes now?

@albertvillanova albertvillanova added the 😴 stale No update from the author, will be closed soon label Jun 3, 2026
@albertvillanova

albertvillanova commented Jun 3, 2026

Copy link
Copy Markdown
Member

Closing as stale and also because CPO was moved to experimental.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

😴 stale No update from the author, will be closed soon

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Mecoli1219 @HuggingFaceDocBuilderDev @kashif @shimizust @albertvillanova