Skip to content

🐯 Integrate Liger GRPO Loss to GRPO Trainer#3184

Merged
kashif merged 20 commits into
huggingface:mainfrom
shivam15s:shisahni/liger_grpo
Apr 3, 2025
Merged

🐯 Integrate Liger GRPO Loss to GRPO Trainer#3184
kashif merged 20 commits into
huggingface:mainfrom
shivam15s:shisahni/liger_grpo

Conversation

@shivam15s

@shivam15s shivam15s commented Mar 31, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What does this PR do?

Integrates LigerFusedLinearGRPOLoss to GRPOTrainer

Currently works with DP and DDP. Will work on FSDP support in a subsequent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@kashif kashif self-assigned this Mar 31, 2025
qgallouedec
qgallouedec reviewed Apr 1, 2025
View reviewed changes
Comment thread trl/trainer/grpo_config.py Outdated
qgallouedec
qgallouedec reviewed Apr 1, 2025
View reviewed changes
Comment thread trl/trainer/grpo_trainer.py
qgallouedec
qgallouedec reviewed Apr 2, 2025
View reviewed changes
Comment thread trl/trainer/grpo_trainer.py Outdated
@shivam15s shivam15s marked this pull request as ready for review April 2, 2025 22:11
@shivam15s shivam15s changed the title Integrate Liger GRPO Loss to GRPO Trainer 🐯 Integrate Liger GRPO Loss to GRPO Trainer Apr 2, 2025
@shivam15s

shivam15s commented Apr 2, 2025

Copy link
Copy Markdown
Contributor Author

liger-kernel v0.5.6 is out which has the changes needed for grpo, so we can officially test this integration!

@HuggingFaceDocBuilderDev

HuggingFaceDocBuilderDev commented Apr 3, 2025

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

qgallouedec
qgallouedec approved these changes Apr 3, 2025
View reviewed changes

@qgallouedec qgallouedec left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!!

Comment thread tests/slow/test_grpo_slow.py Outdated
@kashif kashif merged commit 793735a into huggingface:main Apr 3, 2025
This was referenced Apr 5, 2025
Closed
Closed
yxliu-TAMU pushed a commit to mincheolseong/ECEN743-GRPO-Project-Proposal that referenced this pull request Apr 20, 2025
@shivam15s @kashif @qgallouedec
🐯 Integrate Liger GRPO Loss to GRPO Trainer (huggingface#3184)
d24d3c3 
Co-authored-by: Ubuntu <azureuser@liger-ci-h100-vm.kvghai4yzzmufguwws3040dwlf.dx.internal.cloudapp.net>
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
@akshansh47 akshansh47 mentioned this pull request May 19, 2026
Closed
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qgallouedec qgallouedec qgallouedec approved these changes

Assignees

@kashif kashif

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@shivam15s @HuggingFaceDocBuilderDev @qgallouedec @kashif