~80% memory reduction with native liger-kernel losses. For more information, see https://github.com/linkedin/Liger-Kernel/releases/tag/v0.5.0 - [x] DPO #2568 - [ ] BCO - [ ] ORPO #2482 - [ ] SimPO (in CPO) #2506 - [ ] JSD (in GKD) https://github.com/huggingface/trl/pull/2573 - [x] GRPO https://github.com/huggingface/trl/pull/3184 - [x] KTO https://github.com/huggingface/trl/pull/2812
~80% memory reduction with native liger-kernel losses. For more information, see https://github.com/linkedin/Liger-Kernel/releases/tag/v0.5.0