Many bug fixes by danielhanchen · Pull Request #1900 · unslothai/unsloth

danielhanchen · 2025-03-05T13:04:11Z

Solves many issues. For Colab / Kaggle, please restart and run all.
For local machines, please do:

pip install --force-reinstall --upgrade --no-cache-dir --no-deps unsloth unsloth_zoo

The main fixes are:

Fixes gibberish output for GRPO when not using vLLM
Mistral should work on GRPO
Torch 2.6 fixes
Various bug fixes and performance enhancements

Solves these issues:

NameError: name 'bias' is not defined [FIXED] #1895
modules_to_save doesn't work with PyTorch 2.6 #1882
The V100 ran GRPO training with incorrect output. #1888 Matrix mismatch runtime error #1875 GRPOTrainer example works with trl but generate "noise" with unsloth #1844 GRPO training error #1836 unsloth=2025.2.15 training result is werd when torch=2.6 #1825 GRPO without VLLM? #1810 - fixed all gibberish output for GRPO - apologies on the issue.
Slow Import Times (60+ seconds) When Using Unsloth via Entry Points #1859 Partially solved
UnslothTrainer applies ChatML template although passed train dataset is pre-tokenized and contains 'input_ids' field #1843
Llama AttributeError: 'bool' object has no attribute 'all_special_tokens' #1809
New GRPO doesnt support models besides LLAMA - (Mistral) #1790
No label_names provided for model class PeftModelForCausalLM. Since PeftModel hides base models input arguments, if label_names is not given, label_names can't be set automatically within Trainer. #1788

* Update rl.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * autocast * Update rl_replacements.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update llama.py * Update rl_replacements.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update rl_replacements.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update pyproject.toml * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update llama.py * Update _utils.py * Update llama.py * Update _utils.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update rl_replacements.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * GRPO optimized * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Selective Log softmax * Fix GRPO bsz * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Fix TRL * Metrics GRPO * Update rl_replacements.py * Update rl_replacements.py * No compile * Update rl.py * Remove docs * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * llama-quantize on WINDOWS WSL error fix - edit save.py (gguf saving breaks) (unslothai#1649) * edit save.py to fix gguf saving breaks. * add check for .exe or not exe file extension for linux and windows * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update llama.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * unsloth_num_chunks * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py (unslothai#1754) Fix typo in comment: know -> now. This was printed when running the Llama3.1_(8B)-GRPO.ipynb example notebook, so I'd expect others to run into it as well. * Optional logits * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * fix an import error (unslothai#1767) * fix an import error * Delete .gitignore * Update loader.py * Update save.py --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> * SamplingParams * Convert mask to float (unslothai#1762) * [Windows Support] Add latest `xformers` wheels to pyproject.toml (unslothai#1753) * Add latest xformers * Add a couple of lines to docs * vLLMSamplingParams * Update __init__.py * default num_chunks == -1 * Versioning * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update _utils.py * Update rl_replacements.py * Update rl_replacements.py * Update pyproject.toml * Update pyproject.toml * Export Model to ollama.com (unslothai#1648) * Ollama Export Model to ollama.com Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> * Check for model_name Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> * subprocess use instead of requests | added check for ollama server Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> * create_ollama_model Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> * create_ollama_model | fix Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> * Push to Ollama Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> --------- Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> * Update cross_entropy_loss.py * torch_cuda_device * Update utils.py * Update utils.py * Update utils.py * device * device * Update loader.py * Update llama.py * Update README.md * Update llama.py * Update llama.py * Update _utils.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * __version__ * Update rl.py * Bug fixes * Bug fixes * Update llama.py * Update _utils.py * _wrap_fast_inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update _utils.py * SFT dataset prepare * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update llama.py * Update llama.py * Update utils.py * bug fix * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update __init__.py --------- Signed-off-by: Jyotin Goel <b22ai063@iitj.ac.in> Co-authored-by: Gennadii Manzhos <105049664+everythingisc00l@users.noreply.github.com> Co-authored-by: Seth Weidman <seth@sethweidman.com> Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com> Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com> Co-authored-by: Ben <6579034+versipellis@users.noreply.github.com> Co-authored-by: Jyotin Goel <120490013+gjyotin305@users.noreply.github.com>

danielhanchen added 30 commits February 12, 2025 02:27

Update rl.py

9aad48e

Update llama.py

f121a5c

Update llama.py

5052d35

Update llama.py

a11aa96

Update llama.py

a6abe02

autocast

d867faa

Update rl_replacements.py

44c9228

Update llama.py

e83d854

Update rl_replacements.py

623eb65

Update rl_replacements.py

7e612f0

Update rl_replacements.py

a45266b

Update rl_replacements.py

c855d7e

Update llama.py

d7cefba

Update rl_replacements.py

52d996a

Update llama.py

56f5b31

Update llama.py

5f1e98c

Update llama.py

e713129

Update llama.py

310fc16

Update llama.py

76a122e

Update rl_replacements.py

2dd29e5

Update llama.py

3c5be91

Update llama.py

e548b15

Update llama.py

296b3b3

Update llama.py

8de588b

Update pyproject.toml

f87909a

Update llama.py

2704440

Update llama.py

42e1967

Update llama.py

36bf805

Update llama.py

a3af8e3

Update llama.py

9d10d2f

danielhanchen added 27 commits March 4, 2025 18:21

Update llama.py

109364b

Update llama.py

dd4bd07

Update llama.py

b356fce

Update llama.py

e022016

Update llama.py

12094a7

Update llama.py

2836128

Update llama.py

c956616

Update llama.py

e887f43

Update llama.py

95f872d

Update llama.py

647dbb4

Update _utils.py

f640c8d

SFT dataset prepare

91a4fce

Update pyproject.toml

4495148

Update rl_replacements.py

f41dff5

Update rl_replacements.py

0a3dbfa

Update rl_replacements.py

7d8f100

Update rl.py

413ea80

Update llama.py

3f5ce93

Update llama.py

185bced

Update utils.py

fd11ad7

bug fix

97ed0b4

Update llama.py

68eca88

Update llama.py

5daf9b5

Update llama.py

858bb76

Update llama.py

daedc34

Update llama.py

95e2371

Update __init__.py

fccd68a

danielhanchen merged commit f48cb41 into main Mar 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Many bug fixes#1900

Many bug fixes#1900
danielhanchen merged 535 commits into
mainfrom
nightly

danielhanchen commented Mar 5, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Conversation

danielhanchen commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

danielhanchen commented Mar 5, 2025 •

edited

Loading