On a Geforce 4060, 16 GB ram on Windows.
I have trained an 8b deepseek model, and am trying to save it:
model.save_pretrained_gguf("model", tokenizer, quantization_method = "q4_k_m")
However, this is happening. A 7MB size unsloth.BF16.gguf file is being saved, (I have gotten up to 6% progress before the script moves ahead to the next line) and I am getting no 4 bit quantized file.
I can, however, save the lora adapters properly. Is there any way I can combine them with my model manually?
On a Geforce 4060, 16 GB ram on Windows.
I have trained an 8b deepseek model, and am trying to save it:
However, this is happening. A 7MB size
unsloth.BF16.gguffile is being saved, (I have gotten up to 6% progress before the script moves ahead to the next line) and I am getting no 4 bit quantized file.I can, however, save the lora adapters properly. Is there any way I can combine them with my model manually?