The below error occured while trying to convert model to gguf format.
I noticed that quantized folder resides in llama.cpp/examples/quantize
RuntimeError: Unsloth: The file 'llama.cpp/llama-quantize' or 'llama.cpp/quantize' does not exist.
But we expect this file to exist! Maybe the llama.cpp developers changed the name?
# Save to q4_k_m GGUF
if True: model.save_pretrained_gguf("model", tokenizer, quantization_method = "q4_k_m")
The below error occured while trying to convert model to gguf format.
I noticed that quantized folder resides in
llama.cpp/examples/quantize