Skip to content

RuntimeError: Missing 'llama-quantize' File When Pushing to Hub in Unsloth #1905

@nativeanish

Description

@nativeanish

Attempting to push a model to the Hugging Face Hub using unsloth results in a RuntimeError, indicating that the required llama-quantize file is missing.

model.push_to_hub_gguf(
    "aollm/qwen", 
    tokenizer, 
    quantization_method="q4_k_m", 
    token=""
)

RuntimeError: Unsloth: The file ('llama.cpp/llama-quantize' or 'llama.cpp/llama-quantize.exe' if you are on Windows WSL) or 'llama.cpp/quantize' does not exist.
But we expect this file to exist! Maybe the llama.cpp developers changed the name or check extension of the llama-quantize file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions