Skip to content

Eval bug: Gemma4ForCausalLM not supported by convert_hf_to_gguf.py #23674

@aoleg

Description

@aoleg

Name and Version

I get an error converting the model https://huggingface.co/louismuk/gemma-4-31B-heretic-NVFP4 with convert_hf_to_gguf.py

python convert_hf_to_gguf.py .\louismuk\gemma-4-31B-heretic-NVFP4
INFO:hf-to-gguf:Loading model: gemma-4-31B-heretic-NVFP4
INFO:hf-to-gguf:Model architecture: Gemma4ForCausalLM
ERROR:hf-to-gguf:Model Gemma4ForCausalLM is not supported

Hacking its config.json to replace "Gemma4ForCausalLM" with "Gemma4ForConditionalGeneration" works, and results in a loadable model (tested with coherent output).

Operating systems

Windows

GGML backends

CUDA

Hardware

RTX 5090 33GB, 64GB RAM

Models

https://huggingface.co/louismuk/gemma-4-31B-heretic-NVFP4

Problem description & steps to reproduce

Run "python convert_hf_to_gguf.py .\louismuk\gemma-4-31B-heretic-NVFP4", observe the error.

First Bad Commit

l https://huggingface.co/louismuk/gemma-4-31B-heretic-NVFP4

Relevant log output

python convert_hf_to_gguf.py .\louismuk\gemma-4-31B-heretic-NVFP4
INFO:hf-to-gguf:Loading model: gemma-4-31B-heretic-NVFP4
INFO:hf-to-gguf:Model architecture: Gemma4ForCausalLM
ERROR:hf-to-gguf:Model Gemma4ForCausalLM is not supported

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions