INFO:lora-to-gguf:Loading base model from Hugging Face: unsloth/gemma-4-e2b-it-unsloth-bnb-4bit
INFO:httpx:HTTP Request: HEAD https://huggingface.co/unsloth/gemma-4-e2b-it-unsloth-bnb-4bit/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
INFO:httpx:HTTP Request: HEAD https://huggingface.co/unsloth/gemma-4-E2B-it-unsloth-bnb-4bit/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
INFO:httpx:HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/unsloth/gemma-4-E2B-it-unsloth-bnb-4bit/5dce9f929eb53aa6ea41eb3954e9df481452ac13/config.json "HTTP/1.1 200 OK"
Traceback (most recent call last):
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 2226, in __getattr__
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 2460, in _get_module
raise e
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 2458, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/usr/local/lib/python3.12/dist-packages/transformers/models/gemma4/configuration_gemma4.py", line 21, in <module>
from ...utils.type_validators import interval
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/type_validators.py", line 8, in <module>
from ..video_utils import VideoMetadataType
File "/usr/local/lib/python3.12/dist-packages/transformers/video_utils.py", line 27, in <module>
from .image_transforms import PaddingMode, to_channel_dimension_format
File "/usr/local/lib/python3.12/dist-packages/transformers/image_transforms.py", line 22, in <module>
from .image_utils import (
File "/usr/local/lib/python3.12/dist-packages/transformers/image_utils.py", line 54, in <module>
from torchvision.io import ImageReadMode, decode_image
File "/usr/local/lib/python3.12/dist-packages/torchvision/__init__.py", line 10, in <module>
from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils # usort:skip
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torchvision/_meta_registrations.py", line 163, in <module>
@torch.library.register_fake("torchvision::nms")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/library.py", line 828, in register
use_lib._register_fake(op_name, func, _stacklevel=stacklevel + 1)
File "/usr/local/lib/python3.12/dist-packages/torch/library.py", line 198, in _register_fake
handle = entry.fake_impl.register(func_to_register, source)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/_library/fake_impl.py", line 31, in register
if torch._C._dispatch_has_kernel_for_dispatch_key(self.qualname, "Meta"):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: operator torchvision::nms does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/kaggle/working/llama.cpp/convert_lora_to_gguf.py", line 372, in <module>
hparams, dir_base_model = load_hparams_from_hf(model_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/kaggle/working/llama.cpp/convert_lora_to_gguf.py", line 317, in load_hparams_from_hf
config = AutoConfig.from_pretrained(hf_model_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/configuration_auto.py", line 377, in from_pretrained
explicit_local_code = has_local_code and not CONFIG_MAPPING[config_dict["model_type"]].__module__.startswith(
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/configuration_auto.py", line 110, in __getitem__
if hasattr(self._modules[module_name], value):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 2314, in __getattr__
raise ModuleNotFoundError(
ModuleNotFoundError: Could not import module 'Gemma4Config'. Are this object's requirements defined correctly?
Name and Version
Used git clone:
Commit 67b2b7f
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
Python/Bash scripts
Command line
python llama.cpp/convert_lora_to_gguf.py ./lora_adaptor \ --outfile lora-adaptor.gguf \ --outtype f16Problem description & steps to reproduce
I fine-tuned a Gemma 4 model using unsloth. I have the LoRA adaptors in
.safetensorsform stored at hugging face. I tried to convert the LoRA adaptor to GGUF using the script. It fails and gives the error as attached.You can check using my adaptor if you want:
snipeart007/student-buddy-mental-health-adaptor
First Bad Commit
No response
Relevant log output
Logs