Skip to content

Misc. bug: issues with hf cache since path consolidation (model loading/leftover files) #21364

Closed
#21413
Closed
Misc. bug: issues with hf cache since path consolidation (model loading/leftover files)#21364
#21413
Labels
bug-unconfirmed
@fkroener

Description

@fkroener
fkroener
opened on Apr 3, 2026

Name and Version

version: 8645 (57ace0d)
built with GNU 15.2.1 for Linux x86_64

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

Other (Please specify in the next section)

Command line

./build/bin/llama-server -dev Vulkan0 --host 0.0.0.0 --offline --models-preset presets.ini

Problem description & steps to reproduce

This might actually be three separate issues, but since they appear to be related to the hugging face hub migration I'm describing the lot of them.

Somehow since the introduction of standard hugging face cache support I'm facing several issues, where models weren't moved completely (even after introduction of support for split files) and using presets results in alias problems, without aliases ever being declared:

e.g. when running in router mode and then selecting gemma4:

[45303] srv   load_models: Loaded 19 custom model presets from /home/fkroener/llm/llama.cpp/presets.ini
[45303] main: failed to initialize router models: alias 'unsloth/gemma-4-E4B-it-GGUF:UD-Q5_K_XL' for model 'HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive:Q6_K' conflicts with alias of model 'AesSedai/Step-3.5-Flash-GGUF:IQ4_XS'

Additionally the llama-server lists models I don't think I actually have downloaded (for unsloth dynamics, where I got unsloth/Qwen3.5-35B-A3B-GGUF:UD-Q6_K_XL it also lists unsloth/Qwen3.5-35B-A3B-GGUF:Q6_K_XL

First Bad Commit

8c7957c

Relevant log output

Log of original model migration 
================================================================================
WARNING: Migrating cache to HuggingFace cache directory
  Old cache: /home/fkroener/.cache/llama.cpp/
  New cache: /home/fkroener/.cache/huggingface/hub
This one-time migration moves models previously downloaded with -hf
from the legacy llama.cpp cache to the standard HuggingFace cache.
Models downloaded with --model-url are not affected.
================================================================================
migrate_single_file: migrated mradermacher_GLM-4.7-Flash-Derestricted-i1-GGUF_GLM-4.7-Flash-Derestricted.i1-Q6_K.gguf -> /home/fkroener/.cache/huggingface/hub/models--mradermacher--GLM-4.7-Flash-Derestricted-i1-GGUF/snapshots/74530c2e8d07e1d323524c4953ee1a523d979192/GLM-4.7-Flash-Derestricted.i1-Q6_K.gguf
migrate_single_file: migrated unsloth_MiniMax-M2.5-GGUF_UD-IQ3_XXS_MiniMax-M2.5-UD-IQ3_XXS-00001-of-00003.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--MiniMax-M2.5-GGUF/snapshots/7c50dca0e5592483ad308ecffc876aecac725660/UD-IQ3_XXS/MiniMax-M2.5-UD-IQ3_XXS-00001-of-00003.gguf
migrate_single_file: migrated unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00001-of-00004.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--MiniMax-M2.5-GGUF/snapshots/7c50dca0e5592483ad308ecffc876aecac725660/UD-Q3_K_XL/MiniMax-M2.5-UD-Q3_K_XL-00001-of-00004.gguf
migrate_single_file: migrated AesSedai_Step-3.5-Flash-GGUF_IQ4_XS_Step-3.5-Flash-IQ4_XS-00001-of-00003.gguf -> /home/fkroener/.cache/huggingface/hub/models--AesSedai--Step-3.5-Flash-GGUF/snapshots/341d41e9b6687b3b3830e130eb1199e60096ebd6/IQ4_XS/Step-3.5-Flash-IQ4_XS-00001-of-00003.gguf
migrate_single_file: migrated gghfez_gpt-oss-120b-Derestricted.MXFP4_MOE-gguf_gpt-oss-120b-Derestricted.MXFP4_MOE.gguf -> /home/fkroener/.cache/huggingface/hub/models--gghfez--gpt-oss-120b-Derestricted.MXFP4_MOE-gguf/snapshots/e17bdc5ae6bb53eac3582001602db0dd100f3259/gpt-oss-120b-Derestricted.MXFP4_MOE.gguf
migrate_single_file: migrated unsloth_GLM-4.7-Flash-GGUF_GLM-4.7-Flash-UD-Q8_K_XL.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--GLM-4.7-Flash-GGUF/snapshots/0d32489ecb9db6d2a4fc93bd27ef01519f95474d/GLM-4.7-Flash-UD-Q8_K_XL.gguf
migrate_single_file: migrated MuXodious_GLM-4.7-Flash-absolute-heresy-GGUF_GLM-4.7-Flash-Absolute-Heresy-MXFP4_MoE.gguf -> /home/fkroener/.cache/huggingface/hub/models--MuXodious--GLM-4.7-Flash-absolute-heresy-GGUF/snapshots/d2764eed8be4e73578d4f74f2fb989671d685831/GLM-4.7-Flash-Absolute-Heresy-MXFP4_MoE.gguf
migrate_single_file: migrated Sabomako_Qwen3.5-122B-A10B-heretic-GGUF_Qwen3.5-122B-A10B-heretic.mxfp4_moe-00001-of-00002.gguf -> /home/fkroener/.cache/huggingface/hub/models--Sabomako--Qwen3.5-122B-A10B-heretic-GGUF/snapshots/c34febe24de2872c110fe4dc23028004949bac6b/Qwen3.5-122B-A10B-heretic.mxfp4_moe-00001-of-00002.gguf
migrate_single_file: migrated Sabomako_Qwen3.5-122B-A10B-heretic-GGUF_mmproj-F32.gguf -> /home/fkroener/.cache/huggingface/hub/models--Sabomako--Qwen3.5-122B-A10B-heretic-GGUF/snapshots/c34febe24de2872c110fe4dc23028004949bac6b/mmproj-F32.gguf
migrate_single_file: migrated unsloth_Qwen3.5-35B-A3B-GGUF_Qwen3.5-35B-A3B-UD-Q6_K_XL.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-35B-A3B-GGUF/snapshots/bc014a17be43adabd7066b7a86075ff935c6a4e2/Qwen3.5-35B-A3B-UD-Q6_K_XL.gguf
migrate_single_file: migrated unsloth_Qwen3.5-35B-A3B-GGUF_mmproj-BF16.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-35B-A3B-GGUF/snapshots/bc014a17be43adabd7066b7a86075ff935c6a4e2/mmproj-BF16.gguf
migrate_single_file: migrated unsloth_Qwen3.5-122B-A10B-GGUF_UD-Q4_K_XL_Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-122B-A10B-GGUF/snapshots/51eab4d59d53f573fb9206cb3ce613f1d0aa392b/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf
migrate_single_file: migrated unsloth_Qwen3.5-122B-A10B-GGUF_mmproj-BF16.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-122B-A10B-GGUF/snapshots/51eab4d59d53f573fb9206cb3ce613f1d0aa392b/mmproj-BF16.gguf
migrate_single_file: unsloth_Qwen3.5-397B-A17B-GGUF_Qwen3.5-397B-A17B-UD-TQ1_0.gguf not found in current repo, deleting...
migrate_single_file: migrated unsloth_Qwen3.5-397B-A17B-GGUF_mmproj-BF16.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-397B-A17B-GGUF/snapshots/da33c16fa4440f831149fcf53b98a22bc07785e5/mmproj-BF16.gguf
migrate_single_file: migrated mradermacher_Qwen3.5-35B-A3B-heretic-i1-GGUF_Qwen3.5-35B-A3B-heretic.i1-Q4_K_M.gguf -> /home/fkroener/.cache/huggingface/hub/models--mradermacher--Qwen3.5-35B-A3B-heretic-i1-GGUF/snapshots/074379c7b5d0ad38b2ec0168f91d98fe414cfc99/Qwen3.5-35B-A3B-heretic.i1-Q4_K_M.gguf
migrate_single_file: migrated unsloth_Qwen3-Coder-Next-GGUF_UD-Q8_K_XL_Qwen3-Coder-Next-UD-Q8_K_XL-00001-of-00003.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3-Coder-Next-GGUF/snapshots/ce09c67b53bc8739eef83fe67b2f5d293c270632/UD-Q8_K_XL/Qwen3-Coder-Next-UD-Q8_K_XL-00001-of-00003.gguf
migrate_single_file: migrated unsloth_Qwen3.5-9B-GGUF_Qwen3.5-9B-Q6_K.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-9B-GGUF/snapshots/3885219b6810b007914f3a7950a8d1b469d598a5/Qwen3.5-9B-Q6_K.gguf
migrate_single_file: migrated unsloth_Qwen3.5-9B-GGUF_mmproj-F16.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-9B-GGUF/snapshots/3885219b6810b007914f3a7950a8d1b469d598a5/mmproj-F16.gguf
migrate_single_file: migrated unsloth_Qwen3.5-4B-GGUF_Qwen3.5-4B-Q8_0.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-4B-GGUF/snapshots/e87f176479d0855a907a41277aca2f8ee7a09523/Qwen3.5-4B-Q8_0.gguf
migrate_single_file: migrated unsloth_Qwen3.5-4B-GGUF_mmproj-F16.gguf -> /home/fkroener/.cache/huggingface/hub/models--unsloth--Qwen3.5-4B-GGUF/snapshots/e87f176479d0855a907a41277aca2f8ee7a09523/mmproj-F16.gguf
migrate_single_file: migrated HauhauCS_Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive_Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf -> /home/fkroener/.cache/huggingface/hub/models--HauhauCS--Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive/snapshots/53367faad177ee6a23601983cdac4308b51393df/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf
migrate_single_file: migrated HauhauCS_Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive_mmproj-Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-f16.gguf -> /home/fkroener/.cache/huggingface/hub/models--HauhauCS--Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive/snapshots/53367faad177ee6a23601983cdac4308b51393df/mmproj-Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-f16.gguf
migrate_single_file: migrated mradermacher_Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-i1-GGUF_Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled.i1-Q4_K_M.gguf -> /home/fkroener/.cache/huggingface/hub/models--mradermacher--Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-i1-GGUF/snapshots/12fd25dc5be1888903333e1b62fbae282a53b07e/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled.i1-Q4_K_M.gguf
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build: 8503 (c9dc43333) with GNU 15.2.1 for Linux x86_64
Current content in ~/.cache/llama.cpp 
ls ~/.cache/llama.cpp/
 AesSedai_Step-3.5-Flash-GGUF_IQ4_XS_Step-3.5-Flash-IQ4_XS-00002-of-00003.gguf                          unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00003-of-00004.gguf.etag
 AesSedai_Step-3.5-Flash-GGUF_IQ4_XS_Step-3.5-Flash-IQ4_XS-00002-of-00003.gguf.etag                     unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00004-of-00004.gguf
 AesSedai_Step-3.5-Flash-GGUF_IQ4_XS_Step-3.5-Flash-IQ4_XS-00003-of-00003.gguf                          unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00004-of-00004.gguf.etag
 AesSedai_Step-3.5-Flash-GGUF_IQ4_XS_Step-3.5-Flash-IQ4_XS-00003-of-00003.gguf.etag                     unsloth_Qwen3-Coder-Next-GGUF_UD-Q8_K_XL_Qwen3-Coder-Next-UD-Q8_K_XL-00002-of-00003.gguf
 Sabomako_Qwen3.5-122B-A10B-heretic-GGUF_Qwen3.5-122B-A10B-heretic.mxfp4_moe-00002-of-00002.gguf        unsloth_Qwen3-Coder-Next-GGUF_UD-Q8_K_XL_Qwen3-Coder-Next-UD-Q8_K_XL-00002-of-00003.gguf.etag
 Sabomako_Qwen3.5-122B-A10B-heretic-GGUF_Qwen3.5-122B-A10B-heretic.mxfp4_moe-00002-of-00002.gguf.etag   unsloth_Qwen3-Coder-Next-GGUF_UD-Q8_K_XL_Qwen3-Coder-Next-UD-Q8_K_XL-00003-of-00003.gguf
 unsloth_MiniMax-M2.5-GGUF_UD-IQ3_XXS_MiniMax-M2.5-UD-IQ3_XXS-00002-of-00003.gguf                       unsloth_Qwen3-Coder-Next-GGUF_UD-Q8_K_XL_Qwen3-Coder-Next-UD-Q8_K_XL-00003-of-00003.gguf.etag
 unsloth_MiniMax-M2.5-GGUF_UD-IQ3_XXS_MiniMax-M2.5-UD-IQ3_XXS-00002-of-00003.gguf.etag                  unsloth_Qwen3.5-122B-A10B-GGUF_UD-Q4_K_XL_Qwen3.5-122B-A10B-UD-Q4_K_XL-00002-of-00003.gguf
 unsloth_MiniMax-M2.5-GGUF_UD-IQ3_XXS_MiniMax-M2.5-UD-IQ3_XXS-00003-of-00003.gguf                       unsloth_Qwen3.5-122B-A10B-GGUF_UD-Q4_K_XL_Qwen3.5-122B-A10B-UD-Q4_K_XL-00002-of-00003.gguf.etag
 unsloth_MiniMax-M2.5-GGUF_UD-IQ3_XXS_MiniMax-M2.5-UD-IQ3_XXS-00003-of-00003.gguf.etag                  unsloth_Qwen3.5-122B-A10B-GGUF_UD-Q4_K_XL_Qwen3.5-122B-A10B-UD-Q4_K_XL-00003-of-00003.gguf
 unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00002-of-00004.gguf                       unsloth_Qwen3.5-122B-A10B-GGUF_UD-Q4_K_XL_Qwen3.5-122B-A10B-UD-Q4_K_XL-00003-of-00003.gguf.etag
 unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00002-of-00004.gguf.etag                  unsloth_Qwen3.5-35B-A3B-GGUF_mmproj-F32.gguf
 unsloth_MiniMax-M2.5-GGUF_UD-Q3_K_XL_MiniMax-M2.5-UD-Q3_K_XL-00003-of-00004.gguf                       unsloth_Qwen3.5-35B-A3B-GGUF_mmproj-F32.gguf.etag
Current content in ~/.cache/huggingface/hub 
ls .cache/huggingface/hub/
 models--AesSedai--Step-3.5-Flash-GGUF                              models--mradermacher--Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-i1-GGUF   models--unsloth--gemma-4-E4B-it-GGUF     models--unsloth--Qwen3.5-122B-A10B-GGUF
 models--gghfez--gpt-oss-120b-Derestricted.MXFP4_MOE-gguf           models--mradermacher--Qwen3.5-35B-A3B-heretic-i1-GGUF                           models--unsloth--GLM-4.7-Flash-GGUF      models--unsloth--Qwen3.5-35B-A3B-GGUF
 models--HauhauCS--Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive   models--MuXodious--GLM-4.7-Flash-absolute-heresy-GGUF                           models--unsloth--LTX-2.3-GGUF            models--unsloth--Qwen3.5-397B-A17B-GGUF
 models--Lightricks--LTX-2.3                                        models--Sabomako--Qwen3.5-122B-A10B-heretic-GGUF                                models--unsloth--MiniMax-M2.5-GGUF       models--unsloth--Qwen3.5-4B-GGUF
 models--mradermacher--GLM-4.7-Flash-Derestricted-i1-GGUF           models--unsloth--gemma-3-12b-it-qat-GGUF                                        models--unsloth--Qwen3-Coder-Next-GGUF   models--unsloth--Qwen3.5-9B-GGUF
llama-server presets list log 
init: using 31 threads for HTTP server
srv   load_models: Loaded 20 cached model presets
srv   load_models: Loaded 19 custom model presets from /home/fkroener/llm/llama.cpp/presets.ini
srv   load_models: Available models (24) (*: custom preset)
srv   load_models:   * AesSedai/Step-3.5-Flash-GGUF:IQ4_XS
srv   load_models:   * HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive:Q6_K
srv   load_models:   * MuXodious/GLM-4.7-Flash-absolute-heresy-GGUF:MXFP4_MOE
srv   load_models:   * Sabomako/Qwen3.5-122B-A10B-heretic-GGUF:MXFP4_MOE
srv   load_models:   * gghfez/gpt-oss-120b-Derestricted.MXFP4_MOE-gguf:MXFP4_MOE
srv   load_models:   * mradermacher/GLM-4.7-Flash-Derestricted-i1-GGUF:Q6_K
srv   load_models:   * mradermacher/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-i1-GGUF:Q4_K_M
srv   load_models:   * mradermacher/Qwen3.5-35B-A3B-heretic-i1-GGUF:Q4_K_M
srv   load_models:   * unsloth/GLM-4.7-Flash-GGUF:Q8_K_XL
srv   load_models:     unsloth/LTX-2.3-GGUF:Q4_K_M
srv   load_models:   * unsloth/MiniMax-M2.5-GGUF:IQ3_XXS
srv   load_models:   * unsloth/MiniMax-M2.5-GGUF:Q3_K_XL
srv   load_models:   * unsloth/Qwen3-Coder-Next-GGUF:Q8_K_XL
srv   load_models:     unsloth/Qwen3.5-122B-A10B-GGUF:Q4_K_XL
srv   load_models:   * unsloth/Qwen3.5-122B-A10B-GGUF:UD-Q4_K_XL
srv   load_models:     unsloth/Qwen3.5-35B-A3B-GGUF:Q6_K_XL
srv   load_models:   * unsloth/Qwen3.5-35B-A3B-GGUF:UD-Q6_K_XL
srv   load_models:   * unsloth/Qwen3.5-397B-A17B-GGUF:UD-TQ1_0
srv   load_models:   * unsloth/Qwen3.5-4B-GGUF:Q8_0
srv   load_models:   * unsloth/Qwen3.5-9B-GGUF:Q6_K
srv   load_models:     unsloth/gemma-3-12b-it-qat-GGUF:Q4_K_XL
srv   load_models:   * unsloth/gemma-4-E4B-it-GGUF:Q5_K_M
srv   load_models:     unsloth/gemma-4-E4B-it-GGUF:Q5_K_XL
srv   load_models:   * unsloth/gemma-4-E4B-it-GGUF:UD-Q5_K_XL
main: starting router server, no model will be loaded in this process
presets.ini 
[*]
dev=Vulkan0
dio=yes
models-max = 1
spec-type = ngram-mod
spec-ngram-size-n = 48 ; 24
sleep-idle-seconds = 900
webui-mcp-proxy = 1
np = 2
ubatch-size  = 512 
batch-size = 2048

[AesSedai/Step-3.5-Flash-GGUF:IQ4_XS]
hf = AesSedai/Step-3.5-Flash-GGUF:IQ4_XS
ub = 256
b = 2048
temp = 1.0
c = 262144
cache-ram = 4096

[gghfez/gpt-oss-120b-Derestricted.MXFP4_MOE-gguf:MXFP4_MOE]
hf = gghfez/gpt-oss-120b-Derestricted.MXFP4_MOE-gguf:MXFP4_MOE
batch-size  = 2048
ubatch-size = 2048
top-p       = 1.0
top-k       = 0
min-p       = 0.01
temp        = 1.0
chat-template-kwargs = {"reasoning_effort": "high"}

[HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive:Q6_K]
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[mradermacher/GLM-4.7-Flash-Derestricted-i1-GGUF:Q6_K]
temp = 0.7
top-p = 1.0
min-p = 0.01
jinja=yes

[mradermacher/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-i1-GGUF:Q4_K_M]
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[mradermacher/Qwen3.5-35B-A3B-heretic-i1-GGUF:Q4_K_M]
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[MuXodious/GLM-4.7-Flash-absolute-heresy-GGUF:MXFP4_MOE]
batch-size  = 2048
ubatch-size = 2048
top-p       = 1.0
top-k       = 0
min-p       = 0.01
temp        = 1.0
chat-template-kwargs = {"reasoning_effort": "high"}

[Sabomako/Qwen3.5-122B-A10B-heretic-GGUF:MXFP4_MOE]
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[unsloth/gemma-4-E4B-it-GGUF:Q5_K_M]
temp=1.0
top-p=0.95
top-k=64

[unsloth/gemma-4-E4B-it-GGUF:UD-Q5_K_XL]
temp=1.0
top-p=0.95
top-k=64

[unsloth/GLM-4.7-Flash-GGUF:Q8_K_XL]
hf = unsloth/GLM-4.7-Flash-GGUF:Q8_K_XL
temp = 0.7
top-p = 1.0
min-p = 0.01

[unsloth/MiniMax-M2.5-GGUF:IQ3_XXS]
hf = unsloth/MiniMax-M2.5-GGUF:IQ3_XXS
temp=1.0
top-p=0.95
top-k=40
c = 196608

[unsloth/MiniMax-M2.5-GGUF:Q3_K_XL]
hf = unsloth/MiniMax-M2.5-GGUF:Q3_K_XL
temp=1.0
top-p=0.95
top-k=40
c = 65536

[unsloth/Qwen3-Coder-Next-GGUF:Q8_K_XL] 
hf = unsloth/Qwen3-Coder-Next-GGUF:Q8_K_XL
temp=1.0
top-p=0.95
top-k=40

[unsloth/Qwen3.5-4B-GGUF:Q8_0]
hf = unsloth/Qwen3.5-4B-GGUF:Q8_0
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[unsloth/Qwen3.5-9B-GGUF:Q6_K] 
hf = unsloth/Qwen3.5-9B-GGUF:Q6_K
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[unsloth/Qwen3.5-35B-A3B-GGUF:UD-Q6_K_XL] 
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[unsloth/Qwen3.5-122B-A10B-GGUF:UD-Q4_K_XL] 
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0

[unsloth/Qwen3.5-397B-A17B-GGUF:UD-TQ1_0] 
temp=0.6
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0
c = 131072

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug-unconfirmed

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions