Skip to content

llama-cli: model params not propagated #23847

@therealkenc

Description

@therealkenc

Name and Version

version: 9366 (ff6dfc3)
built with GNU 15.2.0 for Linux x86_64

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-cli

Command line

{
  ./build/bin/llama-cli -v -st --simple-io --no-warmup \
    --no-display-prompt --no-show-timings \
    --log-colors off --no-log-prefix --no-log-timestamps \
    -m gguf/unsloth/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
    -p "what is the capital of france?" -n 1 \
    >/dev/null
} 2>&1 | grep -E "general\.sampling|sampler params|top_k ="

Problem description & steps to reproduce

Related #17120 #21509

Model top_k is 20 but gets launched as the default 40.

First Bad Commit

WAG maybe #17824, or maybe it only ever worked for llama-server.

Relevant log output

llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
slot launch_slot_: id  0 | task -1 | sampler params: 
	top_k = 40, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.800

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions