The default values for rope_freq_base and rope_freq_scale override the model values #765
Description
The default values from these parameters are set to 10000 and 1:
llama-cpp-python/llama_cpp/llama.py
Lines 232 to 233 in a72efc7
To use the model default values, these should be set to zero instead.
This is the relevant code in llama.cpp:
https://github.com/ggerganov/llama.cpp/blob/2777a84be429401a2b7d33c2b6a4ada1f0776f1b/llama.cpp#L6699-L6701
Setting an incorrect value may result in poor generation quality in models that use a different value for these parameters, such as CodeLlama.