llama : deprecate llama_set_warmup#24009
Merged
Merged
Conversation
0665b8c to
537d4b4
Compare
Contributor
|
Shouldn't at least the server now do a warmup? Or will it be added separately? |
Member
Author
It can be added to the warmup section of |
Contributor
|
Also I think this will negatively impact llama-bench numbers because there will be no warmup |
Member
Author
No, |
danbev
approved these changes
Jun 2, 2026
1 task
[no ci] Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
cont #11571
Deprecate the functionality for pre-loading all MoE experts at the context/graph level. The user code would now have to be responsible to do the necessary warmup runs to guarantee that the weights are hot (in case that is needed by the application).
Additional information
The
cparams.warmupflag changes the tensor shapes in the FFN graph. Before #23861 this wasn't causing problems because we were over-allocating outputs in the compute buffer that silently covered for the extra experts during warmup. Now after being more strict with the output allocations, the issue shows up: https://github.com/ggml-org/llama.cpp/actions/runs/26794936619/job/78989134399#step:5:3668Generally, we want to keep the graphs as static as possible. The goal is once we reserve memory at the start to never have to reserve again (unless some relevant context parameter is modified explicitly).
Extra CI
Requirements