Skip to content

model: using single llm_build per arch#21970

Merged
ngxson merged 5 commits into
ggml-org:masterfrom
ngxson:xsn/one_llm_build_per_arch
Apr 16, 2026
Merged

model: using single llm_build per arch#21970
ngxson merged 5 commits into
ggml-org:masterfrom
ngxson:xsn/one_llm_build_per_arch

Conversation

@ngxson

@ngxson ngxson commented Apr 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Overview

Prepare for #21966

Using one single llm_build_* class per arch will make the migration a bit easier.

Example before:

        case LLM_ARCH_LLAMA:
            {
                llm = std::make_unique<llm_build_llama<false>>(*this, params);
            } break;
        case LLM_ARCH_LLAMA4:
            {
                if (hparams.swa_type == LLAMA_SWA_TYPE_NONE) {
                    llm = std::make_unique<llm_build_llama<false>>(*this, params);
                } else {
                    llm = std::make_unique<llm_build_llama_iswa>(*this, params);
                }
            } break;

Example after:

        case LLM_ARCH_LLAMA:
            {
                llm = std::make_unique<llm_build_llama<false>>(*this, params);
            } break;
        case LLM_ARCH_LLAMA4:
            {
                if (hparams.swa_type == LLAMA_SWA_TYPE_NONE) {
                    llm = std::make_unique<llm_build_llama4<false>>(*this, params);
                } else {
                    llm = std::make_unique<llm_build_llama4<true>>(*this, params);
                }
            } break;

Requirements

@ngxson ngxson requested a review from pwilkin April 15, 2026 22:22
@ngxson ngxson requested review from CISC and ggerganov as code owners April 15, 2026 22:22
CISC
CISC approved these changes Apr 15, 2026
View reviewed changes

@CISC CISC left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will conflict with #21971, mind landing that one first?

@github-actions github-actions Bot added the model Model specific label Apr 15, 2026
ggerganov
ggerganov reviewed Apr 16, 2026
View reviewed changes
Comment thread src/models/models.h
Comment on lines +710 to +711
struct llm_build_t5encoder : public llm_build_t5<true> {
llm_build_t5encoder(const llama_model & model, const llm_graph_params & params);

@ggerganov ggerganov Apr 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to keep this, instead of using llm_build_t5<true>?

@CISC CISC Apr 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's there to match the arch name (for the migration script)?

@ngxson ngxson Apr 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ggerganov the t5encoder has its own tensor loader and hparams loader, so I think it make sense to have a dedicated graph builder. The dedicated class will then expanded into llama_model_t5encoder during the migration (via a script), and its hparams/tensors loader will be moved there

@ngxson ngxson mentioned this pull request Apr 16, 2026
Merged
6 tasks
@ngxson

ngxson commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator Author

Should be ok now I guess? @CISC @ggerganov

@ggerganov ggerganov added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Apr 16, 2026
@ngxson ngxson merged commit 4fbdabd into ggml-org:master Apr 16, 2026
46 of 50 checks passed
cnsiva pushed a commit to saas-home/llama.cpp that referenced this pull request Apr 17, 2026
@ngxson @cnsiva
model: using single llm_build per arch (ggml-org#21970)
e911142 
* model: using single llm_build per arch

* fix merge

* nits
samuraieng pushed a commit to samuraieng/llama.cpp that referenced this pull request Apr 19, 2026
@ngxson @samuraieng
model: using single llm_build per arch (ggml-org#21970)
3754f53 
* model: using single llm_build per arch

* fix merge

* nits
mengqin pushed a commit to mengqin/llama.cpp that referenced this pull request Apr 20, 2026
@ngxson @mengqin
model: using single llm_build per arch (ggml-org#21970)
7621d24 
* model: using single llm_build per arch

* fix merge

* nits
ArberSephirotheca pushed a commit to ArberSephirotheca/llama.cpp that referenced this pull request Apr 21, 2026
@ngxson @ArberSephirotheca
model: using single llm_build per arch (ggml-org#21970)
94adb82 
* model: using single llm_build per arch

* fix merge

* nits
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Apr 23, 2026
@ngxson @arthw
model: using single llm_build per arch (ggml-org#21970)
5d34078 
* model: using single llm_build per arch

* fix merge

* nits
rsenthilkumar6 pushed a commit to rsenthilkumar6/llama.cpp that referenced this pull request May 1, 2026
@ngxson @rsenthilkumar6
model: using single llm_build per arch (ggml-org#21970)
79b94d1 
* model: using single llm_build per arch

* fix merge

* nits
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
@ngxson
model: using single llm_build per arch (ggml-org#21970)
ea9998b 
* model: using single llm_build per arch

* fix merge

* nits
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@ngxson
model: using single llm_build per arch (ggml-org#21970)
ff55612 
* model: using single llm_build per arch

* fix merge

* nits
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@ngxson
model: using single llm_build per arch (ggml-org#21970)
f5ea44d 
* model: using single llm_build per arch

* fix merge

* nits
fewtarius pushed a commit to fewtarius/llama.cpp that referenced this pull request May 30, 2026
@ngxson
model: using single llm_build per arch (ggml-org#21970)
2e9c80a 
* model: using single llm_build per arch

* fix merge

* nits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CISC CISC CISC approved these changes

@ggerganov ggerganov ggerganov approved these changes

@pwilkin pwilkin Awaiting requested review from pwilkin

Assignees

No one assigned

Labels

merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. model Model specific

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ngxson @CISC @ggerganov