Skip to content

arch: refactor LLM_TENSOR_NAMES#18051

Merged
ngxson merged 7 commits intoggml-org:masterfrom
ngxson:xsn/arch_refactor_llm_names
Dec 16, 2025
Merged

arch: refactor LLM_TENSOR_NAMES#18051
ngxson merged 7 commits intoggml-org:masterfrom
ngxson:xsn/arch_refactor_llm_names

Conversation

@ngxson
Copy link
Collaborator

@ngxson ngxson commented Dec 15, 2025 •
edited
Loading

Motivation

while working on #18042 , I want to add a new arch GLM4V which is mostly the same as GLM4, but use M-RoPE instead of normal RoPE.

However, I feel like sometimes it can be quite too redundant to duplicate the list of tensors and cgraph (for cgraph, we can reuse the same file, but switch rope type using template)

This PR propose a new way to organize model tensors naming, bring it more aligned with the convert_hf_to_gguf.py script, while also allow multiple models to reuse the same mapping.

TODO in a follow-up PR: also update the mapping in convert_hf_to_gguf.py to allow models to reuse the same mappings

This PR was mostly generated using this script: https://gist.github.com/ngxson/a20411de6bc66a84a4d354b254bfe4da

Before

static const std::map<llm_arch, std::map<llm_tensor, const char *>> LLM_TENSOR_NAMES = {
    {
        LLM_ARCH_LLAMA,
        {
            { LLM_TENSOR_TOKEN_EMBD,      "token_embd" },
            { LLM_TENSOR_OUTPUT_NORM,     "output_norm" },
            { LLM_TENSOR_OUTPUT,          "output" },
            { LLM_TENSOR_ROPE_FREQS,      "rope_freqs" },
            { LLM_TENSOR_ATTN_NORM,       "blk.%d.attn_norm" },
            { LLM_TENSOR_ATTN_Q,          "blk.%d.attn_q" },
            ...

After

static const std::map<llm_tensor, const char *> LLM_TENSOR_NAMES = {
    { LLM_TENSOR_TOKEN_EMBD,                             "token_embd" },
    { LLM_TENSOR_OUTPUT_NORM,                            "output_norm" },
    { LLM_TENSOR_OUTPUT_NORM_LFM2,                       "token_embd_norm" }, // fix for wrong tensor name
    { LLM_TENSOR_OUTPUT,                                 "output" },
...

static std::set<llm_tensor> llm_get_tensor_names(llm_arch arch) {
    switch (arch) {
        case LLM_ARCH_LLAMA:
        case LLM_ARCH_DECI:
        case LLM_ARCH_MISTRAL3:
            return {
                LLM_TENSOR_TOKEN_EMBD,
                LLM_TENSOR_OUTPUT_NORM,
                LLM_TENSOR_OUTPUT,
                LLM_TENSOR_ROPE_FREQS,
                LLM_TENSOR_ATTN_NORM,
                ...

@ngxson ngxson requested a review from ggerganov December 15, 2025 09:09
@ngxson ngxson requested a review from CISC as a code owner December 15, 2025 09:09
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Dec 15, 2025
@ngxson ngxson added the refactoring Refactoring label Dec 15, 2025
@ngxson ngxson mentioned this pull request Dec 15, 2025
Merged
@ngxson ngxson merged commit 7f2b2f3 into ggml-org:master Dec 16, 2025
67 of 68 checks passed
@ngxson ngxson mentioned this pull request Dec 16, 2025
Merged
@loci-dev loci-dev mentioned this pull request Dec 16, 2025
Open
@tdakhran tdakhran mentioned this pull request Dec 17, 2025
Merged
Anico2 added a commit to Anico2/llama.cpp that referenced this pull request Jan 15, 2026
@Anico2
arch: refactor LLM_TENSOR_NAMES (ggml-org#18051)
83c1d89 
* arch: refactor LLM_TENSOR_NAMES

* update docs

* typo

* fix LLM_ARCH_NEMOTRON_H_MOE

* show more meaningful error message on missing tensor

* fix and tested LLM_ARCH_NEMOTRON_H_MOE
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
@ngxson
arch: refactor LLM_TENSOR_NAMES (#18051)
3776456 
* arch: refactor LLM_TENSOR_NAMES

* update docs

* typo

* fix LLM_ARCH_NEMOTRON_H_MOE

* show more meaningful error message on missing tensor

* fix and tested LLM_ARCH_NEMOTRON_H_MOE
@wallentri88 wallentri88 mentioned this pull request Feb 24, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

@CISC CISC Awaiting requested review from CISC CISC is a code owner

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation refactoring Refactoring

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ngxson @ggerganov