…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…F conversion

Adds an opt-in '--fuse-qkv' flag to convert_hf_to_gguf.py that concatenates
separate Q / K / V weight tensors into a single fused attn_qkv tensor during
HF -> GGUF conversion. Fusion happens in the shared base ModelBase.modify_tensors()
sink that subclass overrides forward into, so per-layer Q/K/V tensors are buffered
until all three are seen, then emitted as one fused tensor via torch.cat([q,k,v], dim=0).

At runtime the existing build_qkv helper introduced in ggml-org#21245 already handles the
fused path with one matmul + ggml_view_3d split, so a GGUF produced with --fuse-qkv
mostly reuses that path.

This branch also adds two small C++ correctness fixes for fused-weight + separate-bias
models (qwen2 / phi2 / starcoder2 / stablelm and similar):

  - llama-model.cpp: when wqkv is found but wqkv_b is absent, fall back to loading
    separate wq_b / wk_b / wv_b. --fuse-qkv only fuses weights, not biases.
  - llama-graph.cpp: in build_qkv fused path, when wqkv_b is absent but per-head
    biases exist, concat wq_b + wk_b + wv_b with ggml_concat and add after the
    fused matmul. Without this fix biases were silently dropped, producing garbage
    output.

gguf-py/gguf/constants.py registers MODEL_TENSOR.ATTN_QKV on every arch that
already declares ATTN_Q + ATTN_K + ATTN_V; without this the writer rejects the
fused tensor at format_tensor_name().

Tested on 4x AMD R9700 (gfx1201, ROCm 7.2): 34 architectures match nofuse output
bit-for-bit at Q8_0 and Q4_0; representative pp512 speedups: refact +22.7%,
qwen2 +13.1%, granite +12.5%, seed-oss +12.3%, phi2 +6.3%, mistral3 +5.3%.
test-llama-archs passes (0 FAIL).

This is PR 2 of the split discussed in ggml-org#20628; PR 1 (ggml-org#21245) is already merged.

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s

…ache

Wholesale sync of 384 upstream commits since merge-base 7fc1c4e (2026-04-22).
Headline upstream feature: MTP / Multi-Token Prediction (ggml-org#22673) + spec-decoding stack
(ggml-org#22838 parallel drafting, ggml-org#22227 spec-simple checkpoints, ggml-org#19493 server spec checkpointing,
plus 5 spec bug-fixes).

11 conflicts resolved across CUDA fattn / Metal / Vulkan / common:

ggml/src/ggml-cuda/fattn-mma-f16.cuh
  RDNA config matrix: union TQ's (640, 512) entries with upstream's expanded
  (112..576) RDNA matrix. Took upstream's new sentinel fallback (no ampere
  fallback for RDNA).

ggml/src/ggml-cuda/fattn.cu
  - Extended hoisted ncols2_max to include 640 head-dim.
  - Volta: dropped TQ's local ncols2_max redefinition in favor of upstream's
    hoisted version (with 640 added).
  - WMMA gate: union exclusions (40, 72, 192, 512, 576, 640).
  - Preserved TQ's RDNA4 vector-kernel branch for TurboQuant cache types
    (renamed inner gqa_ratio_eff_rdna4 to avoid shadowing); took upstream's
    restructured MFMA/CDNA path verbatim.

ggml/src/ggml-cuda/ggml-cuda.cu
  Supported-op switch: union TQ's GGML_OP_TURBO_WHT case with upstream's
  GGML_OP_ADD/SUB/MUL/DIV FP16 cases.

ggml/src/ggml-metal/ggml-metal-device.h
  Kept TQ's get_pipeline_turbo_wht declaration; took upstream's new
  get_pipeline_mul_mv_ext(lib, const ggml_tensor * op, ...) signature
  (replaces split tsrc0/tsrc1 args).

ggml/src/ggml-metal/ggml-metal-device.cpp
  Kept TQ's get_pipeline_turbo_wht implementation; took upstream's new
  get_pipeline_mul_mv_ext signature — body already uses op-> for tsrc0/tsrc1/ne12/r2/r3.

ggml/src/ggml-metal/ggml-metal-ops.cpp
  Preserved TQ's is_tq_weight rotate→matmul→unrotate path with original
  hardcoded dispatch shape. Updated non-TQ fallback to upstream's pipeline-
  param dispatch (pipeline.nr0 / nr1 / nsg + (ne11+nr1-1)/nr1 shape).

ggml/src/ggml-vulkan/* (3 files)
  Upstream-wholesale via `git checkout --theirs`. Upstream architecturally
  refactored FA from compile-time DATA_A_* variants to runtime
  FaTypeK/FaTypeV spec-constant switches. TQ's TURBO3_0 GLSL path is
  DEFERRED — Vulkan TURBO3_0 support needs re-implementation against the
  new architecture in a follow-up PR. Mac mini + M5 Max have no Vulkan;
  no in-house validation path for an immediate re-adaptation.

common/arg.cpp
  --spec-default: took upstream's new struct shape
  (params.speculative.types vector + params.speculative.ngram_mod.{n_match,n_min,n_max}).

common/speculative.cpp
  Low-acceptance reset: took upstream's sinfo.n_low / sinfo.i_last
  (variables moved into sinfo struct).

NOT-CONFLICTED upstream additions that touch TQ-adjacent surface
(auto-merged clean, but worth eyes during review):
  - src/llama-memory-recurrent.{cpp,h}  (MTP rollback API)
  - src/llama-memory-hybrid.{cpp,h}     (recall feedback_llama_memory_types
                                         + feedback_layer0_hybrid_trap)
  - src/llama-graph.cpp, src/llama-kv-cache.cpp, src/llama-context.cpp
  - tools/server/server-context.cpp     (+~1100 lines: MTP + parallel
                                         drafting + spec checkpointing)
  - src/models/qwen35*.cpp, qwen3next.cpp, delta-net-base.cpp
    (entirely new in upstream — MTP draft-head integration)

Known-deferred follow-ups:
  1. Vulkan TURBO3_0 re-implementation against runtime spec-constant FA arch
  2. PR ggml-org#21245 QKV refactor helpers — landed; TQ models not migrated to use
     them. Migrate in a focused follow-up; do not bundle here.

Validation gate (pending):
  M2 mini PPL/decode comparison @ Qwen2.5-7B-Q8_0 K=q8_0/V=turbo4 asymmetric
  ctx 2048 + 16384 — see PR body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai

…ers (ggml-org#21245)

* model : refactor QKV into common build_qkv and create_tensor_qkv helpers

* model : extend build_qkv to bert/mpt/dbrx/olmo/lfm2/nemotron-h/granite-hybrid/gemma3n-iswa/t5-dec and fix wqkv_s