Summary

Fix ggml_mul_mat assertion failure (ggml_can_mul_mat(a, b)) when using --pooling mean with embedding models (e.g. bge-m3).

Regression introduced by #20340 (commit d28961d, first affected release: b8278). Same class of bug as #12517, fixed by #12545.

Root cause

The chunked fused GDN detection in sched_reserve() calls:

graph_reserve(16*n_seqs, n_seqs, n_outputs, mctx.get(), true);

where n_outputs = n_seqs. This builds the full computation graph including build_pooling(). For embedding models with mean pooling:

• build_inp_mean() creates a tensor with shape [n_tokens = 16*n_seqs, n_seqs_unq]
• res->t_embd is reduced to [n_outputs = n_seqs, ...] via ggml_get_rows(cur, inp_out_ids)
• ggml_mul_mat(transpose(t_embd), inp_mean) asserts because n_seqs != 16*n_seqs

The other detection calls (Flash Attention, GDN AR) pass n_tokens=1, which triggers the rounding guard (n_tokens % n_seqs != 0 → n_outputs = max(n_outputs, n_tokens)), so they don't crash. The chunked path with n_tokens = 16*n_seqs passes the guard evenly and hits the mismatch.

Fix

Pass n_tokens as n_outputs in the chunked GDN graph reservation, matching the pattern used by the pp/tg worst-case reservations:

const uint32_t n_tokens_ch = 16*n_seqs;
auto * gf = graph_reserve(n_tokens_ch, n_seqs, n_tokens_ch, mctx.get(), true);

Also adds test_embedding_pooling_mean and test_embedding_pooling_mean_multiple to the server test suite — this pooling mode was previously untested.

Reproduction

llama-server -m bge-m3-Q8_0.gguf --pooling mean --embedding -c 8192 --parallel 2
# GGML_ASSERT(ggml_can_mul_mat(a, b)) in build_pooling

• Crashes on: b8278+ (master as of 2026-03-12)
• Works on: b8155 and earlier
• --pooling cls and --pooling last are not affected (use ggml_get_rows, not ggml_mul_mat)

Testing

Tested on AMD gfx1151 (ROCm 7.11) and CPU-only builds:

Existing server test suite: 21/21 passed. 2 new mean pooling tests added and passing locally.

AI was used in an assistive capacity (analysis and drafting). All code was manually reviewed and tested.