My Computer:

• Framework 13 /w Ryzen AI 9 HX 370 (12c/24t)
• 128gb RAM (2x64gb DDR5 5600 SODIMM)
• 8tb SSD (WD_BLACK SN850X), /w BTRFS ZSTD:5 global compression
• 48gb ZRAM device
• Arch Linux /w CachyOS-influenced performance optimisations

My benchmark:

~/Development/colibri/c main !2 ?1 ❯ ./coli run "Explain how a hash map works." --model /home/mateogrgic/Development/colibri/c/GLM-5.2-colibri-int4 --ram 110 --ngen 150

 ▄▀▀▀▄  ▄        colibrì v1.0

▄▄▄▄▀▀▀▀▄▀▀ piccolo motore, modello immenso
▀▀▀▀▀▀▀ GLM-5.2 · 744B MoE · int4 · streaming CPU
▀▀▀▀ run
▀
──────────────────────────────────────────────────────────
== Motore C GLM (glm_moe_dsa), cache=8 expert/layer | expert@8-bit densa@8-bit | idot: avx512-vnni ==
caricato in 4.18s | densa residente: 9780.06 MB | layers=78 experts=256 | MTP ATTIVA (draft=3)
[MTP] attiva: decodifica speculativa nativa (draft=3)
[RAM_GB=110.0] cap=8 ok (proiezione picco 28.0 GB)
[stop] 3 token di stop: 154820 154827 154829
prompt: 14 token | genero fino a 150 (stop EOS=154820) | draft n-gram=3
[gMASK]<|user|>Explain how a hash map works.<|assistant|>[prefill] layer 1/78 · 14 token
[prefill] layer 5/78 · 14 token
[prefill] layer 9/78 · 14 token
[prefill] layer 13/78 · 14 token
[prefill] layer 17/78 · 14 token
[prefill] layer 21/78 · 14 token
[prefill] layer 25/78 · 14 token
[prefill] layer 29/78 · 14 token
[prefill] layer 33/78 · 14 token
[prefill] layer 37/78 · 14 token
[prefill] layer 41/78 · 14 token
[prefill] layer 45/78 · 14 token
[prefill] layer 49/78 · 14 token
[prefill] layer 53/78 · 14 token
[prefill] layer 57/78 · 14 token
[prefill] layer 61/78 · 14 token
[prefill] layer 65/78 · 14 token
[prefill] layer 69/78 · 14 token
[prefill] layer 73/78 · 14 token
[prefill] layer 77/78 · 14 token
[prefill] layer 78/78 · 14 token
At its core, a hash map[MTP] acceptance 0% dopo 24 proposte: draft disattivati
(also known as a hash table
[t=16 RSS 21.97 GB hit 11% 0.14 tok/s 1.00 tok/fw]
, dictionary, or associative array) is a data structure used to store information in
[t=32 RSS 21.97 GB hit 19% 0.21 tok/s 1.00 tok/fw]
key-value pairs. You give the hash map a "key" (
[t=48 RSS 21.97 GB hit 22% 0.24 tok/s 1.00 tok/fw]
like a word you are looking up in a dictionary), and it instantly returns the
[t=64 RSS 21.98 GB hit 24% 0.26 tok/s 1.00 tok/fw]
"value" associated with that key (like the definition of that word).

To
[t=80 RSS 21.98 GB hit 25% 0.27 tok/s 1.00 tok/fw]
understand how it works, let's break it down into four simple steps: The
[t=96 RSS 21.98 GB hit 25% 0.27 tok/s 1.00 tok/fw]
Magic Box, The Math Behind the Magic, Handling Collisions, and Why They
[t=112 RSS 21.98 GB hit 26% 0.28 tok/s 1.00 tok/fw]
Are Useful.

1. The Underlying Structure (The "Magic

[t=128 RSS 21.99 GB hit 27% 0.29 tok/s 1.00 tok/fw]
Box")
Imagine a giant, empty bookshelf with exactly 10 numbered slots.
[t=144 RSS 21.99 GB hit 28% 0.29 tok/s 1.00 tok/fw]

• You want to store

150 token in 513.99s (0.29 tok/s) | hit-rate expert 28.0% | RSS 21.99 GB
expert caricati/token: 750.0 (per-layer 10.00 su 75; baseline topk=8) | TOPK=0 TOPP=0.00
speculazione: 1.01 token/forward (149 fw per 150 tok) | MTP acceptance 0% (0/24)
PROFILO: expert-disk 221.9s | expert-matmul 235.3s | attention 36.6s (di cui kvb 0.1s) | lm_head 0.0s | altro 20.2s
~/Development/colibri/c main !2 ?1 ❯ 8m 38s  26.4.0 19:26:49