Summary

• val_bpb: 0.9300 (3-seed mean, std 0.0006)
• Artifact: ~15.6 MB (all seeds < 16MB)
• Training: 600s on 8xH100 SXM | Eval: ~297s (SLOT)

Novel Mechanisms

• Scored-position SLOT mask — delta training aligned to eval scoring positions (last stride=64 per window)
• Per-sample delta [bsz,1,512] instead of shared [1,1,512]
• Logit bias [bsz,1,vocab] for direct logit-space adaptation
• Training-data GPTQ calibration — 256 batches real data instead of AR self-gen
• Cosine LR schedule — 0.008→0.0008 over 16 AdamW steps

Credits

• Base: PR Record: AR Self-Gen GPTQ + XSA-all + BigramHash 3072×112 — val_bpb 1.11473 (3-seed mean) #1019 by @abaybektursun
• SLOT mechanism: arXiv:2505.12392v2, PR Record: SLOT + Split-LR + Full GPTQ + XSA-all — val_bpb 1.1015 (3-seed mean) #1172 / PR Record: QK-Gain 4.0 + XSA-11 + Muon-TTT + SLOT — val_bpb 1.0914 (3-seed mean) #1176
• Sigmoid-gated skips, Brotli: PR Record: SLOT + Split-LR + Full GPTQ + XSA-all — val_bpb 1.1015 (3-seed mean) #1172
• QK-Gain 4.0: PR Non-record: XSA-All + QK Gain 4.0 + LN Scale — 45 Experiments on 1×RTX 5090 #1125 / PR Record: QK-Gain 4.0 + XSA-11 + Muon-TTT + SLOT — val_bpb 1.0914 (3-seed mean) #1176

3-Seed Results

Beats merged SOTA (1.1194) by 0.189. Clears 0.005 nats threshold by 38x.

Compliance

• Score-first SLOT (frozen model, torch.no_grad hidden states, causal shift)
• Self-contained (zero env var overrides)
• All seeds within time and size budgets