docs(evidence): qwen2-0.5b bisection — root cause via apr diff (LAYOUT-001/002 violation)#1466
Closed
noahgift wants to merge 1 commit into
Closed
docs(evidence): qwen2-0.5b bisection — root cause via apr diff (LAYOUT-001/002 violation)#1466noahgift wants to merge 1 commit into
noahgift wants to merge 1 commit into
Conversation
…diff Captures the empirical root-cause finding for Qwen2-0.5B-Instruct gibberish: LAYOUT-001/002 contract violation in safetensors→APR FFN tensor import. `apr diff <0.5b.apr> <0.5b.gguf> --values --limit 5` outputs literally: DIAGNOSIS: Values identical, shapes transposed (format layout diff) Evidence: - gguf-trace-coherent-logits.json — `apr trace` on same model converted to GGUF produces coherent top-5 logits (`<|im_end|>` 10.54, " The" 10.47, " For" 10.18, " Each" 10.15) — proves model weights are correct. - findings.md — Five Whys, falsified hypotheses (4), trustworthy facts (3), next-session fix scope (~50 LOC in safetensors→APR FFN import path). Methodology lesson: this investigation burned ~15 turns on falsified hypotheses (shape orientation, dtype case, F16 dispatch) before running `apr diff`. CLAUDE.md mandates "use apr tools first" — internalized as: when investigating any model output defect, run `apr diff` and `apr qa --verbose` BEFORE any code reading. No code changes in this PR — fix scope is bounded and isolated to a follow-up. This commit only ships evidence + memory of the diagnosis so the work survives session boundaries. Related: - LAYOUT-001/002 (CLAUDE.md, contracts/tensor-layout-v1.yaml) - tied-embeddings-v1 phase `transpose_embed` - SHIP-007 cascade (apr-vs-gguf-forward-parity-v1) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 4, 2026
Contributor
Author
|
Closing — findings.md interpretation is wrong (asserts LAYOUT-001/002 violation but the contract documents that 'TRANSPOSED' shape labels between GGUF and APR are contract-specified behavior, not a defect). The empirical evidence (apr diff output, gguf-trace JSON) is real and valid; only the interpretation needs re-authoring once the actual root cause is found. The evidence files remain on branch docs/qwen2-0.5b-bisection-evidence for future re-investigation. Six hypotheses falsified this session — investigation needs clean restart with proper instrumentation. |
auto-merge was automatically disabled
May 4, 2026 12:14
Pull request was closed
4 tasks
noahgift
added a commit
that referenced
this pull request
May 4, 2026
…oupling finding (#1472) Adds §50 documenting the architecture-mismatch finding caught after §49.6 steps 3+4 landed (PR #1470 contract + PR #1471 wire-up). The remaining §49.6 step 5 was scoped at "0 LOC, just run apr pretrain --init" — that assumption is empirically wrong. Empirical finding (§50.1): pretrain_real.rs:38-46 HARDCODES Llama370MConfig::* for every architectural constant. Qwen2.5-Coder-0.5B-Instruct has different shape across the board: Param | Llama370M | Qwen2.5-Coder-0.5B -----------------|-----------|-------------------- hidden_size | 1024 | 896 num_attention_heads | 16 | 14 num_kv_heads | 4 (GQA-4:1) | 2 (GQA-7:1) intermediate_size | 2816 | 4864 vocab_size | 50_257 | 151_936 rope_theta | 10_000 | 1_000_000 Every tensor mismatches. Loading Qwen2.5 weights into a Llama370M- shaped optimizer is a category error. Three options surfaced (§50.3): A: Find/build a Llama-shaped 0.5B pretrained checkpoint (~5K LOC + multi-week training; recreates §24/§25 corpus problem) B: Make trainer architecture-polymorphic (~200-400 LOC; preserves §24/§25 falsification; recommended) C: Replace Llama370MConfig with Qwen2_5_Coder_0_5B_Config outright (~300 LOC; deletes a working falsification path) Recommendation (§50.5): Option B — preserves §24/§25 falsification evidence, exercises TransformerConfig's designed polymorphism, binds each new component (qwen2_0_5b constructor, GQA-7:1 attention, Qwen tokenizer surface) to its own falsifier. Re-scoped roadmap (§50.4) — 8 sub-steps replacing original step 5: 5a. Author apr-pretrain-arch-polymorphic-v1.yaml contract (~80 LOC) 5b. TransformerConfig::qwen2_0_5b() constructor (~40 LOC) 5c. Extract arch from init APR file metadata (~80 LOC) 5d. Qwen tokenizer-vocab compatibility check (~30 LOC) 5e. GQA-7:1 attention forward-pass verification (~50 LOC) 5f. Wire actual weight load (~120 LOC) 5g. LIVE 500-step smoke fine-tune (operator dispatch) 0 LOC 5h. Stamp + publish as MODEL-2 v2 (~10 LOC) Total: ~410 LOC + 1 LIVE training run. Five Whys (§50.6): 1. Why didn't §49 catch this? §49 was authored from strategy/ data-budget reasoning; the 0-LOC step-5 cost implicitly assumed polymorphism. Live source inspection (this section's empirical move) revealed pretrain_real.rs:38-46 predates the assumption. 2. Why catch this NOW and not in step 5 implementation? Per feedback_no_guessing.md: read live source before forming implementation plan. Surfacing the mismatch BEFORE writing 200 LOC of weight-load code that fails at runtime is the cheapest place to pay cost-of-defect. The §50-prior wrong- premise PRs (#1466/#1467/#1468 closed) on the SHIP-007 / 0.5B gibberish track were the same defect class. 3. Why option B over A or C? Preserves §24/§25 falsification evidence (we KEEP knowing from-scratch fails at 9.75; we just don't ship it as MODEL-2). Exercises the polymorphism TransformerConfig was designed for. Each new component becomes its own falsifier rather than a hidden coupling. 4. Why is FALSIFY-005 the right place to fail-fast? PR #1470 already pinned "Architecture mismatch is FAIL-FAST, not silent- truncate". Step 4 (PR #1471) doesn't enforce arch matching yet — returns "not yet wired" before getting there. So FALSIFY-005 is currently UNBOUND but its discharge gate is well-defined: read APR header, compare against pretrain target, error with names of mismatched fields. 5. Why isn't this a "punt"? A punt would say "blocked, await operator". This amendment names three options with LOC estimates, recommends one with reasoning, gives a concrete 8- step roadmap with falsifier discharge mapped to each sub-step. The work IS shippable; it's just bigger than 0 LOC. Plain ship-% update: - MODEL-1: unchanged at 91% (SHIP-007 cascade infrastructure track) - MODEL-2: unchanged at 57% — first ship-% movement gated on §50.4 step 5g (LIVE 500-step fine-tune producing val_loss < 9.38). Sub-steps 5a-5f can each individually move 1% with falsifier discharge (architecture-polymorphic infrastructure shipped == evidence that the §49 path is REACHABLE, not just theoretical). Refs: - §49 — MODEL-2 strategy pivot (PR #1461) - PR #1470 — apr-pretrain-from-init-v1 v1.0.0 PROPOSED contract - PR #1471 — apr pretrain --init clap field + magic-byte validate - feedback_no_guessing.md — read source before forming hypothesis - feedback_fix_root_cause_never_route_around.md Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Empirical root cause for Qwen2-0.5B-Instruct gibberish, found via existing `apr diff` tool. No code changes — this PR only ships evidence + findings so the work survives session boundaries. The actual fix is a separate follow-up PR with bounded scope (~50 LOC in safetensors→APR FFN import path).
The diagnosis line literally says it
`apr diff <0.5b.apr> <0.5b.gguf> --values --limit 5` outputs:
```
DIAGNOSIS: Values identical, shapes transposed (format layout diff)
```
LAYOUT-001/002 contract violation in safetensors→APR FFN import (CLAUDE.md: "this bug has occurred 100+ times").
Why 7B works, 0.5B fails
What's in this PR
Methodology lesson
Burned ~15 turns on falsified hypotheses (shape orientation, dtype case, F16 dispatch) before running `apr diff`. CLAUDE.md "use apr tools first" is now internalized as: `apr diff` and `apr qa --verbose` come BEFORE any code reading.
Test plan
🤖 Generated with Claude Code