Skip to content

fix(aprender-train): early-return on empty input in quantize_to_gguf_bytes#1433

Merged
noahgift merged 1 commit into
mainfrom
fix/quantize-empty-data-early-return
May 3, 2026
Merged

fix(aprender-train): early-return on empty input in quantize_to_gguf_bytes#1433
noahgift merged 1 commit into
mainfrom
fix/quantize-empty-data-early-return

Conversation

@noahgift

@noahgift noahgift commented May 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Closes the empty-data contract drift surfaced as 3 failures by PR fix(aprender-train): bind quantize_to_gguf_bytes match result so --features hub builds #1432.
  • Adds an early-return for data.is_empty() so contract_pre_quantize!'s input.len() > 0 debug assertion doesn't panic on a documented edge case (test_falsify_quantize_empty_data_*: empty input → empty output, dtype preserved).
  • Net result on --features hub: 11 → 9 pre-existing test failures (the remaining 9 are F32 GGUF roundtrip byte-encoding bugs — separate root cause, separate PR).

Five Whys

  1. Why was the contract precondition tripping? Generated from a YAML contract declaring input.len() > 0.
  2. Why are tests asserting empty → empty? Real callers pass zero-length slices for skipped tensors.
  3. Why not loosen the contract? Domain restrictions are intentional — the precision postcondition is undefined on empty input.
  4. Why early-return rather than restructure the match? Less indentation; the empty case only computes the dtype tag.
  5. Why ship now? Per Toyota Way each defect gets a focused fix; this closes the empty-data class cleanly without entangling the unrelated roundtrip bug.

Test plan

  • cargo test -p aprender-train --lib --features hub gguf_writer → 23/23 pass (was 20/23)
  • cargo test -p aprender-train --lib --features hub → 9 failures (was 11; net 2 closed)
  • cargo fmt --all -- --check → no diff in touched file

Out of scope (follow-up: F32 GGUF roundtrip bug)

8 remaining failures of the form expected [5.0, 6.0, 7.0, 8.0] vs got [0.0, 5.93e-39, ...] — looks like a header-alignment / padding miscalculation in the GGUF writer pipeline. Real serialization defect, separate PR.

🤖 Generated with Claude Code

…bytes

Closes the empty-data contract drift surfaced by PR #1432: the 3
`test_falsify_quantize_empty_data_*` tests assert that empty input
must produce empty output (dtype preserved), but `contract_pre_quantize!`
asserts `input.len() > 0` and panicked on the empty case.

Resolution: handle empty input in a fast-path BEFORE the precondition
fires. The contract's domain is non-empty inputs (where the precision
bound is meaningful); empty is a documented edge case the test suite
already encodes.

Five Whys:
1. Why was the contract precondition tripping? `contract_pre_quantize!`
   is generated from a YAML contract that declares `input.len() > 0` as
   a domain restriction.
2. Why are tests then asserting empty input → empty output? Because real
   callers pass zero-length slices for skipped tensors / no-op cases;
   panic-on-empty would force every caller to defensively check len().
3. Why not loosen the contract? Domain restrictions are intentional —
   the precision bound (the postcondition) is undefined on empty inputs.
   Loosening the precondition without a thoughtful adjustment to the
   bound's meaning would weaken the math.
4. Why early-return rather than `if data.is_empty() { ... } else { ... }`
   inside the match? Less indentation; the empty path doesn't compute
   anything quantization-mode-specific other than the dtype tag, so a
   tiny early-return is cleaner than a branch around the whole body.
5. Why ship now? PR #1432 surfaced this as one of 11 pre-existing test
   failures with --features hub. Per Toyota Way each defect gets its own
   focused fix; this PR closes the 3 empty-data tests cleanly. The
   remaining 8 (F32 GGUF roundtrip byte-encoding bugs) are tracked for
   follow-up — they're a real serialization defect, separate root cause.

Verified locally:
- `cargo test -p aprender-train --lib --features hub gguf_writer` →
    23/23 pass (was 20/23: 3 empty-data tests now green)
- `cargo test -p aprender-train --lib --features hub` →
    7977 passed / 9 failed (was 7975 / 11). Net 2 fewer failures.
- `cargo fmt --all -- --check` → no diff in touched file

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@noahgift noahgift merged commit a4d4baf into main May 3, 2026
11 checks passed
@noahgift noahgift deleted the fix/quantize-empty-data-early-return branch May 3, 2026 18:08
noahgift added a commit that referenced this pull request May 3, 2026
…n test helpers (#1434)

* fix(aprender-train): early-return on empty input in quantize_to_gguf_bytes

Closes the empty-data contract drift surfaced by PR #1432: the 3
`test_falsify_quantize_empty_data_*` tests assert that empty input
must produce empty output (dtype preserved), but `contract_pre_quantize!`
asserts `input.len() > 0` and panicked on the empty case.

Resolution: handle empty input in a fast-path BEFORE the precondition
fires. The contract's domain is non-empty inputs (where the precision
bound is meaningful); empty is a documented edge case the test suite
already encodes.

Five Whys:
1. Why was the contract precondition tripping? `contract_pre_quantize!`
   is generated from a YAML contract that declares `input.len() > 0` as
   a domain restriction.
2. Why are tests then asserting empty input → empty output? Because real
   callers pass zero-length slices for skipped tensors / no-op cases;
   panic-on-empty would force every caller to defensively check len().
3. Why not loosen the contract? Domain restrictions are intentional —
   the precision bound (the postcondition) is undefined on empty inputs.
   Loosening the precondition without a thoughtful adjustment to the
   bound's meaning would weaken the math.
4. Why early-return rather than `if data.is_empty() { ... } else { ... }`
   inside the match? Less indentation; the empty path doesn't compute
   anything quantization-mode-specific other than the dtype tag, so a
   tiny early-return is cleaner than a branch around the whole body.
5. Why ship now? PR #1432 surfaced this as one of 11 pre-existing test
   failures with --features hub. Per Toyota Way each defect gets its own
   focused fix; this PR closes the 3 empty-data tests cleanly. The
   remaining 8 (F32 GGUF roundtrip byte-encoding bugs) are tracked for
   follow-up — they're a real serialization defect, separate root cause.

Verified locally:
- `cargo test -p aprender-train --lib --features hub gguf_writer` →
    23/23 pass (was 20/23: 3 empty-data tests now green)
- `cargo test -p aprender-train --lib --features hub` →
    7977 passed / 9 failed (was 7975 / 11). Net 2 fewer failures.
- `cargo fmt --all -- --check` → no diff in touched file

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(aprender-train): account for GGUF tensor-data alignment padding in test helpers

Closes the F32 GGUF roundtrip drift surfaced by PR #1432: 8 tests failing
with `expected [5.0, 6.0, 7.0, 8.0]` vs `got [0.0, 5.93e-39, ...]` plus 1
inline equivalent in `pipeline.rs`.

Root cause: `aprender::format::gguf::export_tensors_to_gguf` (types.rs:445)
writes `padding_for_alignment(header_size, GGUF_DEFAULT_ALIGNMENT)` zero
bytes between the tensor-info section and the tensor data so the data
section starts at a 32-byte-aligned offset. Two test helpers — the shared
`find_data_section_start` in `gguf_verify/tests/mod.rs` and an inline
clone in `pipeline.rs::test_falsify_pipeline_f32_data_integrity_through_pipeline`
— skipped header + metadata + tensor info but NOT the trailing alignment
padding, then read f32 bytes from the wrong offset. The "wrong offset"
landed in the zero-padding region, which produced the characteristic
`[0.0, ~5.93e-39, ~5.95e-39, ...]` pattern (the trailing values are the
first 12 bytes of actual tensor data, just shifted by `padding` bytes
and reinterpreted as f32 from the offset boundary).

Fix: add `pos += padding_for_alignment(pos, GGUF_DEFAULT_ALIGNMENT)` in
both helpers to mirror the writer's behavior.

Five Whys:
1. Why was the data section misaligned? Helpers had a comment claiming
   "NO alignment padding" — incorrect; the writer DOES pad (types.rs:445).
2. Why didn't anyone notice before? `--features hub` was unbuildable on
   main due to PR #1432's syntactic bug; tests couldn't run.
3. Why are there two near-identical helpers (test/mod.rs + pipeline.rs)? A
   refactor extracted `find_data_section_start` for reuse but missed the
   inline copy in `pipeline.rs`. Drift between the two means a fix to one
   is incomplete. This PR fixes both. Follow-up: collapse the inline copy
   to a call into the shared helper.
4. Why use a hardcoded `pos += padding_for_alignment(...)` rather than a
   higher-level "find data section start" abstraction? Bounded scope —
   each fix is a single line + a contextual import. Refactoring the test
   harness is separate work.
5. Why ship now? Per Toyota Way each defect gets a focused fix; this
   single root cause closes all 9 remaining surfaced failures (8 in
   gguf_verify + 1 in pipeline). `--features hub` test pass rate goes
   from 7977/7986 → 7986/7986 (full green).

Verified locally:
- `cargo test -p aprender-train --lib --features hub gguf_verify` → 72/72 pass
- `cargo test -p aprender-train --lib --features hub` → 7986 pass / 0 fail / 16 ignored
- `cargo fmt --all -- --check` → no diff in touched files

Net `--features hub` health across today's session:
  - Pre-#1432: build error (could not run any test)
  - Post-#1432: 7975/7986 pass (build works, 11 pre-existing failures surfaced)
  - Post-#1433: 7977/7986 pass (3 empty-data tests fixed, 9 left)
  - Post-this: 7986/7986 pass — full green ✅

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
noahgift added a commit that referenced this pull request May 3, 2026
…line distill falsifier-parity (#1437)

Canonical record of today's MODEL-2-side hygiene cycle (PRs #1432-#1436).
Pre-cycle: --features hub unbuildable (E0425 in quantize_to_gguf_bytes),
masking 11 pre-existing test failures.

Chain landed:
- #1432: bind match result; build works → 7975/7986 (11 pre-existing surfaced)
- #1433: empty-input early-return → 7977/7986 (3 contract-drift fixed)
- #1434: GGUF alignment-padding skip in 2 test helpers → 7986/7986 ✅
- #1435: WGPU_FALLBACK_LOG_PREFIX + 3 drift-prevention tests (matches v1.2.0)
- #1436: 4 hf_pipeline distill falsifier-parity tests → 7990/7990

§42 documents: what landed (table), net hub health progression, why for
MODEL-2 (parallel-impl drift-protection), Five Whys (build masked
pre-existing failures), coverage update (15+33 unchanged; parallel-impl
uplift), ship % effects (MODEL-1 87→88, MODEL-2 50→54), and
next-session pickup options (CPU-GPU-005 part b OR distill-train
precompute determinism).

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant