v0.0.130 — Pair-type closure captures preserve len field (#535, residual of #514)#569
Conversation
…ual of #514) Closes #535. v0.0.121 fixed nested closures and primitive/ADT captures; this release closes the residual pair-type subset where String / Array<T> captures silently dropped their len field. ## Root cause `vera/wasm/closures.py::_walk_free_vars` resolved each capture's wasm type via `_type_name_to_wasm`, which collapses every composite type to a single "i32" (handler call sites depend on this single-slot return). For pair-typed values, the value lives on the stack as TWO i32 slots (ptr + len). The pre-fix path: - allocated 4 bytes per pair capture instead of 8 - stored only the ptr (the value at `local.get {local_idx}`) - read back only the ptr in the lifted body - len was read as garbage from adjacent struct memory (typically zero) Result: `array_length(@array<Int>.0)` and `string_length(@String.0)` of captured pairs always returned 0; any operation on the captured value saw it as empty. ## Fix (3 sites in lockstep) 1. `vera/wasm/closures.py::_walk_free_vars` — when the resolved type is String / Array / Array<...>, override the wasm tag to "i32_pair". `_type_name_to_wasm` is left alone so handler call sites (handle_state, handle_exn) still get the single-slot return they expect. 2. `vera/wasm/closures.py::_translate_anon_fn` — pair-aware closure struct layout (8 bytes, 4-aligned, two consecutive i32 fields) and store (i32.store offset=N for ptr, i32.store offset=N+4 for len, reading from `local_idx` and `local_idx + 1` respectively). 3. `vera/codegen/closures.py::_compile_lifted_closure` — mirror the layout on read. Allocate two consecutive i32 locals (ptr, len), load both halves from the env struct, and push only the ptr into the SlotEnv (matching the let-binding and parameter convention). Sanity assert that the two locals are consecutive. 4. GC handling: extend the captured-pointer shadow_push loop to handle pair captures (push the ptr local; the len is a byte count, not a heap pointer). ## Tests `TestPairCapture535` (5 cases): - test_array_capture_length_in_closure (returns 21 not 0) - test_string_capture_length_in_closure (returns 15 not 0) - test_adt_capture_still_works (regression pin: 126) - test_primitive_capture_still_works (regression pin: 21) - test_mixed_pair_and_primitive_capture (layout-order exercise: 312) 3,680 -> 3,685 tests. mypy clean, 82 conformance programs pass, 33 examples pass. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughv0.0.130 fixes issue ChangesPair-Type Closure Capture Fix
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #569 +/- ##
==========================================
+ Coverage 90.94% 90.96% +0.01%
==========================================
Files 59 59
Lines 22552 22588 +36
Branches 259 259
==========================================
+ Hits 20511 20547 +36
Misses 2034 2034
Partials 7 7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
CI lint job's "Security lint (ruff S rules)" step failed because vera/codegen/closures.py used a bare assert for the consecutive-locals invariant. Ruff S101 forbids assert in production code since it gets stripped under python -O. Converted to an explicit if ... raise RuntimeError so the check survives optimization. Behavior is unchanged in normal runs. Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/test_codegen.py (1)
14890-14997:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd zero-length pair-capture edge cases (
""and[]).Nice regression coverage for non-empty pair captures, but the fix is specifically about preserving the
lenslot. Please add explicit emptyStringand emptyArraycapture-in-closure tests to pinlen == 0behaviour as well.Suggested tests
class TestPairCapture535: + def test_empty_array_capture_length_in_closure(self) -> None: + src = """ +public fn main(`@Unit` -> `@Int`) + requires(true) ensures(true) effects(pure) +{ + let `@Array`<Int> = []; + let `@Array`<Int> = array_map( + array_range(0, 3), + fn(`@Int` -> `@Int`) effects(pure) { + nat_to_int(array_length(`@Array`<Int>.0)) + } + ); + array_fold(`@Array`<Int>.0, 0, fn(`@Int`, `@Int` -> `@Int`) effects(pure) { `@Int.0` + `@Int.1` }) +} +""" + assert _run(src) == 0 + + def test_empty_string_capture_length_in_closure(self) -> None: + src = """ +public fn main(`@Unit` -> `@Int`) + requires(true) ensures(true) effects(pure) +{ + let `@String` = ""; + let `@Array`<Int> = array_map( + array_range(0, 3), + fn(`@Int` -> `@Int`) effects(pure) { + nat_to_int(string_length(`@String.0`)) + } + ); + array_fold(`@Array`<Int>.0, 0, fn(`@Int`, `@Int` -> `@Int`) effects(pure) { `@Int.0` + `@Int.1` }) +} +""" + assert _run(src) == 0🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_codegen.py` around lines 14890 - 14997, Add two tests that capture empty pair-layout values to ensure the preserved `len == 0` behavior: add a test named like test_empty_string_capture_in_closure which sets let `@String` = "" and uses array_map + string_length(`@String.0`) then asserts result == 0 (3 iterations → 0), and a test named like test_empty_array_capture_in_closure which sets let `@Array`<Int> = array_range(0,0) or let `@Array`<Int> = [] then uses array_map + nat_to_int(array_length(`@Array`<Int>.0)) and asserts result == 0; follow the existing test pattern (use _run(src) assertions, array_map/array_fold structure) so the coverage for the captured `len` slot (in functions like _translate_anon_fn / _compile_lifted_closure) includes zero-length pair captures.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@vera/codegen/closures.py`:
- Around line 308-316: The loop that pushes captured locals using cap_locals /
cap_local_kinds via gc_shadow_push is currently added into gc_prologue before
load_instrs, so pushes occur while cap_local still holds default 0; move the
capture-root pushes to run after the env-loads are emitted (i.e., after
load_instrs are generated) so captured heap pointers are rooted only after they
are loaded from $env; you can either relocate the loop that iterates over
cap_locals/cap_local_kinds to after load_instrs creation or collect those push
instructions into a separate list (e.g., gc_prologue_after_loads) and append it
to gc_prologue only after load_instrs, ensuring gc_shadow_push is called when
cap_local contains the loaded value.
---
Outside diff comments:
In `@tests/test_codegen.py`:
- Around line 14890-14997: Add two tests that capture empty pair-layout values
to ensure the preserved `len == 0` behavior: add a test named like
test_empty_string_capture_in_closure which sets let `@String` = "" and uses
array_map + string_length(`@String.0`) then asserts result == 0 (3 iterations →
0), and a test named like test_empty_array_capture_in_closure which sets let
`@Array`<Int> = array_range(0,0) or let `@Array`<Int> = [] then uses array_map +
nat_to_int(array_length(`@Array`<Int>.0)) and asserts result == 0; follow the
existing test pattern (use _run(src) assertions, array_map/array_fold structure)
so the coverage for the captured `len` slot (in functions like
_translate_anon_fn / _compile_lifted_closure) includes zero-length pair
captures.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c9068144-acac-4d53-9dce-615d281f5224
⛔ Files ignored due to path filters (5)
docs/index.htmlis excluded by!docs/**docs/index.mdis excluded by!docs/**docs/llms-full.txtis excluded by!docs/**docs/llms.txtis excluded by!docs/**uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (11)
CHANGELOG.mdHISTORY.mdKNOWN_ISSUES.mdREADME.mdROADMAP.mdTESTING.mdpyproject.tomltests/test_codegen.pyvera/__init__.pyvera/codegen/closures.pyvera/wasm/closures.py
💤 Files with no reviewable changes (1)
- KNOWN_ISSUES.md
Inline (1):
1. vera/codegen/closures.py 308-316 — capture-root GC pushes were
emitted into gc_prologue (before load_instrs), but the env-loads
hadn't run yet at that point so cap_local still held the default
0; gc_shadow_push snapshots the local's CURRENT value, so a zero
was being written to the shadow stack instead of the loaded ptr.
Captured heap pointers reachable only through the closure could
be GC'd while in use.
Pre-existing for i32 captures (PR #569 just extended the
pattern to i32_pair). Fix: split the prologue into two phases —
gc_prologue (sp save + param pushes, runs before load_instrs)
and gc_capture_pushes (capture roots, runs AFTER load_instrs).
Assembly order now: locals decl, sp save + param pushes, env
loads, capture roots, body, epilogue.
Outside-of-diff (1):
2. tests/test_codegen.py — added two edge-case tests:
test_empty_string_capture_in_closure (empty String captured)
test_empty_array_capture_in_closure (empty Array<Int> captured)
Both assert _run(src) == 0. Pin that the post-fix path correctly
preserves a zero-length pair capture (rather than reading
garbage from an unallocated len slot in the closure struct).
Test count: 3,685 → 3,687.
ruff S clean, mypy clean, 82 conformance programs pass, 33 examples pass.
Co-Authored-By: Claude <noreply@anthropic.invalid>
|
@coderabbitai Closing out the outside-of-diff finding from this round: tests/test_codegen.py 14890-14997 — add empty-pair-capture tests ✅ Added in 4b92e42:
Both follow the existing Test count: 3,685 → 3,687. ruff S clean, mypy clean, 82 conformance programs pass, 33 examples pass. |
|
Tip For best results, initiate chat on the files or code changes.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_codegen.py`:
- Around line 14890-15047: Add a new test (e.g. test_gc_pressure_pair_capture)
modeled on
test_string_capture_length_in_closure/test_array_capture_length_in_closure that
captures a `@String` or `@Array` in the outer scope and, inside the anon fn passed
to array_map, performs heavy allocation (e.g. create many temporary
arrays/strings or call array_range/map repeatedly) before calling
string_length(`@String.0`) or array_length(`@Array`<Int>.0); ensure the final folded
sum equals the expected value (e.g. 5×3 for "hello" or 7×3 for array_range(0,7))
so the test exercises the GC-pressure pair-root shadow-push path while the
captured pair is live. Ensure the new test name and body follow the existing
test style and use _run(src) for validation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 417bd6a2-3924-49a1-a4ae-6a4b9ec6dc55
📒 Files selected for processing (6)
HISTORY.mdREADME.mdROADMAP.mdTESTING.mdtests/test_codegen.pyvera/codegen/closures.py
Inline (1):
tests/test_codegen.py 14890-15047 — add tests that exercise the
round-1 GC-ordering fix (gc_capture_pushes after load_instrs)
under in-closure allocation pressure.
Two tests added under TestPairCapture535:
test_gc_pressure_pair_capture
Captures @array<Int> = array_range(0, 7) (length 7). Inside the
closure body, allocates 3 × array_range(0, 500) = ~12 KB of temp
arrays BEFORE reading array_length(@array<Int>.3) (.3 because
the captured array sits at outer-scope index 3, after the three
inner let-bindings at 0, 1, 2). Asserts folded sum = 21 (3 × 7).
test_gc_pressure_string_capture
Same shape with @string = "hello"; asserts 15 (3 × 5).
If a future change re-broke the GC ordering (capture pushes back
in gc_prologue before load_instrs), these tests would catch it
when in-body allocations cross the GC threshold and a $gc_collect
sweeps the captured pair as unreachable.
Test count: 3,687 → 3,689.
ruff S clean, mypy clean, full pytest passes.
Co-Authored-By: Claude <noreply@anthropic.invalid>
Summary
String,Array<T>) now preserve their len field. Residual of Closures capturing heap-allocated values produce invalid WASM (nested-closure case is a narrow instance) #514 (v0.0.121 fixed nested closures + primitive/ADT captures; this closes the pair-type subset).vera/wasm/closures.pyandvera/codegen/closures.py. No grammar/parser/type-checker changes.TestPairCapture535. Test count 3,680 → 3,685.Root cause
_walk_free_varsresolved each capture's wasm type via_type_name_to_wasm, which collapses every composite type to a single"i32"(handler call sites —handle_state,handle_exn— depend on this single-slot return). For pair-typed values, the runtime stack representation is two i32 slots (ptr + len). Pre-fix:So
array_length(@Array<Int>.0)/string_length(@String.0)of any captured pair always returned 0; any operation on the captured value saw it as empty.Fix (3 sites)
vera/wasm/closures.py::_walk_free_vars— when the resolved type isString/Array/Array<...>, override the wasm tag to"i32_pair"._type_name_to_wasmis left alone so handler call sites still get their single-slot return.vera/wasm/closures.py::_translate_anon_fn— pair-aware closure struct layout (8 bytes, 4-aligned, two consecutive i32 fields). Store:i32.store offset=Nfor ptr,i32.store offset=N+4for len, reading fromlocal_idxandlocal_idx + 1(the let-binding / parameter pair convention).vera/codegen/closures.py::_compile_lifted_closure— mirror the layout on read. Allocate two consecutive i32 locals (ptr, len), load both halves from the env struct, and push only the ptr into theWasmSlotEnv(matching the let-binding and parameter convention so the closure body resolves the pair correctly). Sanity-assert that the two locals are consecutive.GC — extend the captured-pointer shadow_push loop to handle pair captures (push the ptr local; len is a byte count, never a heap pointer).
Tests added
test_array_capture_length_in_closuretest_string_capture_length_in_closuretest_adt_capture_still_workstest_primitive_capture_still_workstest_mixed_pair_and_primitive_captureThe mixed-capture test is the most interesting: it captures both a primitive
Int(i64, 8 bytes, 8-aligned) and a pairArray<Int>(i32_pair, 8 bytes, 4-aligned) in the same closure, exercising that_translate_anon_fnand_compile_lifted_closureagree on the field-packing order.Test plan
mypy vera/cleanCloses #535
Summary by CodeRabbit
Bug Fixes
Testing
Documentation
Chores