Skip to content

v0.0.130 — Pair-type closure captures preserve len field (#535, residual of #514)#569

Merged
aallan merged 4 commits into
mainfrom
claude/fix-535-pair-capture
May 5, 2026
Merged

v0.0.130 — Pair-type closure captures preserve len field (#535, residual of #514)#569
aallan merged 4 commits into
mainfrom
claude/fix-535-pair-capture

Conversation

@aallan

@aallan aallan commented May 5, 2026

Copy link
Copy Markdown
Owner

Summary

Root cause

_walk_free_vars resolved each capture's wasm type via _type_name_to_wasm, which collapses every composite type to a single "i32" (handler call sites — handle_state, handle_exn — depend on this single-slot return). For pair-typed values, the runtime stack representation is two i32 slots (ptr + len). Pre-fix:

  • 4 bytes allocated per pair capture instead of 8
  • Only the ptr was stored
  • The lifted body read only the ptr; the len was read as garbage from adjacent struct memory (typically zero)

So array_length(@Array<Int>.0) / string_length(@String.0) of any captured pair always returned 0; any operation on the captured value saw it as empty.

Fix (3 sites)

  1. vera/wasm/closures.py::_walk_free_vars — when the resolved type is String / Array / Array<...>, override the wasm tag to "i32_pair". _type_name_to_wasm is left alone so handler call sites still get their single-slot return.

  2. vera/wasm/closures.py::_translate_anon_fn — pair-aware closure struct layout (8 bytes, 4-aligned, two consecutive i32 fields). Store: i32.store offset=N for ptr, i32.store offset=N+4 for len, reading from local_idx and local_idx + 1 (the let-binding / parameter pair convention).

  3. vera/codegen/closures.py::_compile_lifted_closure — mirror the layout on read. Allocate two consecutive i32 locals (ptr, len), load both halves from the env struct, and push only the ptr into the WasmSlotEnv (matching the let-binding and parameter convention so the closure body resolves the pair correctly). Sanity-assert that the two locals are consecutive.

  4. GC — extend the captured-pointer shadow_push loop to handle pair captures (push the ptr local; len is a byte count, never a heap pointer).

Tests added

Test Pre-fix Post-fix
test_array_capture_length_in_closure returns 0 returns 21 (3 × len 7)
test_string_capture_length_in_closure returns 0 returns 15 (3 × len "hello")
test_adt_capture_still_works works 126 (regression pin)
test_primitive_capture_still_works works 21 (regression pin)
test_mixed_pair_and_primitive_capture partial 312 (layout-order exercise)

The mixed-capture test is the most interesting: it captures both a primitive Int (i64, 8 bytes, 8-aligned) and a pair Array<Int> (i32_pair, 8 bytes, 4-aligned) in the same closure, exercising that _translate_anon_fn and _compile_lifted_closure agree on the field-packing order.

Test plan

  • All 5 new tests pass
  • Pre-existing 3,680 tests pass (no regressions)
  • mypy vera/ clean
  • 82 conformance programs pass
  • 33 examples pass (check + verify)
  • doc-counts + version-sync + limitations-sync + license-check + site-assets all clean
  • CI green

Closes #535

Summary by CodeRabbit

  • Bug Fixes

    • Fixed closure captures for String and Array (pair-shaped) values so captured length fields are preserved.
  • Testing

    • Added comprehensive regression tests for pair-type captures, including GC-pressure scenarios.
    • Overall test count updated 3,680 → 3,689.
  • Documentation

    • Updated changelog, history, roadmap, README, known-issues and testing docs for v0.0.130.
  • Chores

    • Version bumped to v0.0.130.

…ual of #514)

Closes #535. v0.0.121 fixed nested closures and primitive/ADT captures;
this release closes the residual pair-type subset where String / Array<T>
captures silently dropped their len field.

## Root cause

`vera/wasm/closures.py::_walk_free_vars` resolved each capture's wasm
type via `_type_name_to_wasm`, which collapses every composite type to
a single "i32" (handler call sites depend on this single-slot return).
For pair-typed values, the value lives on the stack as TWO i32 slots
(ptr + len). The pre-fix path:

  - allocated 4 bytes per pair capture instead of 8
  - stored only the ptr (the value at `local.get {local_idx}`)
  - read back only the ptr in the lifted body
  - len was read as garbage from adjacent struct memory (typically zero)

Result: `array_length(@array<Int>.0)` and `string_length(@String.0)` of
captured pairs always returned 0; any operation on the captured value
saw it as empty.

## Fix (3 sites in lockstep)

1. `vera/wasm/closures.py::_walk_free_vars` — when the resolved type is
   String / Array / Array<...>, override the wasm tag to "i32_pair".
   `_type_name_to_wasm` is left alone so handler call sites
   (handle_state, handle_exn) still get the single-slot return they
   expect.

2. `vera/wasm/closures.py::_translate_anon_fn` — pair-aware closure
   struct layout (8 bytes, 4-aligned, two consecutive i32 fields) and
   store (i32.store offset=N for ptr, i32.store offset=N+4 for len,
   reading from `local_idx` and `local_idx + 1` respectively).

3. `vera/codegen/closures.py::_compile_lifted_closure` — mirror the
   layout on read. Allocate two consecutive i32 locals (ptr, len),
   load both halves from the env struct, and push only the ptr into
   the SlotEnv (matching the let-binding and parameter convention).
   Sanity assert that the two locals are consecutive.

4. GC handling: extend the captured-pointer shadow_push loop to handle
   pair captures (push the ptr local; the len is a byte count, not a
   heap pointer).

## Tests

`TestPairCapture535` (5 cases):

  - test_array_capture_length_in_closure (returns 21 not 0)
  - test_string_capture_length_in_closure (returns 15 not 0)
  - test_adt_capture_still_works (regression pin: 126)
  - test_primitive_capture_still_works (regression pin: 21)
  - test_mixed_pair_and_primitive_capture (layout-order exercise: 312)

3,680 -> 3,685 tests. mypy clean, 82 conformance programs pass,
33 examples pass.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 91202901-5e0d-4920-867a-fd6ded476b6d

📥 Commits

Reviewing files that changed from the base of the PR and between 4b92e42 and 9cb98c5.

📒 Files selected for processing (5)
  • HISTORY.md
  • README.md
  • ROADMAP.md
  • TESTING.md
  • tests/test_codegen.py

📝 Walkthrough

Walkthrough

v0.0.130 fixes issue #535: pair-type closure captures (String, Array<T>) now preserve the captured length field by treating captures as 8‑byte i32_pair (ptr + len) across capture inference, closure struct layout, serialization, environment unpacking, and GC rooting.

Changes

Pair-Type Closure Capture Fix

Layer / File(s) Summary
Capture Type Inference
vera/wasm/closures.py
Free-variable discovery maps String, Array, and Array<...> captures to "i32_pair" rather than collapsing them to a single i32; closure struct layout sizing reserves 8 bytes (ptr + len) with 4‑byte alignment for i32_pair.
Capture Serialisation
vera/wasm/closures.py
Closure allocation/serialization writes two consecutive i32 stores for i32_pair captures (ptr at cap_offset, len at cap_offset + 4), using adjacent local indices for the pair halves.
Capture Layout Computation
vera/codegen/closures.py
Capture environment layout computation accounts for i32_pair as occupying 8 bytes and produces cap_offsets for subsequent loads.
Closure Lifting & Environment Unpacking
vera/codegen/closures.py
Lifted closure unpacking allocates two consecutive i32 locals for i32_pair captures and emits two i32.loads from the environment (cap_offset and cap_offset + 4); records only the pointer local for environment wiring.
GC Root Handling
vera/codegen/closures.py
GC prologue now uses a per-capture cap_local_kinds list and emits gc_capture_pushes after env-loads so only pointer locals for i32_pair captures are rooted (len locals are skipped).
Regression Tests
tests/test_codegen.py
Added TestPairCapture535 with multiple end‑to‑end tests verifying Array<Int> and String capture length preservation, ADT/primitive/mixed capture cases and empty-pair edge cases; includes GC-pressure tests.
Version Bump
vera/__init__.py, pyproject.toml
Package version updated from 0.0.129 to 0.0.130.
Docs & Release Notes
CHANGELOG.md, HISTORY.md, KNOWN_ISSUES.md, README.md, ROADMAP.md, TESTING.md
Added v0.0.130 release notes documenting the pair-capture fix; removed the #535 bug entry; updated version references, test counts and related metrics across documentation.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • aallan/vera#536: Prior changes to closure capture/lifting logic touching the same closure-capture functionality addressed here.
  • aallan/vera#454: Modifies closure lowering and wasm closure serialization for pair-shaped (i32_pair) parameters/captures; related implementation area.

Suggested labels

compiler, tests, docs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly describes the main fix: v0.0.130 release enabling pair-type closure captures (String, Array) to preserve the length field, directly addressing issue #535.
Linked Issues check ✅ Passed All objectives from #535 are met: pair captures detected and marked as i32_pair; closure struct layout allocates 8 bytes with ptr/len at consecutive offsets; lifted closure deserialization mirrors serialization; GC rooting extended for pair captures; comprehensive regression tests added.
Out of Scope Changes check ✅ Passed All changes are scoped to pair-type capture semantics and supporting documentation/tests. No unrelated refactorings, grammar changes, or type-checker modifications introduced.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/fix-535-pair-capture

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented May 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.96%. Comparing base (2c38a7e) to head (9cb98c5).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #569      +/-   ##
==========================================
+ Coverage   90.94%   90.96%   +0.01%     
==========================================
  Files          59       59              
  Lines       22552    22588      +36     
  Branches      259      259              
==========================================
+ Hits        20511    20547      +36     
  Misses       2034     2034              
  Partials        7        7              
Flag Coverage Δ
javascript 56.83% <ø> (ø)
python 94.80% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

CI lint job's "Security lint (ruff S rules)" step failed because
vera/codegen/closures.py used a bare assert for the
consecutive-locals invariant. Ruff S101 forbids assert in
production code since it gets stripped under python -O.

Converted to an explicit if ... raise RuntimeError so the check
survives optimization. Behavior is unchanged in normal runs.

Co-Authored-By: Claude <noreply@anthropic.invalid>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/test_codegen.py (1)

14890-14997: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add zero-length pair-capture edge cases ("" and []).

Nice regression coverage for non-empty pair captures, but the fix is specifically about preserving the len slot. Please add explicit empty String and empty Array capture-in-closure tests to pin len == 0 behaviour as well.

Suggested tests
 class TestPairCapture535:
+    def test_empty_array_capture_length_in_closure(self) -> None:
+        src = """
+public fn main(`@Unit` -> `@Int`)
+  requires(true) ensures(true) effects(pure)
+{
+  let `@Array`<Int> = [];
+  let `@Array`<Int> = array_map(
+    array_range(0, 3),
+    fn(`@Int` -> `@Int`) effects(pure) {
+      nat_to_int(array_length(`@Array`<Int>.0))
+    }
+  );
+  array_fold(`@Array`<Int>.0, 0, fn(`@Int`, `@Int` -> `@Int`) effects(pure) { `@Int.0` + `@Int.1` })
+}
+"""
+        assert _run(src) == 0
+
+    def test_empty_string_capture_length_in_closure(self) -> None:
+        src = """
+public fn main(`@Unit` -> `@Int`)
+  requires(true) ensures(true) effects(pure)
+{
+  let `@String` = "";
+  let `@Array`<Int> = array_map(
+    array_range(0, 3),
+    fn(`@Int` -> `@Int`) effects(pure) {
+      nat_to_int(string_length(`@String.0`))
+    }
+  );
+  array_fold(`@Array`<Int>.0, 0, fn(`@Int`, `@Int` -> `@Int`) effects(pure) { `@Int.0` + `@Int.1` })
+}
+"""
+        assert _run(src) == 0
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_codegen.py` around lines 14890 - 14997, Add two tests that capture
empty pair-layout values to ensure the preserved `len == 0` behavior: add a test
named like test_empty_string_capture_in_closure which sets let `@String` = "" and
uses array_map + string_length(`@String.0`) then asserts result == 0 (3 iterations
→ 0), and a test named like test_empty_array_capture_in_closure which sets let
`@Array`<Int> = array_range(0,0) or let `@Array`<Int> = [] then uses array_map +
nat_to_int(array_length(`@Array`<Int>.0)) and asserts result == 0; follow the
existing test pattern (use _run(src) assertions, array_map/array_fold structure)
so the coverage for the captured `len` slot (in functions like
_translate_anon_fn / _compile_lifted_closure) includes zero-length pair
captures.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@vera/codegen/closures.py`:
- Around line 308-316: The loop that pushes captured locals using cap_locals /
cap_local_kinds via gc_shadow_push is currently added into gc_prologue before
load_instrs, so pushes occur while cap_local still holds default 0; move the
capture-root pushes to run after the env-loads are emitted (i.e., after
load_instrs are generated) so captured heap pointers are rooted only after they
are loaded from $env; you can either relocate the loop that iterates over
cap_locals/cap_local_kinds to after load_instrs creation or collect those push
instructions into a separate list (e.g., gc_prologue_after_loads) and append it
to gc_prologue only after load_instrs, ensuring gc_shadow_push is called when
cap_local contains the loaded value.

---

Outside diff comments:
In `@tests/test_codegen.py`:
- Around line 14890-14997: Add two tests that capture empty pair-layout values
to ensure the preserved `len == 0` behavior: add a test named like
test_empty_string_capture_in_closure which sets let `@String` = "" and uses
array_map + string_length(`@String.0`) then asserts result == 0 (3 iterations →
0), and a test named like test_empty_array_capture_in_closure which sets let
`@Array`<Int> = array_range(0,0) or let `@Array`<Int> = [] then uses array_map +
nat_to_int(array_length(`@Array`<Int>.0)) and asserts result == 0; follow the
existing test pattern (use _run(src) assertions, array_map/array_fold structure)
so the coverage for the captured `len` slot (in functions like
_translate_anon_fn / _compile_lifted_closure) includes zero-length pair
captures.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c9068144-acac-4d53-9dce-615d281f5224

📥 Commits

Reviewing files that changed from the base of the PR and between 2c38a7e and 914d136.

⛔ Files ignored due to path filters (5)
  • docs/index.html is excluded by !docs/**
  • docs/index.md is excluded by !docs/**
  • docs/llms-full.txt is excluded by !docs/**
  • docs/llms.txt is excluded by !docs/**
  • uv.lock is excluded by !**/*.lock, !uv.lock
📒 Files selected for processing (11)
  • CHANGELOG.md
  • HISTORY.md
  • KNOWN_ISSUES.md
  • README.md
  • ROADMAP.md
  • TESTING.md
  • pyproject.toml
  • tests/test_codegen.py
  • vera/__init__.py
  • vera/codegen/closures.py
  • vera/wasm/closures.py
💤 Files with no reviewable changes (1)
  • KNOWN_ISSUES.md

Comment thread vera/codegen/closures.py Outdated
Inline (1):
  1. vera/codegen/closures.py 308-316 — capture-root GC pushes were
     emitted into gc_prologue (before load_instrs), but the env-loads
     hadn't run yet at that point so cap_local still held the default
     0; gc_shadow_push snapshots the local's CURRENT value, so a zero
     was being written to the shadow stack instead of the loaded ptr.
     Captured heap pointers reachable only through the closure could
     be GC'd while in use.

     Pre-existing for i32 captures (PR #569 just extended the
     pattern to i32_pair). Fix: split the prologue into two phases —
     gc_prologue (sp save + param pushes, runs before load_instrs)
     and gc_capture_pushes (capture roots, runs AFTER load_instrs).
     Assembly order now: locals decl, sp save + param pushes, env
     loads, capture roots, body, epilogue.

Outside-of-diff (1):
  2. tests/test_codegen.py — added two edge-case tests:
       test_empty_string_capture_in_closure (empty String captured)
       test_empty_array_capture_in_closure (empty Array<Int> captured)
     Both assert _run(src) == 0. Pin that the post-fix path correctly
     preserves a zero-length pair capture (rather than reading
     garbage from an unallocated len slot in the closure struct).

Test count: 3,685 → 3,687.
ruff S clean, mypy clean, 82 conformance programs pass, 33 examples pass.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan

aallan commented May 5, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Closing out the outside-of-diff finding from this round:

tests/test_codegen.py 14890-14997 — add empty-pair-capture tests ✅ Added in 4b92e42:

  • test_empty_string_capture_in_closurelet @String = "", captured into array_map's closure, asserts array_fold(...) == 0
  • test_empty_array_capture_in_closurelet @Array<Int> = array_range(0, 0), same shape, asserts 0

Both follow the existing TestPairCapture535 pattern (_run(src) + array_map/array_fold). They pin that the post-fix path correctly preserves a zero-length pair capture (rather than reading garbage from an unallocated len slot in the closure struct) — the pre-fix bug's always-zero output happened to coincide with the empty case but for the wrong reason.

Test count: 3,685 → 3,687. ruff S clean, mypy clean, 82 conformance programs pass, 33 examples pass.

@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@aallan: Good closure on that finding. The critical correctness point is well-captured in your description: the pre-fix codepath produced 0 for the empty case by accident (garbage read of an unallocated len slot that happened to contain zero), whereas the post-fix path produces 0 for the right reason (a correctly-loaded len = 0 from the closure struct). The two new tests now pin that distinction — any future regression that re-introduces the garbage-read path would still pass on non-empty pairs (caught by the existing TestPairCapture535 suite) but also correctly fail here if the empty case accidentally "works" for the wrong reason.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/test_codegen.py`:
- Around line 14890-15047: Add a new test (e.g. test_gc_pressure_pair_capture)
modeled on
test_string_capture_length_in_closure/test_array_capture_length_in_closure that
captures a `@String` or `@Array` in the outer scope and, inside the anon fn passed
to array_map, performs heavy allocation (e.g. create many temporary
arrays/strings or call array_range/map repeatedly) before calling
string_length(`@String.0`) or array_length(`@Array`<Int>.0); ensure the final folded
sum equals the expected value (e.g. 5×3 for "hello" or 7×3 for array_range(0,7))
so the test exercises the GC-pressure pair-root shadow-push path while the
captured pair is live. Ensure the new test name and body follow the existing
test style and use _run(src) for validation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 417bd6a2-3924-49a1-a4ae-6a4b9ec6dc55

📥 Commits

Reviewing files that changed from the base of the PR and between 5fc6316 and 4b92e42.

📒 Files selected for processing (6)
  • HISTORY.md
  • README.md
  • ROADMAP.md
  • TESTING.md
  • tests/test_codegen.py
  • vera/codegen/closures.py

Comment thread tests/test_codegen.py
Inline (1):
  tests/test_codegen.py 14890-15047 — add tests that exercise the
  round-1 GC-ordering fix (gc_capture_pushes after load_instrs)
  under in-closure allocation pressure.

Two tests added under TestPairCapture535:

  test_gc_pressure_pair_capture
    Captures @array<Int> = array_range(0, 7) (length 7).  Inside the
    closure body, allocates 3 × array_range(0, 500) = ~12 KB of temp
    arrays BEFORE reading array_length(@array<Int>.3) (.3 because
    the captured array sits at outer-scope index 3, after the three
    inner let-bindings at 0, 1, 2).  Asserts folded sum = 21 (3 × 7).

  test_gc_pressure_string_capture
    Same shape with @string = "hello"; asserts 15 (3 × 5).

If a future change re-broke the GC ordering (capture pushes back
in gc_prologue before load_instrs), these tests would catch it
when in-body allocations cross the GC threshold and a $gc_collect
sweeps the captured pair as unreachable.

Test count: 3,687 → 3,689.
ruff S clean, mypy clean, full pytest passes.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pair-type captures (String, Array<T>) silently drop the len field

1 participant