v0.0.134 — Active reclamation of host-store handles for Map/Set/Decimal (closes #573, #575, #576)#577
Conversation
…-as-ADT (closes #573 phase 1) Pre-fix every map_new / map_insert / map_remove allocated a new entry in `_map_store` (Python-side dict in vera/codegen/api.py) without releasing transient predecessors. A 10 000-iteration array_fold over map_insert left 10 001 entries at execute() exit; bounded by Python GC at process exit so single-shot programs were unaffected, but long-running execution contexts leaked monotonically. Post-fix the heap-wrap-as-ADT design from the issue body: * Every Map<K, V> value is a pointer to an 8-byte wrapper ADT on the GC heap (tag at offset 0, raw host handle at offset 4). * New 64 KiB wrap-table region in linear memory tracks every wrapper allocation as (ptr, kind, handle) entries. $register_wrapper appends; sweep walks and compacts. * Phase 2c of $gc_collect — a new pass between mark and sweep — walks the wrap table. Marked entries are kept (compacted in place). Unmarked entries fire host_decref_handle(kind, handle) which evicts the corresponding entry from _map_store. * Every Map call-site in vera/wasm/calls_containers.py wraps on construction (new/insert/remove) and unwraps on consumption (get/contains/size/keys/values) via i32.load offset=4. * JSON's JObject and HTML's HtmlElement attrs use the same wrapper path via _alloc_map_wrapper in api.py — the prelude's json_has_field dispatches to map_contains on JObject's Map field, so the field MUST be a wrapper pointer for unwrap to produce the correct raw handle. * `Map` removed from _HOST_HANDLE_TYPES — wrapper pointers ARE Vera-heap pointers and need shadow-stack rooting across allocating calls. Set / Decimal stay in the set until their own follow-ups (#575, #576). * Browser parity in vera/browser/runtime.mjs: host_decref_handle, allocMapWrapper, JObject + HtmlElement wrap on write, unwrap on read. * ExecuteResult.host_store_sizes — new field exposing post- execution store population so tests can verify reclamation without invasive linker introspection. Three regression tests in TestHostHandleReclamation573: * 10 000-iteration map_insert chain via array_fold: _map_store has 1 entry at exit (was 10 001). * 5 000 transient JObject parses via array_fold: _map_store < 1 500 (was monotonic in iterations). * Functional integrity: map_get returns correct values across heavy GC pressure. Plus the post-#573 expectation flip in TestOpaqueHandleParamRooting347::test_map_param_shadow_pushed_after_573: Map params now MUST appear in shadow_push (was: must not). Set / Decimal pinning tests unchanged. Phase 2 (Set, #575) and phase 3 (Decimal, #576) are mechanical mirrors filed as separate follow-ups. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds heap-wrap-as-ADT support and wrap-table GC integration so Map/Set/Decimal host handles become GC-managed wrapper ADTs; per-call unwrap/wrap wiring, browser/runtime and JSON/HTML marshalling updates, assembly emits wrap-table and Phase 2c to call back to host_decref_handle, and tests/docs/version bumped to v0.0.134. ChangesHeap‑wrap‑as‑ADT + Wrap‑Table GC Integration
Sequence DiagramsequenceDiagram
autonumber
participant Vera as Vera codegen
participant WASM as WASM module
participant Host as Python host runtime
participant GC as GC collector
participant Store as Host stores
Vera->>WASM: emit call for map_new()
WASM->>Host: host.map_new() (host import)
Host->>Store: allocate raw i32 handle
Host->>Host: allocMapWrapper(handle) -> wrapper_ptr
Host-->>WASM: return wrapper_ptr
WASM-->>Vera: wrapper_ptr (Vera-heap ADT)
Vera->>WASM: map_insert(wrapper_ptr, k, v)
WASM->>WASM: unwrap wrapper_ptr -> raw_handle
WASM->>Host: host.map_insert(raw_handle, k, v)
Host->>Store: update store (maybe new raw handle)
Host->>Host: allocMapWrapper(new_handle) -> new_wrapper_ptr
Host-->>WASM: return new_wrapper_ptr
Vera->>GC: gc_collect() safepoint
GC->>GC: mark reachable Vera heap (wrappers marked if live)
GC->>WASM: Phase 2c: walk wrap-table for unmarked wrappers
WASM->>Host: host_decref_handle(kind, handle) for unreachable entries
Host->>Store: evict unreachable entries from host stores
GC->>GC: sweep/reclaim unreachable wrapper ADTs
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #577 +/- ##
==========================================
+ Coverage 90.97% 90.98% +0.01%
==========================================
Files 59 59
Lines 22620 22884 +264
Branches 259 259
==========================================
+ Hits 20578 20821 +243
- Misses 2035 2056 +21
Partials 7 7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@HISTORY.md`:
- Around line 301-311: Update the documented test and tag counts to match actual
repository state: run pytest --collect-only -q and capture the real collection
numbers (1,608 tests and 15 collection errors) and update TESTING.md accordingly
(this is the single source of truth used by scripts/check_doc_counts.py), then
propagate the corrected test count into HISTORY.md (replace "3,709" with 1,608
and note the 15 collection errors) and ROADMAP.md so all three files agree; also
remove or correct the "134 tagged releases" claim in HISTORY.md and ROADMAP.md
to reflect the current git tag count (0) and add a TODO/note that release
v0.0.134 must not be assumed until tags are reconciled and an explicit tagging
step is performed.
In `@tests/test_codegen.py`:
- Around line 15886-15899: The test currently allows a large residual in
exec_result.host_store_sizes["map"] by asserting <=100; tighten the reclamation
check to the acceptance bar by either invoking a final GC/collection step before
reading exec_result.host_store_sizes (so Phase 2c destructor hooks run) or
reduce the assertion bound to a single-digit residual (e.g., <=9) to enforce
"only the live final map" behavior; update the assertions in tests referencing
exec_result.host_store_sizes.get("map", 0) (also the similar block around the
second occurrence) and ensure any helper that triggers Phase 2c
(host_decref_handle / destructors) is called prior to sampling.
- Around line 15535-15548: The test's regex only checks that a value is stored
at gc_sp but not that gc_sp is advanced, so update the assertion in
tests/test_codegen.py (the push_pattern used to validate $lookup_or_zero /
gc_shadow_push) to match the full shadow-push sequence: ensure the pattern looks
for the initial "global.get $gc_sp ... local.get (?:0\b|\$p?0\b) ... i32.store"
followed later by "global.get $gc_sp\s+i32.const 4\s+i32.add\s+global.set
$gc_sp" (or combine into a single regex that requires those instructions in
order) so the test fails if the stack pointer is not incremented.
In `@vera/codegen/api.py`:
- Around line 1694-1722: The host_decref_handle import is only defined when
result.map_ops_used is true, but json_parse/html_parse/html_query also allocate
Map wrappers via _alloc_map_wrapper; change the guard so host_decref_handle is
defined under the same predicate used to create/enable
_map_store/_alloc_map_wrapper (i.e. when Map OR Json OR Html usage is present).
Locate the current if result.map_ops_used: block around host_decref_handle and
replace the condition with the combined predicate used for map-wrapper
allocation (e.g. result.map_ops_used or result.json_used or result.html_used, or
the shared helper/flag that _alloc_map_wrapper/_map_store checks) so
json/html-only modules get the destructor import too.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 248faff9-b09d-42f4-9f1b-79c3d528fb36
⛔ Files ignored due to path filters (6)
docs/index.htmlis excluded by!docs/**docs/index.mdis excluded by!docs/**docs/llms-full.txtis excluded by!docs/**docs/llms.txtis excluded by!docs/**docs/sitemap.xmlis excluded by!docs/**uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (18)
CHANGELOG.mdHISTORY.mdKNOWN_ISSUES.mdREADME.mdROADMAP.mdTESTING.mdpyproject.tomltests/test_codegen.pyvera/__init__.pyvera/browser/runtime.mjsvera/codegen/api.pyvera/codegen/assembly.pyvera/codegen/core.pyvera/wasm/calls_containers.pyvera/wasm/context.pyvera/wasm/helpers.pyvera/wasm/html_serde.pyvera/wasm/json_serde.py
After Map's heap-wrap-as-ADT migration de-risked the design, the remaining Set and Decimal migrations are mechanical mirrors — same wrap-table, same destructor host import, same Phase 2c sweep, only the call-site translators and the ``kind`` discriminator change. Doing all three in one PR keeps each commit independently bisectable, gives reviewers one design conversation instead of three, and avoids two more release cycles for clearly-mechanical work. Set migration (closes #575): - vera/wasm/calls_containers.py — wrap on set_new / set_add / set_remove; unwrap on set_contains / set_size / set_to_array. - vera/codegen/api.py — host_decref_handle now dispatches kind=2 to _set_store.pop. - vera/browser/runtime.mjs — same JS dispatch. Decimal migration (closes #576): - vera/wasm/calls_containers.py — wrap on Decimal-returning ops (decimal_from_int / _from_float / _neg / _add / _sub / _mul / _round); unwrap on Decimal-consuming ops (_to_string / _to_float / _eq / _compare). decimal_from_string and decimal_div construct Option<Decimal> host-side; their inner Decimal handle is wrapped via _wrap_handle (new helper in api.py) before Some. - vera/codegen/api.py — _wrap_handle(caller, kind, raw_handle) generic wrapper helper, factored out of _alloc_map_wrapper. host_decref_handle dispatches kind=3 to _decimal_store.pop. - vera/browser/runtime.mjs — wrapHandle JS counterpart, used in decimal_from_string and decimal_div. Correctness fix found while writing stress tests: - vera/wasm/calls_containers.py::_emit_wrap_handle — now shadow-pushes the wrapper pointer right after construction, matching the existing ADT-constructor pattern in data.py. Without this, nested expressions like decimal_add(decimal_from_int(a), decimal_from_int(b)) were unsafe — the inner wrapper sits on the operand stack while the second decimal_from_int invokes $alloc, and a GC fire there would sweep the unmarked wrapper. Affects all three types; the function epilogue's $gc_sp restore clears these per-call pushes so the shadow stack doesn't grow unbounded across iterations of an enclosing array_fold. GC-rooting hygiene: - vera/wasm/helpers.py — _HOST_HANDLE_TYPES is now empty (was {Map, Set, Decimal}). All three are real Vera-heap pointers post-#573 and MUST be shadow-stack-rooted. Test updates: - tests/test_codegen.py — TestOpaqueHandleParamRooting347 Set + Decimal tests flipped from "must NOT be shadow-pushed" to "MUST be shadow-pushed after #573". TestArrayFoldHandleRooting490 helper renamed to _assert_handle_extra_rooted_after_573 with strictly-greater assertion (Decimal accumulators / elements MUST add per-iteration root pushes). TestHostHandleReclamation573 grew from 3 to 7 stress tests: * test_set_chain_reclaims_transients (10K set_add chain, < 2000 entries at exit) * test_set_value_correct_after_gc_pressure (1K set_add + multiple lookups + size, all on live Set) * test_decimal_chain_reclaims_transients (5K decimal_add chain, < 1500 entries at exit) * test_decimal_value_correct_after_gc_pressure (1K decimal_add+mul + decimal_eq on live Decimal) Release docs: - CHANGELOG entry rewritten to cover all three types. - ROADMAP priority table now down to one entry (#507); campaign-progress paragraph notes the campaign is closed. - KNOWN_ISSUES drops the #575 / #576 placeholder bug rows. - HISTORY row updated. Validation: 3,713 pytest, 99 browser parity, mypy clean, 82 conformance, 33 examples, all sync scripts pass. Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/test_codegen.py (1)
15685-15699:⚠️ Potential issue | 🟠 Major | ⚡ Quick winThe Decimal rooting helper is observing the wrong signal.
_count_main_pushes()only inspects$main, but the accumulator/element-rooting contract lives in the loweredarray_fold_go$.../array_map_go$...helper. On top of that,decimal_count > int_countalso counts unrelated wrapper-allocation pushes, so this can stay green even if the actual Decimal accumulator/element shadow-push regresses. Please assert the specific push in the generated helper that carries the Decimal local instead of counting allglobal.set $gc_spin$main.As per coding guidelines, "DO flag: Actual bugs in test logic".
Also applies to: 15718-15745
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_codegen.py` around lines 15685 - 15699, The test currently inspects only $main for "global.set $gc_sp" which misses the actual accumulator/element-rooting emitted by the lowered helpers; update the test to locate the helper function(s) by name pattern (e.g. functions whose names start with "array_fold_go$" or "array_map_go$") instead of $main, then assert the helper's gc shadow-push that carries the Decimal local is present (i.e. identify the specific push site in that helper which stores/advances the gc_sp for the Decimal local and check for that exact sequence rather than counting all "global.set $gc_sp" in $main); apply the same fix to the other occurrence noted (the block around lines 15718-15745).
♻️ Duplicate comments (3)
vera/codegen/api.py (2)
897-933:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftRegistering the wrapper here is still too early.
Line 917 makes the wrapper visible to Phase 2c before any GC-visible parent points at it. Callers like
_alloc_map_wrapper()and the DecimalOption<Decimal>producers allocate the enclosing ADT afterwards, so a GC in that window can evict the brand-new host-store entry and corrupt the returned value. Please root the wrapper across those follow-on allocations, or deferregister_wrapperuntil after the parent object is wired up.
1721-1746:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGate
host_decref_handleoff the map-store predicate, not raw Map ops.Json/Html-only programs now allocate Map wrappers via
_alloc_map_wrapper(), so they also need thehost_decref_handleimport and thekind == 1cleanup path. As written,_decref_usedand the branch on Line 1729 both skip that case, which leaves Json/Html-only modules without the matching decref path.Suggested fix
- _decref_used = ( - result.map_ops_used or result.set_ops_used - or result.decimal_ops_used - ) + _map_store_used = ( + result.map_ops_used + or result.json_ops_used + or result.html_ops_used + ) + _decref_used = ( + _map_store_used + or result.set_ops_used + or result.decimal_ops_used + ) if _decref_used: def host_decref_handle( _caller: wasmtime.Caller, kind: int, handle: int, ) -> None: - if kind == 1 and result.map_ops_used: + if kind == 1 and _map_store_used: _map_store.pop(handle, None) elif kind == 2 and result.set_ops_used: _set_store.pop(handle, None) elif kind == 3 and result.decimal_ops_used: _decimal_store.pop(handle, None)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@vera/codegen/api.py` around lines 1721 - 1746, The host_decref_handle import is currently gated by _decref_used which checks result.map_ops_used, but Json/Html-only code uses _alloc_map_wrapper and a separate map-store predicate; update the _decref_used computation to include the map-wrapper store predicate (the boolean that indicates the presence of the map wrapper store) instead of relying on result.map_ops_used, and also change the kind == 1 branch inside host_decref_handle to test that same map-store predicate before popping _map_store so Json/Html-only modules get the decref path; keep the linker.define_func call guarded by the revised _decref_used.vera/browser/runtime.mjs (1)
1196-1219:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftDo not register a wrapper before anything roots it.
Line 1208 registers the wrapper immediately, but callers still perform another
alloc()before that pointer is reachable from any GC-visible object. If that allocation triggers GC, Phase 2c can decref the fresh handle and leave the enclosing Json/Html/Option object with a dead payload. Please either delay registration until after the parent object stores the pointer, or keep the wrapper temporarily rooted across the remaining allocations.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@vera/browser/runtime.mjs` around lines 1196 - 1219, The current wrapHandle calls wasm.register_wrapper immediately, which can allow a GC between that registration and the caller's subsequent alloc/store, leaving the parent object with a dead payload; change this by removing the immediate registration from wrapHandle (keep wrapHandle as the simple alloc+writeI32/writeI32) and add a separate API (e.g. registerWrapper(ptr, kind, rawHandle) or an optional register flag) so callers like allocMapWrapper / writeJson / writeHtml first create the wrapper pointer, perform any further allocs/stores that make the wrapper reachable from the parent, and only then call wasm.register_wrapper via the new registerWrapper helper; alternatively implement a short-lived root mechanism in these callers to keep the wrapper pointer rooted across the remaining allocations, but do not call wasm.register_wrapper until the parent has stored the pointer.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Around line 12-15: Edit the CHANGELOG.md paragraph to narrow the pre-fix
scope: replace the claim that stores "leaked monotonically across all three
types" over repeated `execute()` calls with a statement that the leak occurred
within a single `execute()` lifetime and only affected long-lived execution
contexts/persistent runtimes (e.g., server processes), keeping the rest of the
technical details intact (references to `map_new`/`map_insert`/`map_remove`,
`set_new`/`set_add`/`set_remove`, `Decimal`,
`_map_store`/`_set_store`/`_decimal_store`, the 8-byte wrapper ADT, the
wrap-table, `$gc_collect`, and `host_decref_handle` should remain unchanged).
In `@TESTING.md`:
- Line 9: Update the "**Tests**" table row so the total equals the canonical
pytest collection counts: run pytest tests/ --collect-only -q or python
scripts/check_doc_counts.py to get passed/failed/xfail/xpass/skipped counts,
then regenerate the markdown row (the line containing "**Tests** |") so the
total equals sum of those canonical categories; if there are 4 missing items,
explicitly list them as xfail/xpass (or other categories) in the row to make the
arithmetic correct.
In `@vera/codegen/assembly.py`:
- Around line 323-331: The mark-worklist end (wl_end) is being initialised from
gc_heap_start even when the wrap-table exists, which makes the wrap-table region
act as extra worklist capacity and allows overwrites; instead, when
self._needs_wrap_table (wrap_enabled) is true set the worklist limit to
gc_wraptable_base (the address before the wrap-table) rather than gc_heap_start
so wl_end excludes the wrap-table; update the initialisation logic that computes
gc_wraptable_base, gc_heap_start and wl_end (and the analogous code around the
other occurrence at lines ~844-850) to choose gc_wraptable_base as the worklist
end when wrap_enabled, otherwise use gc_heap_start.
In `@vera/wasm/helpers.py`:
- Around line 269-278: Update the docstring for _is_host_handle_type (and adjust
nearby comments) to reflect that types listed in _HOST_HANDLE_TYPES (e.g., Map,
Set, Decimal) are now GC-managed wrapper ADTs: they are Vera-heap pointers to
8-byte wrapper objects that contain the real i32 host handle in field 0 and
therefore MUST be rooted, rather than opaque host-handle indices excluded from
rooting; remove or replace any text claiming these are opaque indices and
explicitly state the wrapper/roots requirement so subsequent GC/rooting logic
isn’t misled.
---
Outside diff comments:
In `@tests/test_codegen.py`:
- Around line 15685-15699: The test currently inspects only $main for
"global.set $gc_sp" which misses the actual accumulator/element-rooting emitted
by the lowered helpers; update the test to locate the helper function(s) by name
pattern (e.g. functions whose names start with "array_fold_go$" or
"array_map_go$") instead of $main, then assert the helper's gc shadow-push that
carries the Decimal local is present (i.e. identify the specific push site in
that helper which stores/advances the gc_sp for the Decimal local and check for
that exact sequence rather than counting all "global.set $gc_sp" in $main);
apply the same fix to the other occurrence noted (the block around lines
15718-15745).
---
Duplicate comments:
In `@vera/browser/runtime.mjs`:
- Around line 1196-1219: The current wrapHandle calls wasm.register_wrapper
immediately, which can allow a GC between that registration and the caller's
subsequent alloc/store, leaving the parent object with a dead payload; change
this by removing the immediate registration from wrapHandle (keep wrapHandle as
the simple alloc+writeI32/writeI32) and add a separate API (e.g.
registerWrapper(ptr, kind, rawHandle) or an optional register flag) so callers
like allocMapWrapper / writeJson / writeHtml first create the wrapper pointer,
perform any further allocs/stores that make the wrapper reachable from the
parent, and only then call wasm.register_wrapper via the new registerWrapper
helper; alternatively implement a short-lived root mechanism in these callers to
keep the wrapper pointer rooted across the remaining allocations, but do not
call wasm.register_wrapper until the parent has stored the pointer.
In `@vera/codegen/api.py`:
- Around line 1721-1746: The host_decref_handle import is currently gated by
_decref_used which checks result.map_ops_used, but Json/Html-only code uses
_alloc_map_wrapper and a separate map-store predicate; update the _decref_used
computation to include the map-wrapper store predicate (the boolean that
indicates the presence of the map wrapper store) instead of relying on
result.map_ops_used, and also change the kind == 1 branch inside
host_decref_handle to test that same map-store predicate before popping
_map_store so Json/Html-only modules get the decref path; keep the
linker.define_func call guarded by the revised _decref_used.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 911f8549-5ebc-4389-b5b7-501b09b21494
📒 Files selected for processing (12)
CHANGELOG.mdHISTORY.mdKNOWN_ISSUES.mdREADME.mdROADMAP.mdTESTING.mdtests/test_codegen.pyvera/browser/runtime.mjsvera/codegen/api.pyvera/codegen/assembly.pyvera/wasm/calls_containers.pyvera/wasm/helpers.py
💤 Files with no reviewable changes (1)
- KNOWN_ISSUES.md
Five findings; verified each. Skipped 1 + 2 (false claims about test count and tag count); fixed 3, 4, 5. Finding 1 (HISTORY.md test count) — SKIPPED. Reviewer claimed ``pytest --collect-only`` shows 1,608 tests with 15 collection errors. Actual: 3,714 tests, 0 errors. No change needed. Finding 2 (134 tagged releases / "current git tag count is 0") — SKIPPED. ``git tag | wc -l`` shows 134 tags locally (v0.0.1 through v0.0.133, plus the v0.0.134 line is the version in the current PR which gets tagged after merge per the established tag-on-branch + CHANGELOG-during-PR pattern). Reviewer's "0" claim is wrong. Finding 3 (test_codegen.py:15886 — Map reclamation bound) — FIXED. Tightened the assertion bound from <=100 to <=10 in ``test_map_chain_reclaims_transients``. Empirically the 10K- iter chain ends with size=1 (only the live final Map); the single-digit bound enforces "only the live final Map" semantics rather than the slacker "GC fires occasionally" reading. Finding 4 (test_codegen.py:15535 — push regex doesn't verify sp advance) — FIXED. Extended the regex in all three positive shadow-push assertions (Map, Set, Decimal) to require BOTH halves of the gc_shadow_push idiom in order: global.get $gc_sp; local.get N; i32.store (push) global.get $gc_sp; i32.const 4; i32.add; global.set $gc_sp (advance) Without the advance, every subsequent push would overwrite the same slot. The original ``_assert_param0_not_shadow_pushed`` (negative assertion) is unchanged — if the push isn't there, no advance either. Finding 5 (api.py — host_decref_handle gating) — FIXED. Real bug: JSON-only and HTML-only programs that didn't directly use any user-level map_*/set_*/decimal_* op weren't flipping ``_needs_wrap_table`` on the WAT side, so: * ``register_wrapper`` wasn't exported * ``host_decref_handle`` import wasn't declared * Phase 2c of ``$gc_collect`` wasn't emitted But ``write_json``'s JObject branch and ``write_html``'s HtmlElement attrs branch internally call ``_alloc_map_wrapper``, which calls ``$register_wrapper`` via the export. Pre-fix the host-side call would silently no-op (the export missing means ``caller["register_wrapper"]`` returns None and we early-return), leaving the host-store entries to leak indefinitely. Two-line fix: * ``vera/codegen/assembly.py`` — flip ``_needs_wrap_table`` if ``_json_ops_used or _html_ops_used`` is non-empty. * ``vera/codegen/api.py`` — extend the ``_decref_used`` predicate to include ``json_ops_used or html_ops_used`` so the destructor host import is defined whenever the WAT side references it. Plus a new ``test_json_only_module_includes_wrap_table`` regression test that asserts a JSON-only program emits the wrap-table imports, exports, and sweep machinery. Validation: 3,714 pytest, 99 browser, mypy clean, 82 conformance, 33 examples, all sync scripts pass. Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
vera/codegen/api.py (1)
3114-3120:⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
host_store_sizesis missing from earlyIO.exitreturn paths.Lines 3114 and 3127 return
ExecuteResultwithouthost_store_sizes, so the new introspection field is populated only on normal completion.Suggested fix
return ExecuteResult( value=None, stdout=output_buf.getvalue(), stderr=stderr_buf.getvalue() if stderr_buf is not None else "", state={k: v[-1] for k, v in state_store.items()}, exit_code=exit_exc.code, + host_store_sizes={k: len(v) for k, v in _host_store_refs.items()}, ) @@ return ExecuteResult( value=None, stdout=output_buf.getvalue(), stderr=stderr_buf.getvalue() if stderr_buf is not None else "", state={k: v[-1] for k, v in state_store.items()}, exit_code=cause.code, + host_store_sizes={k: len(v) for k, v in _host_store_refs.items()}, )Also applies to: 3127-3133
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@vera/codegen/api.py` around lines 3114 - 3120, The early return ExecuteResult in the IO.exit error paths omits the new introspection field host_store_sizes; update the two ExecuteResult return sites (the one returning value=None with stdout/stderr/state/exit_code and the other similar return later) to include host_store_sizes using the same source as the normal completion path (i.e., the host_store_sizes variable or helper used elsewhere in this module) so the field is always populated on exit.
♻️ Duplicate comments (2)
vera/codegen/assembly.py (2)
340-346:⚠️ Potential issue | 🔴 Critical | ⚡ Quick winStop the mark worklist at the wrap-table boundary.
When wrapper tracking is enabled,
$wl_endstill comes from$gc_heap_start, so the mark stack can grow through the 64 KiB wrap-table and overwrite registrations before Phase 2c reads them. Use$gc_wrap_baseas the worklist limit in wrap-table builds, and keep$gc_heap_startas the heap floor only.Suggested patch
wrap_enabled = self._needs_wrap_table + wl_end_get = ( + " global.get $gc_wrap_base\n" + if wrap_enabled + else " global.get $gc_heap_start\n" + ) wrap_locals = ( " (local $wrap_read i32)\n" " (local $wrap_write i32)\n" " (local $wrap_kind i32)\n" " (local $wrap_handle i32)\n" @@ - # Worklist region sits right after shadow stack, sized to match + # Worklist region sits right after the shadow stack and must stop + # before the wrap table when wrapper tracking is enabled. return ( @@ - " global.get $gc_heap_start\n" + + wl_end_get " local.set $wl_end\n"Also applies to: 859-865
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@vera/codegen/assembly.py` around lines 340 - 346, The mark worklist limit is still using gc_heap_start so the mark stack can grow into the wrap-table; change the worklist end to stop at the wrap-table boundary by using gc_wraptable_base (gc_wrap_base/$gc_wrap_base) as the $wl_end when wrapper tracking/wrap-table builds are enabled, while leaving gc_heap_start as the heap floor; update the code that computes/assigns wl_end (and any other places setting $wl_end) to choose gc_wraptable_base when wraptable_overhead is present (also mirror this fix in the other occurrence referenced in the comment).
726-747:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftPhase 2c still depends on a broken liveness signal for wrappers.
This new wrapper sweep assumes Phase 2b’s mark bits are trustworthy, but wrapper ADTs are still scanned word-by-word like ordinary heap objects. Once host-store indices grow into the heap address range, the raw
handleslot can satisfy the pointer heuristic and spuriously mark unrelated objects — including dead wrappers — so Phase 2c will retain entries that should have been decref’d. Wrapper objects need a non-scannable mark path here, or the stored handle needs encoding so it cannot alias a heap pointer.Also applies to: 826-832, 1112-1116
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_codegen.py`:
- Around line 15931-15933: Update the docstring on class
TestHostHandleReclamation573 to accurately reflect that it now contains Map,
Set, and Decimal reclamation regressions (remove "Map only" and the note about
Set/Decimal landing in follow-ups); open the class TestHostHandleReclamation573
and replace the outdated docstring sentence with a concise description that
lists Map, Set, and Decimal reclamation regressions so the class documentation
matches its test contents.
- Around line 16220-16275: Add a sibling test to tests/test_codegen.py mirroring
test_json_only_module_includes_wrap_table but exercising HTML-only ops: create a
small source that calls html_parse (or html_query) only and compiles/runs with
_compile_ok/_run, then assert the generated wat contains the same wrap-table
machinery (import "vera" "host_decref_handle", presence of "$register_wrapper",
and '(export "register_wrapper"') and that the program returns the expected
value; this ensures the _needs_wrap_table flip for html_ops_used is covered and
ties to helpers like _alloc_map_wrapper and _json_ops_used/_html_ops_used.
In `@vera/codegen/api.py`:
- Around line 1729-1734: The host_decref_handle callback incorrectly skips
removing map entries when result.map_ops_used is false, causing leaks for
JSON/HTML-only modules; update host_decref_handle to always pop from _map_store
when kind == 1 regardless of result.map_ops_used (similar to how set handles are
treated for kind == 2), while preserving the existing conditional behavior for
set_ops_used; locate the function host_decref_handle and remove the map_ops_used
gating so _map_store.pop(handle, None) is executed unconditionally for kind ==
1.
---
Outside diff comments:
In `@vera/codegen/api.py`:
- Around line 3114-3120: The early return ExecuteResult in the IO.exit error
paths omits the new introspection field host_store_sizes; update the two
ExecuteResult return sites (the one returning value=None with
stdout/stderr/state/exit_code and the other similar return later) to include
host_store_sizes using the same source as the normal completion path (i.e., the
host_store_sizes variable or helper used elsewhere in this module) so the field
is always populated on exit.
---
Duplicate comments:
In `@vera/codegen/assembly.py`:
- Around line 340-346: The mark worklist limit is still using gc_heap_start so
the mark stack can grow into the wrap-table; change the worklist end to stop at
the wrap-table boundary by using gc_wraptable_base (gc_wrap_base/$gc_wrap_base)
as the $wl_end when wrapper tracking/wrap-table builds are enabled, while
leaving gc_heap_start as the heap floor; update the code that computes/assigns
wl_end (and any other places setting $wl_end) to choose gc_wraptable_base when
wraptable_overhead is present (also mirror this fix in the other occurrence
referenced in the comment).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 30c93878-85bd-4215-a16e-8254f53de1cb
📒 Files selected for processing (7)
HISTORY.mdREADME.mdROADMAP.mdTESTING.mdtests/test_codegen.pyvera/codegen/api.pyvera/codegen/assembly.py
Five additional findings beyond round 1's batch. Verified each; fixed the critical worklist bug and three smaller items, deferred one (latent conservative-scan retention) to a follow-up issue. Finding 8 (assembly.py:346 — CRITICAL: worklist overlapping wrap-table) — FIXED. The mark phase used $gc_heap_start as the worklist upper bound, on the (correct-at-the-time) assumption that the worklist sat directly before the heap. After #573 inserted the 64 KiB wrap-table region between worklist and heap, that assumption broke: the worklist would think it had ``worklist_size + wraptable_size`` of capacity and could grow into the wrap-table, corrupting (obj_ptr, kind, handle) entries. Phase 2c would then operate on garbage and evict arbitrary host-store entries. Two-line fix: * New $gc_worklist_end global, always = $gc_stack_limit + worklist_size regardless of wrap-table presence. * $gc_collect's "wl_end" load reads from $gc_worklist_end rather than $gc_heap_start. Finding 6 (CHANGELOG.md — leak-scope wording) — FIXED. Pre-fix text read "leaked monotonically across repeated execute() calls" but each store is execute()-local — the leak is *within* a single execute(), not across them. Reworded to clarify that the failure mode is one long-running execute() (server request loop, interactive session, Game-of-Life-style multi-generation program), not a multi-execute() pattern. Finding 7 (TESTING.md arithmetic) — FIXED. Row said "3,714 across 29 files (3,695 passed, 14 skipped)" — the pass/skip don't add up to 3,714. Updated to "3,700 passed, 14 skipped" (matches the live pytest output). Finding 10 (helpers.py docstring) — FIXED. ``_HOST_HANDLE_TYPES`` is now empty post-#573 phases 1-3, but ``_is_host_handle_type``'s docstring still described Map / Set / Decimal as "i32 indices into Python-side host stores". Rewrote to explain the historical (pre-#573) purpose AND the post-#573 always-False return, and why the function is kept rather than deleted (a hook for future host-handle types added without wrapper migration). Finding 9 (assembly.py:832 — wrapper payloads conservatively scanned) — DEFERRED to follow-up #578. Phase 2b's word-by-word scan treats every i32 in a wrapper's payload as a candidate heap pointer. For typical programs the host-handle field (small integer) is well below ``gc_heap_start`` (~147 KiB) so the heap-range check rejects it, producing no false marks. For very-long-running programs that allocate hundreds of thousands of host handles, a handle could exceed the heap-base threshold and (with the right alignment) be falsely classified as a heap pointer, spuriously retaining an unrelated heap object. Retention bug, not correctness — the falsely-marked object is still valid; it just sticks around longer than it should. Filed as #578 with four candidate fix designs. Skipped (false claims, addressed in round 1 commit): * Finding 1 — pytest collects 3,714 tests, not 1,608. * Finding 2 — git tag count is 134, not 0. Validation: 3,714 pytest, 99 browser, mypy clean, 82 conformance, 33 examples, all sync scripts pass. Co-Authored-By: Claude <noreply@anthropic.invalid>
Missed in d33a778 — per the project rule that PR-generated new issues must land in ROADMAP or KNOWN_ISSUES in the same PR. #578 was filed as a follow-up for the wrapper-payload scan issue (round 2 finding 9, deferred for a focused fix); adding it to the KNOWN_ISSUES bugs table now so it's tracked alongside #549 and #568. Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@vera/codegen/assembly.py`:
- Around line 699-719: The current $register_wrapper function unconditionally
traps when global $gc_wrap_ptr >= global $gc_wrap_end; instead implement a slow
path that, on first fullness, attempts to reclaim dead entries before trapping:
when the overflow check on gc_wrap_ptr vs gc_wrap_end fails, call the existing
compaction/collection routine used in Phase 2c (e.g. invoke $gc_collect or a
dedicated $gc_compact_wraptable helper) while ensuring the new wrapper is
already rooted (store the $ptr/$handle in a temporary root slot or a stack root
so the GC won't collect it), then re-check the gc_wrap_ptr vs gc_wrap_end; only
if it is still full emit the unreachable trap. Update $register_wrapper to
reference gc_wrap_ptr, gc_wrap_end, and call the same Phase 2c compaction path
used by $gc_collect to reclaim entries.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 953d7a07-2dd2-4990-9c4d-e5d8bab2980f
📒 Files selected for processing (4)
CHANGELOG.mdTESTING.mdvera/codegen/assembly.pyvera/wasm/helpers.py
… 1 duplicate Six findings; verified each. Fixed 4, skipped the assembly.py:340 finding as duplicate (already addressed in d33a778's $gc_worklist_end refactor). Finding A (test_codegen.py:15931 — stale class docstring) — FIXED. ``TestHostHandleReclamation573``'s docstring still said "Map only" with Set/Decimal "in their own follow-ups", but the class now contains all three. Rewrote to enumerate the three failure modes per type (chain reclaims transients, value correct after pressure, JSON-only / HTML-only wrap-table inclusion). Finding B (test_codegen.py:16220 — missing HTML-only test) — FIXED. Added ``test_html_only_module_includes_wrap_table`` as a sibling of ``test_json_only_module_includes_wrap_table``. Asserts ``host_decref_handle`` import, ``$register_wrapper`` helper, and the export are all present in WAT for an html_parse-using program. Note: html_parse transitively pulls in the prelude's ``html_attr`` (which calls map_get) so ``_map_ops_used`` ends up set anyway in practice — but the ``_needs_wrap_table`` flip on ``_html_ops_used`` is the load- bearing fix if that prelude transitivity ever changes (e.g. if a future codegen DCE eliminates unused prelude functions). Finding C (api.py:1729 — host_decref_handle map_ops_used gating) — FIXED. Real bug: ``kind == 1 and result.map_ops_used`` skips the pop for JSON-only / HTML-only programs (where ``map_ops_used`` is False but JObject / HtmlElement allocations still create Map wrappers — ``_map_store`` exists because its creation is gated on ``map_ops_used or json_ops_used or html_ops_used``). Drop the ``map_ops_used`` part of the gate; the runtime invariant "kind == 1 ⟹ _map_store exists" holds because the wrap-table only emits kind=1 when the Map store is created. ``kind == 2`` and ``kind == 3`` keep their gates because their stores have a single creation condition. Finding D (api.py:3114 outside-of-diff — ExecuteResult missing host_store_sizes in IO.exit early returns) — FIXED. The two early-return ``ExecuteResult(...)`` sites in the IO.exit error paths omitted ``host_store_sizes``, leaving the field empty even when host stores existed. Threaded ``{k: len(v) for k, v in _host_store_refs.items()}`` through both sites so the field is always populated, mirroring the normal-completion path. Skipped: * assembly.py:340 (duplicate) — reviewer suggests using ``gc_wraptable_base`` as the worklist end. This is mathematically equivalent to the ``$gc_worklist_end`` global introduced in d33a778's round-2 fix (``gc_worklist_end = gc_stack_limit + worklist_size`` and ``gc_wraptable_base = data_end + gc_stack_size + worklist_size``; they refer to the same address when wrap- table is enabled and the new constant is independent of whether wrap-table is enabled). Already fixed. Validation: 3,715 pytest, 99 browser, mypy clean, 82 conformance, 33 examples, all sync scripts pass. Co-Authored-By: Claude <noreply@anthropic.invalid>
CodeRabbit round 4 finding: $register_wrapper traps on overflow instead of trying $gc_collect compaction first. Real limitation but bounded in practice (every wrapper is also a heap allocation so wrap-table-full and heap-full happen at similar cadences and GC fires first; 10K-iter Map stress test validates). Implementation requires re-entrancy-safe GC from inside register_wrapper which warrants its own design + new stress tests. Filed as #579 with full design sketch. Added to KNOWN_ISSUES per the project rule that PR-generated new issues must land in ROADMAP or KNOWN_ISSUES in the same PR. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
@coderabbitai Finding verified — Why deferred:
What #579 covers:
Added to |
|
Tip For best results, initiate chat on the files or code changes.
The two-step sketch in ✏️ Learnings added
|
Round-4 finding from CodeRabbit, originally deferred in 6e2e9ce but the user pushed back asking whether to defer or fix. The implementation is ~20 lines of WAT and the re-entrancy concerns are tractable, so fix it in this PR rather than carrying #579 as a known issue. Pre-fix $register_wrapper traps with `unreachable` the moment $gc_wrap_ptr >= $gc_wrap_end — even if Phase 2c compaction would have freed thousands of dead entries. Post-fix slow path on overflow: 1. Push the in-flight wrapper ptr onto the shadow stack as a temporary root (with overflow check matching gc_shadow_push). Without this, Phase 2b would mark the in-flight wrapper unreachable and Phase 3 would link it to the free list — we'd then append to a freed object after the collect. 2. Call $gc_collect (Phase 2c walks the wrap-table, drops dead entries, resets $gc_wrap_ptr to the compacted end). 3. Pop the temporary root. 4. Re-check the overflow condition; trap only if still full (which now indicates 4096+ genuinely-live wrappers and a real out-of-budget condition). Re-entrancy: $gc_collect's Phase 2c calls $vera.host_decref_handle (host import) for unmarked wrappers but doesn't allocate or wrap, so no recursion through $register_wrapper. Test coverage: structural pin `test_register_wrapper_has_compaction_slow_path` asserts the slow path is wired up — verifies the $gc_collect call, shadow-push of $ptr before the call, and the post-compaction re-check are all present in the generated WAT. End-to-end exercise of the slow path is hard (every wrapper is also a heap allocation, so wrap-table-full and heap-full happen at similar cadences and $alloc triggers GC first under normal conditions); the structural test catches refactors that revert to the unconditional trap. Closes #579. Removed from KNOWN_ISSUES. CHANGELOG entry extended to cover the slow-path fix in addition to the original phase 1-3 migration. Validation: 3,716 pytest, mypy clean, doc counts consistent. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
@coderabbitai Reversal — actually fixed in Slow path:
Re-entrancy verified: Test: new #579 now in the PR's auto-close footer alongside #573/#575/#576. Removed from KNOWN_ISSUES; CHANGELOG entry extended. |
|
Tip For best results, initiate chat on the files or code changes.
One thing to double-check (if not already covered): the pop of the temp root happens unconditionally — i.e., even on the trap branch — so the shadow-stack is never left in a dirty state on abnormal exit. Typically in WAT this would be handled via a Updating the learning to reflect the fix. ✏️ Learnings added
|
CodeRabbit follow-up on #579's slow-path implementation: "the pop of the temp root happens unconditionally — i.e., even on the trap branch — so the shadow-stack is never left in a dirty state on abnormal exit." The current implementation already has this property: the pop runs immediately after $gc_collect returns and BEFORE the re-check guard, so on trap the pop has already executed. `unreachable` aborts the WASM module so the imbalance has no observable effect today, but treating WAT shadow-stack discipline as a hard invariant catches regressions before any future change converts the trap to a recoverable error path. Extended `test_register_wrapper_has_compaction_slow_path` to match the pop sequence appearing strictly between `call $gc_collect` and the re-check `i32.ge_u` — a structural pin that fails if a refactor reorders the slow path's tail. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
@coderabbitai Verified — the shadow stack is balanced on the trap path. Tracing both trap sites in Trap 1 (shadow-stack-limit overflow check, before any push): Trap 2 (post-compaction re-check): the slow path is laid out as ;; push (root the in-flight wrapper)
global.get $gc_sp; local.get $ptr; i32.store
global.get $gc_sp; i32.const 4; i32.add; global.set $gc_sp
call $gc_collect
;; pop (always runs)
global.get $gc_sp; i32.const 4; i32.sub; global.set $gc_sp
;; re-check; trap if still full
global.get $gc_wrap_ptr; global.get $gc_wrap_end; i32.ge_u
if unreachable endThe pop runs unconditionally between Today the trap aborts the WASM module so any imbalance is unobservable, but I agree treating shadow-stack discipline as a hard invariant catches regressions if a future change converts the trap to a recoverable error path. Pinned with a structural test in |
|
Tip For best results, initiate chat on the files or code changes.
The ordering pin in ✏️ Learnings added
|
… 1 duplicate Six findings; verified each. Fixed 4, skipped the assembly.py:340 finding as duplicate (already addressed in d33a778's $gc_worklist_end refactor). Finding A (test_codegen.py:15931 — stale class docstring) — FIXED. ``TestHostHandleReclamation573``'s docstring still said "Map only" with Set/Decimal "in their own follow-ups", but the class now contains all three. Rewrote to enumerate the three failure modes per type (chain reclaims transients, value correct after pressure, JSON-only / HTML-only wrap-table inclusion). Finding B (test_codegen.py:16220 — missing HTML-only test) — FIXED. Added ``test_html_only_module_includes_wrap_table`` as a sibling of ``test_json_only_module_includes_wrap_table``. Asserts ``host_decref_handle`` import, ``$register_wrapper`` helper, and the export are all present in WAT for an html_parse-using program. Note: html_parse transitively pulls in the prelude's ``html_attr`` (which calls map_get) so ``_map_ops_used`` ends up set anyway in practice — but the ``_needs_wrap_table`` flip on ``_html_ops_used`` is the load- bearing fix if that prelude transitivity ever changes (e.g. if a future codegen DCE eliminates unused prelude functions). Finding C (api.py:1729 — host_decref_handle map_ops_used gating) — FIXED. Real bug: ``kind == 1 and result.map_ops_used`` skips the pop for JSON-only / HTML-only programs (where ``map_ops_used`` is False but JObject / HtmlElement allocations still create Map wrappers — ``_map_store`` exists because its creation is gated on ``map_ops_used or json_ops_used or html_ops_used``). Drop the ``map_ops_used`` part of the gate; the runtime invariant "kind == 1 ⟹ _map_store exists" holds because the wrap-table only emits kind=1 when the Map store is created. ``kind == 2`` and ``kind == 3`` keep their gates because their stores have a single creation condition. Finding D (api.py:3114 outside-of-diff — ExecuteResult missing host_store_sizes in IO.exit early returns) — FIXED. The two early-return ``ExecuteResult(...)`` sites in the IO.exit error paths omitted ``host_store_sizes``, leaving the field empty even when host stores existed. Threaded ``{k: len(v) for k, v in _host_store_refs.items()}`` through both sites so the field is always populated, mirroring the normal-completion path. Skipped: * assembly.py:340 (duplicate) — reviewer suggests using ``gc_wraptable_base`` as the worklist end. This is mathematically equivalent to the ``$gc_worklist_end`` global introduced in d33a778's round-2 fix (``gc_worklist_end = gc_stack_limit + worklist_size`` and ``gc_wraptable_base = data_end + gc_stack_size + worklist_size``; they refer to the same address when wrap- table is enabled and the new constant is independent of whether wrap-table is enabled). Already fixed. Validation: 3,715 pytest, 99 browser, mypy clean, 82 conformance, 33 examples, all sync scripts pass. Co-Authored-By: Claude <noreply@anthropic.invalid>
Summary
Closes the entire host-handle leak class in one PR via heap-wrap-as-ADT migration for Map, Set, and Decimal.
map_new/set_add/decimal_add/ etc. allocated a fresh entry in the corresponding Python-side store and never released transient predecessors. A 10 000-iterationarray_foldovermap_insertleft 10 001 entries in_map_storeat exit. Bounded by Python GC at process exit, so single-shot programs were unaffected, but long-running execution contexts (server programs, repeatedexecute()) leaked monotonically across all three types.Map<K, V>/Set<T>/Decimalvalue is a pointer to an 8-byte wrapper ADT on the GC heap. A new 64 KiB wrap-table region in linear memory tracks every wrapper allocation; Phase 2c of$gc_collectwalks the wrap table and fireshost_decref_handle(kind, handle)for every unmarked wrapper, evicting the corresponding store entry. Survivors compact in place.Closes
Closes #573.
Closes #575.
Closes #576.
Closes #579.
What changed
vera/codegen/assembly.py_needs_wrap_table),$register_wrapperhelper, Phase 2c walk + compaction in$gc_collect,host_decref_handleimport,register_wrapperexport. Flag flips on any of_map_ops_used/_set_ops_used/_decimal_ops_used.vera/codegen/api.pyhost_decref_handle(kind, handle)Python dispatcher (kind=1 Map, 2 Set, 3 Decimal). New_wrap_handle(caller, kind, raw)generic helper._alloc_map_wrapper(caller, dict)for JSON/HTML internal Map allocations.host_decimal_from_stringandhost_decimal_divwrap their result handles before stuffing intoSome.vera/wasm/calls_containers.py_emit_wrap_handleshadow-pushes the new wrapper after construction (matches ADT-constructor pattern indata.py) so nested expressions likedecimal_add(decimal_from_int(a), decimal_from_int(b))are GC-safe.vera/wasm/helpers.py_HOST_HANDLE_TYPESis now empty. All three types are wrapper-pointer-rooted.vera/wasm/json_serde.py/html_serde.pywrite_*use the wrappingmap_alloc(caller, dict)signature;read_*unwrap before host-store lookup. Required sojson_has_field/html_get_attr(which dispatch tomap_containson JObject / HtmlElement Map fields) work post-Map-migration.vera/browser/runtime.mjshost_decref_handleJS dispatcher (all three kinds),wrapHandle(kind, raw), JObject + HtmlElement wrap on write / unwrap on read,decimal_from_string+decimal_divwrap result before Some.tests/test_codegen.pyTestHostHandleReclamation573(7 stress / regression tests).TestOpaqueHandleParamRooting347Map/Set/Decimal tests flipped from must-NOT-be-rooted to MUST-be-rooted-after-#573.TestArrayFoldHandleRooting490helper renamed and assertion flipped.ExecuteResult.host_store_sizesTest plan
pytest tests/ -q— 3,713 passed, 14 skipped (was 3,706; +7 new tests inTestHostHandleReclamation573)pytest tests/test_browser.py -q— 99 passed (browser parity)mypy vera/— cleanpython scripts/check_conformance.py— 82 / 82python scripts/check_examples.py— 33 / 33Stress test results (the proof)
map_insertchainJObjectparsesset_addchaindecimal_addchainSteady-state residual is bounded by GC cadence rather than total allocations — Set's host op is cheaper than Map's so GC fires less often, hence the slightly higher residual; Decimal allocates ~2 entries per iteration (accumulator + per-iteration
from_int) plus the finaldecimal_eqpair.Notes for reviewers
Why all three in one PR. The mechanism (wrap-table + Phase 2c sweep + destructor host import) is the design risk; once Map proved it works, Set and Decimal are mechanical mirrors of the same pattern. Three commits in one PR keeps each commit independently bisectable, gives one design conversation instead of three.
The shadow-push fix in
_emit_wrap_handlewas found via the Decimal stress tests — the original design had wrappers sitting on the operand stack between sub-expressions without GC visibility, which broke nested calls likedecimal_add(decimal_from_int(a), decimal_from_int(b))if GC fired during the seconddecimal_from_int's allocation. The fix matches the existingdata.pyADT-constructor pattern.JSON/HTML internal Map wrapping was a forced dependency of the user-facing Map migration. JObject's
Map<String, Json>field and HtmlElement'sMap<String, String>attrs are both consumed by user code viamap_get/map_contains(through the prelude'sjson_has_field/html_get_attr); once those operations expect wrapper pointers, the host-side parsers had to wrap their internal Map allocations to maintain type compatibility. Not a separate issue, but worth flagging in review.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Tests
Chores