Fix #578: wrapper-handle bit-31 tagging — last bug standing#673
Conversation
Pre-fix, Map / Set / Decimal wrapper ADTs stored the raw host- store handle (a small i32) at body offset 4. The conservative GC scan in Phase 2b walks every reachable object's payload word-by-word, checking whether each i32 looks like a heap pointer (in heap range, 8-byte aligned). For typical programs the handle counter stayed below gc_heap_start (~147 KiB) so the heap-range check rejected it. But a long-running program allocating >100K host-store entries in a single execute() could see a handle exceed that threshold and (with the right alignment) be falsely classified as a heap pointer — silently retaining an unrelated heap object. Retention bug, not correctness, but unbounded retention for long sessions. This was the last open bug in the KNOWN_ISSUES bug-tracker section (bug list now empty for the first time since ~v0.0.80). ## Fix design Option 1 from the issue body, "self-describing wrappers". Store the handle ORed with 0x80000000 at body offset 4 so the in-heap field is always >= 2 GB, structurally outside any plausible heap-range check. Unwrap recovers the raw handle by ANDing with 0x7FFFFFFF. Heap-ceiling guard in $alloc enforces heap_ptr < 0x80000000, so the disjointness invariant (tagged handles never collide with heap pointers) holds by construction. Why this design over the other three candidates in the issue: - Option 2 (skip-scan flag) touches the header layout and every header reader (~30 sites). Option 1 touches two emission helpers. - Option 3 (wrap-table cross-ref in scan) is O(n*m) and slow enough to fail "correctness over performance". - Option 4 (max-handle lower-bound) is broken — once max_handle > gc_heap_start, handles and heap pointers share the same numeric range and no lower bound can separate them. Maps cleanly to DESIGN.md principle #1 (checkability) via a single structural invariant: heap_ptr < 0x80000000 ⇒ tagged values (>= 2 GB) are outside the scan's heap-range check. ## Code changes - `vera/wasm/calls_containers.py::_emit_wrap_handle` — OR with 0x80000000 before `i32.store offset=4` - `vera/wasm/calls_containers.py::_emit_unwrap_handle` — AND loaded value with 0x7FFFFFFF before returning - `vera/codegen/assembly.py` $alloc — heap-ceiling guard: trap if heap_ptr + total >= 0x80000000 - `vera/codegen/api.py::_wrap_handle` — host-side wrapper allocator also writes the tagged value (used by Option<Decimal> Some payloads etc.) - `vera/wasm/json_serde.py::read_json` and `vera/wasm/html_serde.py::read_html` — host-side readers that walk wrapper.handle directly (bypassing _emit_unwrap_handle) AND with 0x7FFFFFFF to recover the raw handle for map_store lookup ## Tests - `tests/test_codegen.py::TestWrapperHandleTagging578` — 5 tests pinning the contract: 1. Wrap site emits `i32.const 0x80000000; i32.or` 2. Unwrap site emits `i32.load offset=4; i32.const 0x7FFFFFFF; i32.and` 3. $alloc body contains the heap-ceiling guard 4. End-to-end wrap/unwrap round trip preserves the raw handle 5. html_to_string produces correct length (pins host-side mask) ## Documentation - `KNOWN_ISSUES.md` — removed #578. Bug list empty. - `CHANGELOG.md` v0.0.155 entry - `HISTORY.md` v0.0.155 row + footer "155 tagged releases" - README test count 3893 → 3898 Validation: pytest 3868/3868, mypy clean, ruff S clean, conformance 86/86, examples 34/34, doc-counts consistent, site assets up to date. Closes #578. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR implements fix ChangesWrapper-handle bit-31 tagging for GC safety
Sequence Diagram(s)sequenceDiagram
participant Host as api._wrap_handle
participant Validator as api._validate_wrap_handle
participant WAT as wasm emitter
participant Wasm as Wasm runtime
participant Deser as read_html/read_json
participant MapStore as map_store
Host->>Validator: validate raw_handle
Host->>WAT: emit store body_ptr+4 = raw | 0x80000000
WAT->>Wasm: emit $alloc heap-ceiling check
Wasm->>Wasm: load wrapper_ptr+4 then and 0x7FFFFFFF
Deser->>MapStore: lookup using unmasked handle
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Per feedback on #673: retain the Bugs section heading but use a plain "No known bugs." rather than the longer self-referential note about the section having been empty before. The HISTORY.md cross-reference doesn't add operational value at this point — anyone reading KNOWN_ISSUES who needs that context can follow the top-of-file pointer to the issue tracker. Co-Authored-By: Claude <noreply@anthropic.invalid>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #673 +/- ##
=======================================
Coverage 91.02% 91.02%
=======================================
Files 60 60
Lines 23409 23413 +4
Branches 259 259
=======================================
+ Hits 21308 21312 +4
Misses 2094 2094
Partials 7 7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_codegen.py`:
- Around line 15467-15553: The tests currently perform brittle substring checks
on result.wat (e.g. looking for "0x80000000", "i32.or", "0x7FFFFFFF",
"i32.ge_u") and locate functions with plain find("(func $alloc"), which can
false-pass; update the three tests (particularly test_unwrap_emits_mask_and and
test_alloc_emits_heap_ceiling_guard) to use boundary-safe regex searches on
result.wat to assert exact adjacent opcode sequences (for wrap: i32.const
0x80000000\s+i32\.or; for unwrap: i32\.load\s+offset=4\s+i32\.const
0x7FFFFFFF\s+i32\.and) and locate the $alloc function header with a regex like
r"\(func\s+\$alloc\b" then extract its body up to the next "(func " and assert
the heap-guard sequence appears in order (global.get \$heap_ptr, local.get
\$total, i32.add, i32.const 0x80000000, i32.ge_u, if/unreachable/end) using a
single multi-line regex or explicit ordered searches to ensure adjacency and
correct ordering rather than loose substring presence.
In `@vera/codegen/api.py`:
- Around line 959-967: The code is tagging raw_handle with 0x80000000 without
checking that raw_handle fits in 31 bits, which can alias large handles; add a
guard before the call to _write_i32/_call_register_wrapper that validates
raw_handle < 0x80000000 (or (raw_handle & 0x80000000) == 0) and raise a clear
exception (e.g., ValueError/RuntimeError with raw_handle, kind, body_ptr) if it
doesn't, so we fail fast instead of corrupting the host-store lookups; put this
check immediately before the existing calls involving raw_handle in the block
containing _write_i32 and _call_register_wrapper.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: d8adf94b-617c-426b-ad8c-f5cfb460fb1e
⛔ Files ignored due to path filters (5)
docs/index.htmlis excluded by!docs/**docs/index.mdis excluded by!docs/**docs/llms-full.txtis excluded by!docs/**docs/llms.txtis excluded by!docs/**uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (14)
CHANGELOG.mdHISTORY.mdKNOWN_ISSUES.mdREADME.mdROADMAP.mdTESTING.mdpyproject.tomltests/test_codegen.pyvera/__init__.pyvera/codegen/api.pyvera/codegen/assembly.pyvera/wasm/calls_containers.pyvera/wasm/html_serde.pyvera/wasm/json_serde.py
**Finding 1: substring-only WAT checks could false-pass** (valid).
The previous tests asserted constants like `0x80000000`, `i32.or`,
`0x7FFFFFFF`, and `i32.ge_u` in isolation. But:
- `i32.const 0x80000000` appears in BOTH the wrap-site tag AND
the heap-ceiling guard. Stripping the wrap OR would still
leave the substring (from the guard) and the test would pass.
- `i32.or` also appears in header-mark manipulation, unrelated
to the wrap path.
Switched all three tests to boundary-safe adjacent-sequence
regex:
- `test_wrap_emits_tag_or`: `re.search(r"i32\.const 0x80000000\s+i32\.or", ...)`
- `test_unwrap_emits_mask_and`: `re.search(r"i32\.load\s+offset=4\s+i32\.const 0x7FFFFFFF\s+i32\.and", ...)`
- `test_alloc_emits_heap_ceiling_guard`: extract `$alloc` via
`re.search(r"\(func \$alloc\b", ...)` (boundary-safe vs the
previous `find("(func $alloc")` which could false-match
`$alloc_xxx`), then assert the EXACT ordered 8-instruction
guard sequence (`global.get $heap_ptr; local.get $total;
i32.add; i32.const 0x80000000; i32.ge_u; if; unreachable;
end`) via a single multi-line regex with `re.DOTALL`.
**Finding 2: _wrap_handle doesn't check raw_handle < 0x80000000**
(valid). If a handle ever has bit 31 set, `raw_handle |
0x80000000` is a no-op and the unwrap mask `& 0x7FFFFFFF`
returns the WRONG handle — silent host-store-lookup corruption
that's exactly the latent failure class #578 itself sought to
eliminate. Practically infeasible (counters bounded well below
2^31; a 2B-handle session is wall-clock-infeasible) but the
invariant deserves an explicit assertion per DESIGN.md principle
#1 (checkability). Added `if raw_handle & 0x80000000: raise
RuntimeError(...)` immediately before the `_write_i32` /
`_call_register_wrapper` block in
`vera/codegen/api.py::_wrap_handle`. Error includes
raw_handle, kind, body_ptr per CR's exact recommendation.
**Not addressed: CR's 3rd finding**
("Missing the long-run high-handle-counter regression that
#578 is meant to prevent" — labelled Major, Heavy lift).
Declined with reasoning:
1. **#578 was a RETENTION bug, not a correctness bug.** Pre-fix
the conservative scan could mark unrelated heap objects when
a handle value matched the heap-range check. In an
IMMUTABLE language those objects can't corrupt state — only
memory grows. No output assertion can distinguish pre-fix
from post-fix at modest scale.
2. **A true regression test requires heap-growth inspection.**
Vera's test infra doesn't expose `$heap_ptr` post-run; adding
that machinery is a separate infrastructure change. An
alternative — measure WASM `memory.size` deltas — needs the
same instrumentation.
3. **The structural invariants are mathematically sufficient.**
The 3 tightened tests + `$alloc` heap-ceiling guard prove
by construction that no tagged value can EVER match the
heap-range check: heap_ptr is hard-capped at 0x80000000, and
every tagged value is >= 0x80000000. Sets don't overlap.
This is a stronger guarantee than a stochastic
non-occurrence stress test could provide.
4. **A weaker "scale" test would be misleading.** 200K wrap
allocations under eager GC complete fine pre-fix too (the
retention is bounded and self-clearing once the falsely-
retained object becomes legitimately unreachable in a later
cycle). Such a test would pass on broken code, providing
false confidence.
If we add `$heap_ptr` inspection to the test harness in future
(a worthwhile general improvement), I'd be happy to file a
follow-up that includes a deterministic retention regression.
Tracked as a possible follow-up issue.
Validation: pytest 3868/3868, mypy clean, ruff S clean,
doc-counts consistent.
Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
vera/codegen/api.py (1)
968-985:⚠️ Potential issue | 🟠 Major | ⚡ Quick winStrengthen the handle-range guard to reject all non-31-bit values.
Line 968 only checks bit 31, so values like
0x1_0000_0001can pass, then be truncated by_write_i32and alias to a different unwrapped handle. Use an explicit range check (0 <= raw_handle < 0x80000000) to preserve the round-trip invariant.Proposed fix
- if raw_handle & 0x80000000: + if raw_handle < 0 or raw_handle >= 0x80000000: raise RuntimeError( f"#578: raw_handle={raw_handle!r} (kind={kind!r}, " f"body_ptr={body_ptr!r}) has bit 31 set; cannot tag " f"for the conservative-scan disjointness invariant. " f"Host-store handle counters must stay below " f"0x80000000 (2^31). Either a counter overflowed or " f"a non-counter value flowed into _wrap_handle." )#!/bin/bash # Verify that bit-31 testing alone misses some out-of-range ints that alias after i32 truncation. python - <<'PY' samples = [0x7fffffff, 0x80000000, 0x100000001, -0x100000000] for h in samples: bit31_guard = bool(h & 0x80000000) stored_i32 = (h | 0x80000000) & 0xFFFFFFFF unwrapped = stored_i32 & 0x7FFFFFFF print(f"h={h} bit31_guard={bit31_guard} stored_i32=0x{stored_i32:08X} unwrapped={unwrapped}") PYAs per coding guidelines:
vera/**/*.pyshould be reviewed for correctness and type safety.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@vera/codegen/api.py` around lines 968 - 985, The current guard only checks raw_handle & 0x80000000 and misses values outside the 31-bit non-negative range; change the check to explicitly validate that raw_handle is an integer in the range 0 <= raw_handle < 0x80000000 (e.g. raise RuntimeError if not), before calling _write_i32 or _call_register_wrapper, to prevent truncation/aliasing; update the error message in the RuntimeError to reflect the explicit range check and reference raw_handle, kind and body_ptr as currently done.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_codegen.py`:
- Around line 15604-15629: Add a sibling unit test to validate the host-side
0x7FFFFFFF mask is applied for json_serde the same way it is for html_serde:
create a new test (e.g., test_json_round_trip_uses_host_side_mask) modeled on
test_html_round_trip_uses_host_side_mask that builds a small program which
constructs a JSON object via JsonValue/Map, serializes it with json_to_string,
and asserts the string length matches the expected value when the attribute/map
is present; ensure the test exercises the path that reads the wrapper handle
field directly (the same pattern that causes the wrapper_ptr+4 read in
vera/wasm/json_serde.py) so the host-side masking must be applied to avoid a
missing map and a wrong output.
---
Duplicate comments:
In `@vera/codegen/api.py`:
- Around line 968-985: The current guard only checks raw_handle & 0x80000000 and
misses values outside the 31-bit non-negative range; change the check to
explicitly validate that raw_handle is an integer in the range 0 <= raw_handle <
0x80000000 (e.g. raise RuntimeError if not), before calling _write_i32 or
_call_register_wrapper, to prevent truncation/aliasing; update the error message
in the RuntimeError to reflect the explicit range check and reference
raw_handle, kind and body_ptr as currently done.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 25cf5179-bca3-4165-86b6-6d00e4867191
📒 Files selected for processing (3)
TESTING.mdtests/test_codegen.pyvera/codegen/api.py
**Finding 1: missing JSON round-trip test for the host-side mask**
(valid). The previous suite has `test_html_round_trip_uses_host_
side_mask` which pins the mask in `vera/wasm/html_serde.py` but no
analogous test for the symmetric mask in `vera/wasm/json_serde.py`
(both apply `& 0x7FFFFFFF` when reading `wrapper_ptr + 4`
directly).
Added `test_json_round_trip_uses_host_side_mask` modelled on the
HTML test: builds a `JObject(map_insert(map_new(), "name",
JString("hi")))`, serialises via `json_stringify`, asserts the
length is 14 (`{"name": "hi"}` — Python's `json.dumps` default
separators include a space after the colon). Without the
host-side mask the map_store lookup misses, `read_json` falls
through to the warning + empty-dict path, and `json_stringify`
emits `{}` = 2 chars. A 14-vs-2 assertion clearly discriminates.
**Finding 2: _wrap_handle bit-31 guard misses values outside the
31-bit non-negative range** (valid). The previous check `if
raw_handle & 0x80000000:` only caught values WITH bit 31 set. It
let through:
- Values >= 0x100000000 — wrap on `_write_i32`'s 32-bit truncation
and silently corrupt the round-trip
- Type-confusion sentinels (e.g. None, non-int objects) — would
raise TypeError deeper in the stack rather than at the
diagnosable check point
Tightened to an explicit range check: `if not (isinstance(
raw_handle, int) and 0 <= raw_handle < 0x80000000)`. Error
message updated to describe the range and enumerate the failure
modes it now catches. Negative values, integer overflow, and
type confusion all fail fast at the same point.
Validation: pytest 3869/3869, mypy clean, ruff S clean, doc-counts
consistent (test_codegen.py: 1110 → 1111 tests; total 3898 → 3899).
Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_codegen.py`:
- Around line 15457-15484: The test test_wrap_emits_tag_or currently searches
result.wat for a standalone "i32.const 0x80000000\s+i32.or" which can match
unrelated OR sites; tighten the matcher to assert the tag emission is adjacent
to the wrapper's field store by including the specific store instruction
sequence used by the wrap-site (e.g. the exact "i32.store/ i32.store8" or
wrapper-field access pattern emitted by the compiler) in the regex so it only
passes when the tag is emitted immediately before the wrapper field store;
update the regex in test_wrap_emits_tag_or (which inspects result.wat) to
require that fuller adjacent pattern.
- Around line 15604-15657: The tests test_html_round_trip_uses_host_side_mask
and test_json_round_trip_uses_host_side_mask only assert string length, which
can miss incorrect content; change them to assert exact serialized output
instead of length by using the serializer invocation (via the same helper _run)
to return the actual string and compare to the expected values ("<p
title=\"hello\"></p>" for the HTML test and "{\"name\": \"hi\"}" for the JSON
test) or else call/implement a variant helper (e.g. _run_str or modify _run)
that returns the serialized string, then replace the numeric assertions with
exact string equality checks against those expected strings.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: b863e3ad-e833-49b3-a5b6-9717514d6c4c
📒 Files selected for processing (5)
README.mdROADMAP.mdTESTING.mdtests/test_codegen.pyvera/codegen/api.py
Comprehensive PR review (code-reviewer + pr-test-analyzer + comment-analyzer + silent-failure-hunter, all in parallel) surfaced 9 actionable items. Addressed 7, deferred 2 with reasons. **Critical** 1. **CHANGELOG test count** (comment-analyzer + code-reviewer). The Tests section said "5 new tests" but the class actually has 6 (the JSON sibling added in round 3 wasn't back- propagated). Updated to 6 with the new bullet. 2. **`host_html_parse` broad-except swallowed `_wrap_handle` invariant violations** (silent-failure-hunter). The `except Exception as exc:` at `vera/codegen/api.py:2784` converted any RuntimeError (including the new #578 range- check raise) into a user-domain `Result.Err` string, masking real invariant violations as "malformed HTML" errors. Narrowed to `(ValueError, TypeError, AttributeError)` matching the sibling `host_json_parse` pattern. **Important** 3. **`_wrap_handle` range guard had zero unit tests** (pr-test- analyzer, rating 7). Extracted the validator to a module- level `_validate_wrap_handle` helper so it can be unit- tested directly without standing up a wasmtime instance. `_wrap_handle` now calls the helper. Added 5 tests covering all failure modes: - `test_validate_wrap_handle_accepts_valid_range` (0, 1, 12345, 0x7FFFFFFE, 0x7FFFFFFF) - `test_validate_wrap_handle_rejects_negative` (-1, -12345) - `test_validate_wrap_handle_rejects_at_2gb_boundary` (0x80000000) - `test_validate_wrap_handle_rejects_above_32bit` (0x100000000, 0x100000001 — the round-1 bit-only check would have missed these) - `test_validate_wrap_handle_rejects_non_int` (None, "5", 1.5, [1], {}) 4. **GC mark phase missing defense-in-depth narrative** (silent- failure-hunter). The conservative scan's heap-range check relies on the disjointness invariant (`heap_ptr < 2 GiB` so tagged handles `>= 2 GiB` can never match). Added a substantial comment block at the scan site documenting the invariant and pointing at the two structural tests that pin it. Comment-only (no runtime cost in the GC hot path); the structural tests are the mechanical defense. **Comment accuracy** (comment-analyzer) 5. `test_json_round_trip_uses_host_side_mask` referenced `json_serde.py` line 213 by hardcoded number — drifts on any nearby edit. Replaced with the "unknown JObject handle" warning reference. 6. `2 GB` → `2 GiB` throughout (assembly.py + tests/test_ codegen.py). Strictly `0x80000000 = 2 GiB` (binary), not 2 GB (decimal = ~1.86 GiB). 7. `"Practical Vera programs use <100 MB"` was an unsubstantiated claim stated as fact. Reworded to `"Programs we have measured stay well below the 2 GiB ceiling"`. 8. The class docstring's `gc_heap_start (~147 KiB)` figure actually depends on the string-pool size (`data_end + 144 KiB`). Softened to `~144 KiB above the data section, so roughly 144 KiB plus the string-pool size`. **Deferred with reason** 9. `$alloc` heap-ceiling trap is bare `unreachable`, surfaces via the trap classifier as `"unreachable"` kind with the match-arm Fix message (silent-failure-hunter, Important). Polishing this requires either a host-import call from `$alloc` to populate `last_violation` (cost on the hot path) or a new classifier kind with a heuristic detector (brittle). Practical programs never reach this trap (heap << 2 GiB), so deferred. Marked with a TODO inside `assembly.py` as a Python comment (NOT a WAT comment — a WAT comment between `if` and `unreachable` would break the adjacent-sequence regex in `test_alloc_emits_heap_ceiling_ guard`). The other declined findings from the agents: - silent-failure-hunter `read_json` warn → raise (pre-existing behavior, separate concern) - silent-failure-hunter `_call_register_wrapper` early-return → assert (pre-existing pattern, out of scope) - silent-failure-hunter wasmtime trap-text enrichment (would need last_violation channel work; separate PR) - code-reviewer i32.add wraparound at heap-ceiling guard (confidence 60 — self-flagged; defended upstream by `memory.grow` and 31-bit-size invariant) Validation: pytest 3,874/3,874 ✓ (added 5 new tests), mypy clean, ruff S clean, all 18 pre-commit hooks green, doc-counts consistent (1,111 → 1,116 tests in test_codegen.py; total 3,899 → 3,904). Co-Authored-By: Claude <noreply@anthropic.invalid>
PR self-review summaryRan Critical (2 found, both fixed)
Important (2 found, both fixed)
Comment accuracy (4 minor, all fixed)
Deferred with reason (1)
Other declined findings (with brief reasons)
Strengths
ValidationAfter
Commit trail on this PR
|
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
tests/test_codegen.py (1)
15478-15485:⚠️ Potential issue | 🟠 Major | ⚡ Quick winTighten the wrap-site regex to avoid false positives.
This pattern can match unrelated WAT (notably GC mark-bit paths), so the test may pass even if wrapper tagging regresses. Please anchor it to the wrapper-field write sequence (e.g. include
i32.store offset=4immediately after thei32.or, or scope to the unwrap/wrap helper body first).Suggested assertion shape
- assert re.search( - r"i32\.const 0x80000000\s+i32\.or", result.wat, - ), ( + assert re.search( + r"i32\.const 0x80000000\s+" + r"i32\.or\s+" + r"i32\.store\s+offset=4", + result.wat, + ), (Based on learnings: In this repo’s Python tests, when asserting specific WAT symbols are present/absent, avoid plain substring checks and use boundary-safe, context-safe matching to prevent false positives.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_codegen.py` around lines 15478 - 15485, Tighten the regex in the failing assertion that checks result.wat to avoid false positives by anchoring the wrap-site sequence: match the adjacent "i32.const 0x80000000" followed by "i32.or" and immediately the wrapper-field write (e.g. "i32.store offset=4") so the pattern targets the wrap-field write site rather than other GC paths; update the assert in tests/test_codegen.py that references result.wat to use a single regex which includes the i32.store offset=4 (or scope the match to the wrap/unwrap helper body) so only the intended wrapper-tag emission satisfies the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@vera/codegen/api.py`:
- Around line 74-77: The current validator allows bools because
isinstance(raw_handle, int) accepts True/False; update the conditional that
checks raw_handle so it explicitly rejects bools (e.g. require type(raw_handle)
is int or add "and not isinstance(raw_handle, bool)") while preserving the range
check against 0x80000000; change the if that contains "isinstance(raw_handle,
int) and 0 <= raw_handle < 0x80000000" to perform the explicit bool exclusion so
True/False no longer pass as 1/0.
In `@vera/codegen/assembly.py`:
- Around line 714-721: The current heap ceiling guard in
vera/codegen/assembly.py performs "global.get $heap_ptr; local.get $total;
i32.add; i32.const 0x80000000; i32.ge_u" which is vulnerable to wraparound;
change the sequence in the heap ceiling guard to first check if local.get $total
is >= 0x80000000 and trap if so, then compute "i32.const 0x80000000; local.get
$total; i32.sub" and compare that to global.get $heap_ptr (use i32.gt_u or
i32.le_u as appropriate) so you test "heap_ptr <= 0x80000000 - total" using
subtraction rather than addition to prevent overflow bypass; update the trap
block (unreachable) to run when either check fails.
---
Duplicate comments:
In `@tests/test_codegen.py`:
- Around line 15478-15485: Tighten the regex in the failing assertion that
checks result.wat to avoid false positives by anchoring the wrap-site sequence:
match the adjacent "i32.const 0x80000000" followed by "i32.or" and immediately
the wrapper-field write (e.g. "i32.store offset=4") so the pattern targets the
wrap-field write site rather than other GC paths; update the assert in
tests/test_codegen.py that references result.wat to use a single regex which
includes the i32.store offset=4 (or scope the match to the wrap/unwrap helper
body) so only the intended wrapper-tag emission satisfies the test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9e1cbf6d-f113-48ef-9d39-1f59773678c8
📒 Files selected for processing (7)
CHANGELOG.mdREADME.mdROADMAP.mdTESTING.mdtests/test_codegen.pyvera/codegen/api.pyvera/codegen/assembly.py
**Finding 1: test_wrap_emits_tag_or regex too narrow** (valid for
symmetry). The previous regex `i32\.const 0x80000000\s+i32\.or`
matched the const-or pair which today is unique to the wrap site,
but the test signalled the WRONG semantic — it pinned an
accidental pair-uniqueness fact rather than the wrapper-store
intent. Tightened to
`i32\.const 0x80000000\s+i32\.or\s+i32\.store offset=4` to pin
the FULL 3-instruction wrap sequence (const + or + store).
Symmetric with `test_unwrap_emits_mask_and` which already pins
the full 3-instruction unwrap sequence (load + const + and).
Both tests now check the SEMANTIC operation (tag-then-store /
load-then-mask) rather than the accidental presence of distinctive
pairs.
**Finding 2: HTML/JSON tests asserted only length** (valid — real
bug class). A hypothetical bug producing wrong content with the
right length (e.g. `<p title="WRONG"></p>` is also 21 chars,
`{"name": "BB"}` is also 14 chars) would have slipped past the
length check. Switched both tests to exact string equality:
- Vera programs now use `IO.print(html_to_string(...))` /
`IO.print(json_stringify(...))` so the rendered output flows
to stdout
- Helper switched from `_run` (int result) to `_run_io` (captured
stdout string)
- Assertion changed from `== 21` / `== 14` to
`== '<p title="hello"></p>'` / `== '{"name": "hi"}'`
The exact string assertion also documents the expected format in
the test itself rather than requiring a length calculation comment.
Validation: pytest 3,874/3,874, mypy clean, ruff S clean,
doc-counts consistent.
Co-Authored-By: Claude <noreply@anthropic.invalid>
CR posted 4 new findings on commits ca7038a + 3bf7e74. All four valid; all four fixed. **Finding 1: test_wrap_emits_tag_or regex too narrow** (Minor). Tightened from `i32\.const 0x80000000\s+i32\.or` to `i32\.const 0x80000000\s+i32\.or\s+i32\.store offset=4` so the test pins the full 3-instruction wrap sequence — const + or + store — rather than the const-or pair which is unique today only by accident. Symmetric with `test_unwrap_emits_mask_and`. **Finding 2: HTML/JSON tests asserted only length** (Minor — but catches a real bug class). Switched both tests from `_run` + length assertion to `_run_io` + exact string equality: - HTML: assert output == `<p title="hello"></p>` (was: == 21) - JSON: assert output == `{"name": "hi"}` (was: == 14) A bug producing wrong content with the right length would now fail, where the length-only check would have passed. **Finding 3: _validate_wrap_handle accepts bool** (Critical). `isinstance(raw_handle, int)` admits `True` and `False` because `bool` subclasses `int` in Python. `True` would silently alias to handle 1, `False` to handle 0 — exactly the silent-corruption class #578 sought to eliminate. Changed to `type(raw_handle) is int` which rejects bool while still accepting plain int. Added `test_validate_wrap_handle_rejects_bool` covering both `True` and `False`. **Finding 4: i32.add wraparound in heap-ceiling guard** (Major). The code-reviewer agent self-flagged this at confidence 60 last round; CR has now elevated it to Major with a concrete failure case. Pre-fix: global.get $heap_ptr local.get $total i32.add ;; wraps on overflow! i32.const 0x80000000 i32.ge_u For `heap_ptr = 0xFFFFFFFF, total = 10`, the add wraps to 0x09 which is < 0x80000000 — the guard silently passes and the heap overflows past 2 GiB, reintroducing #578. Upstream `memory.grow` makes this unreachable in practice but the algebraic gap is real. Replaced with the overflow-safe form: ;; Step 1: total < 2 GiB local.get $total i32.const 0x80000000 i32.ge_u if unreachable end ;; Step 2: heap_ptr < 0x80000000 - total global.get $heap_ptr i32.const 0x80000000 local.get $total i32.sub i32.ge_u if unreachable end Step 1 rejects pathologically-large totals (which would underflow the step-2 subtraction). Step 2 uses subtraction instead of addition, which can't overflow. Note: CR's suggested diff used `i32.gt_u` for step 2; I used `i32.ge_u` to preserve the original semantics (the pre-fix form trapped on `heap_ptr + total >= 0x80000000`, which is equivalent to `heap_ptr >= 0x80000000 - total`). With `gt_u`, `heap_ptr` exactly equal to `0x80000000 - total` would slip through and the post-bump `heap_ptr` would land exactly at the ceiling, violating the invariant. `test_alloc_emits_heap_ceiling_guard` updated to pin both step-1 and step-2 sequences in order (step 2 anchored from step 1's end position). Validation: pytest 3,875/3,875, mypy clean, ruff S clean, doc-counts consistent (1,116 → 1,117 tests in test_codegen.py; total 3,904 → 3,905). Co-Authored-By: Claude <noreply@anthropic.invalid>
CodeRabbit posted three findings on rzyns's PR plus there's a CI gate (doc-counts) and a project convention (empty Bugs table) to address. Two additional minor refactors (DRY constant, public helper rename) included since they touch the same files. **CR Finding 1: _bad_vera missing encoding="utf-8"** (test portability). Pre-existing helper used 30+ times in test_cli.py since long before aallan#685, but CR caught it now because rzyns's new tests added more callers. Windows runs would fall back to cp1252 on the default text-mode encoding and corrupt UTF-8 sequences in Vera fixtures. CLAUDE.md "Cross-platform pitfalls" already flags this as a convention. One-line fix on the helper benefits every caller. **CR Finding 2: needs_pos.category != "failed" too weak** (test_tester.py). In `test_call_precondition_error_fails_caller_ not_callee`, the negative assertion would still pass if needs_pos were classified as "skipped" or any other non-failed bucket. Strengthened to `== "verified"` so the test actually pins that the callee is correctly proven, not just that it avoided the failed bucket. **CR Finding 3: failed_fns dict overwrite is order-dependent** (vera/tester.py). When a single function attracts multiple verifier errors (e.g. `ensures(false)` produces E500 AND `@Nat - @Nat` produces E502 on the same body — the `test_displayed_failed_function_does_not_double_count_verifier_ errors` test exercises exactly this shape), the loop overwrites the dict entry and the displayed `reason` flips with diagnostic iteration order. Switched to first-hit (`if name not in failed_fns`) — `setdefault`-style semantics, no judgement call on which code "should" win. **Doc counts: 7 stale numbers** (CI gate would fail). `check_doc_counts.py` flagged TESTING.md total, test_cli.py + test_tester.py + test_tester_coverage.py rows, and ROADMAP.md test count. Updated all 7 plus README.md test count for consistency: 3,905 → 3,916 total. All counts now consistent. **KNOWN_ISSUES.md empty table** (project convention). PR removed the aallan#674 row but left the table header `| Bug | Issue |` with no body. Per the convention established at aallan#673's release ("retain the Bugs section, and just have some text in it to say 'No known bugs'"), replaced the empty table with the convention line. **Nit 1: DRY** — `{"E500", "E501", "E502"}` was duplicated between vera/tester.py and vera/cli.py. Extracted as `VERIFICATION_ERROR_CODES = frozenset(...)` module-level in vera/tester.py; cli.py imports it. `frozenset` over plain set so the constant is immutable (matches its semantic role as a classifier-set constant). **Nit 2: private name reaching across modules** — `cli.py` was importing `_failed_function_name` from vera/tester.py. The single-underscore prefix signals "private by convention" but cli.py's reliance on it was a leak. Renamed `_failed_function_name` → `failed_function_name` (public); docstring updated to mention the cli.py call site explicitly. Validation: - pytest 3,886 / 3,886 (no test count change — the renames don't affect test behaviour, just the names) - mypy clean - ruff S clean - conformance 86/86, examples 34/34 - check_doc_counts.py consistent - limitations sync consistent - check_changelog_updated.py passes (existing [Unreleased] entry from rzyns's commit + the additional carry-over from PR aallan#684) - check_site_assets.py up-to-date Co-Authored-By: Claude <noreply@anthropic.invalid>
Summary
KNOWN_ISSUES.mdbug-tracker section. Bug list now empty for the first time since ~v0.0.80.handle | 0x80000000so the in-heap field can never be mistaken for a heap pointer by the conservative GC scan. Unwrap recovers the raw handle with& 0x7FFFFFFF.$alloc: traps ifheap_ptr + total >= 0x80000000. The disjointness invariant (heap_ptr < 2 GB⇒ tagged values>= 2 GBnever collide) now holds by construction.Background
Surfaced by CodeRabbit on PR #577 (#573 phase 1-3 migration). After #573 every
Map<K, V>/Set<T>/Decimalvalue is a pointer to an 8-byte wrapper ADT (tag at offset 0, handle at offset 4). Phase 2b of$gc_collectdoes a conservative word-by-word scan, checking each i32 againstval >= gc_heap_start + 4 && val < heap_ptr && (val - gc_heap_start) & 7 == 4.For typical programs the handle counter stayed below
gc_heap_start(~147 KiB) so the heap-range check rejected it. Long-running programs (>100K handle allocations perexecute()) could see a handle exceed that threshold and (with the right alignment) be falsely classified as a heap pointer — retaining an unrelated heap object. Retention bug, not correctness (no use-after-free, no corruption), but unbounded retention for long sessions.Why Option 1 (self-describing wrappers)
The issue body listed four candidate designs. I scored each against
DESIGN.mdprinciples before picking:heap_ptr < 2 GB) verifiable by inspectionmax_handle > gc_heap_start, handles and heap pointers share the same numeric rangeFull design analysis at [comment in #549 follow-up thread] / before-fix discussion.
Changes
Emission (WAT)
vera/wasm/calls_containers.py::_emit_wrap_handle— emitsi32.const 0x80000000; i32.orbeforei32.store offset=4vera/wasm/calls_containers.py::_emit_unwrap_handle— emitsi32.load offset=4; i32.const 0x7FFFFFFF; i32.andvera/codegen/assembly.py$alloc— heap-ceiling guard right before the bump-advanceHost-side mirrors (Python)
vera/codegen/api.py::_wrap_handle— host-side wrapper allocator also writesraw_handle | 0x80000000vera/wasm/json_serde.py::read_json— masks with0x7FFFFFFFwhen readingJObjectmap handlesvera/wasm/html_serde.py::read_html— masks with0x7FFFFFFFwhen readingHtmlElementmap handlesThe serde sites bypass
_emit_unwrap_handlebecause they read wrapper fields directly via wasmtime memory access during host-side JSON/HTML parsing. They need the same mask.Tests
tests/test_codegen.py::TestWrapperHandleTagging578— 5 new tests:test_wrap_emits_tag_or— structural: WAT containsi32.const 0x80000000andi32.ortest_unwrap_emits_mask_and— structural: WAT containsi32.const 0x7FFFFFFFtest_alloc_emits_heap_ceiling_guard— structural:$allocbody contains the guard patterntest_wrap_unwrap_round_trip_preserves_handle— behavioural: Map insert + lookup round-trip returns the original valuestest_html_round_trip_uses_host_side_mask— behavioural:html_to_string(HtmlElement(..., map_with_attrs, ...))produces correct length output (would fail with empty attrs if the host-side mask were missing)Test plan
pytest tests/— 3,868 passed, 14 skipped, 16 stress-deselectedmypy vera/— cleanruff check --select S vera/— cleanpython scripts/check_conformance.py— 86/86python scripts/check_examples.py— 34/34python scripts/check_doc_counts.py— consistent (3,898 tests / 32 files)python scripts/check_version_sync.py— 0.0.155 across 6 filespython scripts/check_site_assets.py— up-to-datepython scripts/check_limitations_sync.py— consistentDoc sweep
Audited via parallel Explore agent across SKILL/CLAUDE/AGENTS/README/vera/README/FAQ/TESTING/HISTORY/KNOWN_ISSUES/spec/docs/examples/tests. No stale #578 references found. No workaround mentions in examples or test code that need cleaning up.
KNOWN_ISSUES.md— removed Conservative GC scan can spuriously retain heap objects via wrapper handle field (#573 latent) #578 row; bug-tracker section now reads "None currently tracked. The bug-tracker section has been empty before — see HISTORY.md for the bug-killing campaigns that closed each one out."CHANGELOG.mdv0.0.155 entry with the full design rationale + host-side mirrors calloutHISTORY.mdrow added + table footer bumped to "155 tagged releases"TESTING.md+ROADMAP.md+README.mdtest counts: 3,893 → 3,898Closes
Closes #578.
🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests
Documentation