Fix #597: walker-completeness audit with pre-commit enforcement#668
Conversation
Every Tier-A bug shipped in v0.0.145..150 (#604, #655, #628, #648, #559) had the same shape: a walker function dispatching on `Expr` subclasses handled N of N+1 cases, the missing case silently fell through to the default (None / False / no-op), and the enclosing function silently produced wrong output. This PR closes the bug class by making "did you handle every subclass?" a mechanically checkable contract. What's in the PR: 1. **`# WALKER_COVERAGE:` checklist comments** on 9 walker functions, classifying every one of the 29 Expr subclasses declared in `vera/ast.py` with one of four dispositions: - **Handled** — explicit `isinstance` branch - **Intentionally ignored** — default fall-through is correct (e.g. literals in a sub-expression-recursing walker) - **Cannot occur** — structurally impossible (e.g. OldExpr in body-only walkers, HoleExpr post-typecheck) - **MISSING** — open bug, branch should exist 2. **`scripts/check_walker_coverage.py`** — parses each walker's `isinstance(_, ast.X)` calls AND its checklist comment text, then verifies the union covers every Expr subclass. Reports any gap as `name (lineno): N subclass(es) not in dispatch or checklist`. 3. **`walker-coverage` pre-commit hook** — runs the script on every commit touching `vera/ast.py`, walker files, or the script itself. A new Expr subclass added to `ast.py` forces every walker to either handle it or document its disposition. 4. **11 defensive `isinstance` branches** added to harden against future regressions of the bug class. Masked today by upstream guards (type checker rejection, [E602] codegen-skip, closure pipeline sibling scans, translator-side registration in calls_math/calls_containers), but plugs the gap if any upstream mechanism is relaxed: - `_scan_io_ops` (compilability.py): +4 (IndexExpr, ArrayLit, InterpolatedString, AnonFn body recursion) - `_scan_expr_for_handlers` (compilability.py): +5 (same shape plus QualifiedCall args) - `_infer_expr_wasm_type` (inference.py): +2 (AnonFn → i32, ModuleCall → fncall-shape lookup) - `_infer_vera_type` (inference.py): +8 (Block, MatchExpr, HandleExpr, AssertExpr, AssumeExpr, AnonFn, QualifiedCall, ModuleCall — defensive `Fn`/`Unit`/return-type fallbacks) 5. **`vera/README.md` "Walker-completeness convention" section** documenting the contract for future contributors. Also adds a step-5 note in "New AST node" reminding contributors that a new Expr subclass requires every walker checklist to be updated. 6. **#604 ROADMAP cleanup** — removes the stale Order-1 row that PR #659 had closed via code fix but didn't delete from ROADMAP.md. Stabilisation tier renumbered 1-6 (was 2-7); Agent-integration tier renumbered 7-9 (was 8-10). 7. **#667 filed and added to ROADMAP** as new Order 6 — SMT translator coverage expansion (FloatLit / ArrayLit / IndexExpr). Surfaced as latent gap during the walker audit; closing requires extending the contract grammar (parser + checker work) which is out of scope for this PR per Option A's scope decision. Release prep - Version bumped to v0.0.151 across __init__.py, pyproject.toml, uv.lock, docs/index.html, README.md. - CHANGELOG.md entry under [0.0.151] with Added + Fixed + Internal sections. - HISTORY.md row added (one-line template-compliant). - TESTING.md + CONTRIBUTING.md hook count bumped 26 → 27. - TESTING.md hook-table row added for `check_walker_coverage.py`. Validation - `python scripts/check_walker_coverage.py` → 9 walkers cover all 29 Expr subclasses (clean) - `pytest tests/ -q` → 3,813 passed, 14 skipped - `mypy vera/` → clean - `python scripts/check_doc_counts.py` → consistent (27 hooks) - `python scripts/check_version_sync.py` → consistent across 6 files - Doc sweep across SKILL / CLAUDE / AGENTS / README / vera/README / spec / docs / examples / tests → zero stale workarounds, all references appropriately tense-framed Closes #597 Co-Authored-By: Claude <noreply@anthropic.invalid>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a walker-completeness audit: a new enforcement script and pre-commit hook, per-walker ChangesWalker-completeness audit with pre-commit enforcement
Sequence Diagram sequenceDiagram
participant Dev as Developer
participant PreCommit as .pre-commit-config.yaml
participant Script as scripts/check_walker_coverage.py
participant AST as vera/ast.py
participant Walkers as vera/**/*.py
Dev->>PreCommit: commit/push triggers hooks
PreCommit->>Script: run check_walker_coverage.py
Script->>AST: parse & extract Expr subclasses
Script->>Walkers: discover functions with "WALKER_COVERAGE:" marker
Walkers->>Script: return function bodies
Script->>Script: compute coverage gaps
Script->>PreCommit: exit non-zero on gaps (fail hook)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested labels
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #668 +/- ##
==========================================
+ Coverage 91.06% 91.08% +0.01%
==========================================
Files 60 60
Lines 23240 23283 +43
Branches 259 259
==========================================
+ Hits 21164 21207 +43
Misses 2069 2069
Partials 7 7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CONTRIBUTING.md`:
- Around line 88-90: The commit-stage hook count is inconsistent: change the
parenthetical in the heading text "**commit-stage** hooks (24, each gated to
relevant files) include:" to read 25 so it matches the earlier sentence that
says "25 are configured at the commit stage"; ensure the numeric count in that
exact phrase is updated to 25 and leave the rest of the paragraph unchanged.
In `@scripts/check_walker_coverage.py`:
- Around line 167-174: The hardcoded WALKER_FILES list in
scripts/check_walker_coverage.py is brittle; replace manual maintenance by
auto-discovering walker files: scan the repo for all vera/**/*.py files and
select those that contain the "WALKER_COVERAGE:" marker, then build WALKER_FILES
from those paths instead of the current static list (use the existing
WALKER_FILES symbol as the target variable to populate and keep the same
downstream behavior); ensure the discovery handles ROOT path resolution and
preserves the current relative/path objects type so other functions that
reference WALKER_FILES continue to work.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 28e8be19-8060-4d7b-aad2-c94e1e6c956f
⛔ Files ignored due to path filters (5)
docs/index.htmlis excluded by!docs/**docs/index.mdis excluded by!docs/**docs/llms-full.txtis excluded by!docs/**docs/llms.txtis excluded by!docs/**uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (17)
.pre-commit-config.yamlCHANGELOG.mdCONTRIBUTING.mdHISTORY.mdREADME.mdROADMAP.mdTESTING.mdpyproject.tomlscripts/check_walker_coverage.pyvera/README.mdvera/__init__.pyvera/checker/expressions.pyvera/codegen/compilability.pyvera/smt.pyvera/wasm/closures.pyvera/wasm/context.pyvera/wasm/inference.py
CR-1 (`CONTRIBUTING.md:90`) — commit-stage hook count drift. The heading "**commit-stage** hooks (24, each gated to relevant files) include:" disagreed with the immediately-preceding sentence "25 are configured at the commit stage". Updated to match (24 -> 25). Pure doc-consistency fix. CR-2 (`scripts/check_walker_coverage.py:167`) — auto-discover walker files instead of hardcoded list. The hardcoded `WALKER_FILES` list silently skipped any new walker file added without updating the list — replicating the exact silent-skip class this script was written to close. Verified by inspection: nothing forced contributors adding a new walker file to also touch this list. Replaced with `_discover_walker_files()` which globs `vera/**/*.py` for files containing the `WALKER_COVERAGE:` marker. Returns a sorted `list[Path]` (same shape as the hardcoded list it replaces) so downstream code referencing `WALKER_FILES` is unchanged. Validation - `python scripts/check_walker_coverage.py` -> 9 walkers cover all 29 Expr subclasses (clean — same output as pre-fix) - Discovered file list matches the pre-fix hardcoded list exactly (6 files, same paths) This also addresses the #1 critical finding from the pr-review-toolkit audit run before this commit. Refs #597 #668 Co-Authored-By: Claude <noreply@anthropic.invalid>
pr-review-toolkit audit outcomeRan the comprehensive PR review (4 specialised agents in parallel) on Resolved in
|
| # | Finding | Severity | Notes |
|---|---|---|---|
| 2 | ModuleCall defensive branch in _infer_expr_wasm_type / _infer_vera_type synthesises a fake FnCall(name, args) discarding expr.path |
Critical | Masked today because ModuleCall is normally desugared to FnCall before reaching these helpers. If un-masked, the lookup could find a same-name local fn from a different module. Fix shape: either properly resolve via the imported-module registry, or return None to surface the failure rather than mask it with a wrong lookup. |
| 3 | AnonFn → "Fn" returns a placeholder string that isn't a real Vera type |
Critical | Masked today because AnonFn is typically lifted before reaching _infer_vera_type. Fix: return None. |
| 4 | CHECKLIST_LINE_RE unanchored — matches any `# Word - |
→ | :line in function body, not just theWALKER_COVERAGE:` block |
| 5 | _scan_io_ops / _scan_expr_for_handlers defensive-add comments incorrectly claim "masked by closure pipeline running its own scan" |
Important | Comment-only — _compile_lifted_closure does NOT call these scanners; the defensive branches ARE the primary defence, not redundant. Worth correcting in the follow-up. |
| 6 | Defensive branches have 0% test coverage (synthetic-AST invocation would exercise them directly) | Important | A future refactor breaking a defensive branch lands silently — no production path exercises them today. |
| 7 | No unit tests for scripts/check_walker_coverage.py |
Important | The script has nontrivial parsing logic with real failure modes (the just-fixed CHECKLIST_LINE_RE anchoring would be one such case). |
| 8 | Dead if expr.expr is not None guards on non-Optional AST fields (Block.expr, HandleExpr.body.expr) |
Important | The guards are dead defensive code — vera/ast.py:470, 481 shows both are non-Optional. Should be assert or removed. |
Strengths confirmed by the audit
- Zero false-positive "Handled" claims across all 9 walker checklists — every
Handledrow has a realisinstancebranch (comment-analyzer verified). - All 3,813 existing tests pass unchanged.
_synth_exprcorrectly serves as the canonical reference set with no Cannot-occur entries.- Issue Expand smt.translate_expr coverage: FloatLit, ArrayLit, IndexExpr #667 citation in
smt.pyaccurately matches the filed issue.
Recommendation
Items 2-8 are real but their fix shapes are heterogeneous (some are comment-only, some are 2-line code edits, some require new test files). Proposing to file a single follow-up issue ("Walker-audit polish — pr-review #668 items 2-8") and tackle it as one focused PR after #668 merges. Net effect on this PR: the critical-and-also-CodeRabbit-flagged item (#1) is fixed; the rest are documented as deferred.
Open to feedback if any of the deferred items should be pulled in before merge.
🤖 Generated with Claude Code
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/check_walker_coverage.py`:
- Around line 146-159: The current extract_checklist_classes searches the whole
body_src and can match checklist-style comments outside the WALKER_COVERAGE
section; modify extract_checklist_classes to first locate the "WALKER_COVERAGE"
marker (e.g. search for the literal "WALKER_COVERAGE" or the marker line used
elsewhere), then slice body_src from that marker onward and restrict the regex
scan to the contiguous checklist block (stop when a non-matching line is
encountered), and finally run CHECKLIST_LINE_RE.findall on that sliced segment
so only entries inside the WALKER_COVERAGE block are returned.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c7cf1c6a-6f84-45c6-93ad-eac002465fb9
📒 Files selected for processing (2)
CONTRIBUTING.mdscripts/check_walker_coverage.py
… block CR-3 (`scripts/check_walker_coverage.py:146-159`) — restrict `extract_checklist_classes` to the WALKER_COVERAGE block only. Pre-fix the helper ran `CHECKLIST_LINE_RE.findall(body_src)` over the entire walker function body, including any function code after the docstring close. A `# Foo → bar`-shaped comment elsewhere in the function body could silently count as "checklist coverage" for a class named Foo — the exact silent- skip class this script was written to close. Also independently flagged as item #4 in the pr-review-toolkit audit posted earlier on this PR. Post-fix `extract_checklist_classes` first locates the `WALKER_COVERAGE:` marker, then slices `body_src` from there through the closing `"""` of the enclosing docstring (since our convention puts WALKER_COVERAGE at the trailing end of the docstring, the next `"""` is the natural block terminator). The regex then scans only that slice. Edge-case behaviour: - No marker found → caller already filters to walkers containing the marker, but defensive `return set()` covers the case anyway. - No closing `"""` after the marker → falls back to end-of-body. Single-quoted triple delimiters `'''` are not used by any walker today, so the fallback preserves the pre-fix coverage detection if convention drifts; the cost is accepting matches in post-block code, same as pre-fix. Validation - `python scripts/check_walker_coverage.py` → 9 walkers cover all 29 Expr subclasses (clean — same output as before) - Spot-check: every WALKER_COVERAGE block correctly reports 29 checklist entries (one per Expr subclass) via the anchored extraction Refs #597 #668 Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/check_walker_coverage.py`:
- Around line 104-106: The type annotations for walkers and the function
parameter currently reference only ast.FunctionDef but your loop handles
ast.AsyncFunctionDef as well; update the annotation for walkers (currently
"walkers: list[tuple[ast.FunctionDef, str]]") to include both types (e.g., use
ast.FunctionDef | ast.AsyncFunctionDef or typing.Union[ast.FunctionDef,
ast.AsyncFunctionDef]), and change the corresponding parameter type annotation
(the function that accepts the walker tuples around line 120) to the same
combined type so mypy sees both ast.FunctionDef and ast.AsyncFunctionDef are
supported.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 79ba044b-2a83-4377-a08d-ceb97f087b65
📒 Files selected for processing (1)
scripts/check_walker_coverage.py
Following user direction to pull all 7 deferred pr-review findings into this PR rather than file a follow-up. The 4 small code fixes (items 2/3/5/8) plus 2 new test files (items 6/7) land here. Code fixes: - **#2 ModuleCall path-drop** (`vera/wasm/inference.py:289-292` and `:~973`) — both `_infer_expr_wasm_type::ModuleCall` and `_infer_vera_type::ModuleCall` previously synthesised a fake `ast.FnCall(name=expr.name, args=expr.args)` for dispatch, silently dropping `expr.path: tuple[str, ...]`. If a regression ever flowed a ModuleCall to either helper, the fake-FnCall lookup could match a same-name local fn from a different module — silent wrong-answer rather than safe failure. Now both return `None` so the unknown-type surfaces cleanly. - **#3 AnonFn placeholder** (`vera/wasm/inference.py:~967`) — `_infer_vera_type::AnonFn` previously returned the literal string `"Fn"` as a placeholder. No callsite recognised `"Fn"` as a real Vera type; downstream type-arg mangling paths (`vera/wasm/calls.py:1525,1533`) would feed it into mangled names like `option_map$Int_Fn`. Now returns `None` for the same reason. - **#5 Factually wrong "closure pipeline" comments** (`vera/codegen/compilability.py:~236, ~393`, both WALKER_ COVERAGE checklists + inline pre-branch comments) — the `AnonFn` defensive branches were described as "masked today by closure pipeline running its own scan", but pr-review surfaced that `vera/codegen/closures.py::_compile_lifted_ closure` does NOT call `_scan_io_ops` or `_scan_body_for_state_handlers` on lifted bodies. The AnonFn branch is the PRIMARY defence, not redundant. Comments now state this directly. - **#8 Dead `is not None` guards** (`vera/wasm/inference.py:~954, ~961`) — `Block.expr` and `HandleExpr.body.expr` are non- Optional in the AST schema (`vera/ast.py:470, 481`). The guards in the `_infer_vera_type` defensive branches were unreachable defensive code. Removed; direct calls now. Test additions: - **#6 Synthetic-AST tests for defensive branches** (`tests/test_walker_defensive_branches_597.py`, 21 tests, 296 lines) — direct AST invocation pinning each of the 11 defensive branches plus the 5 pr-review fixes. Without these the defensive branches have 0% coverage (`coverage run` confirmed) — a future refactor breaking one would land silently. - **#7 Unit tests for the enforcement script** (`tests/test_check_walker_coverage_597.py`, 12 tests, 255 lines) — pins the script's parsing logic: Expr subclass extraction, isinstance flattening (incl. tuple form), checklist-block anchoring (incl. CR-3 regression test: `# Foo → bar` outside the WALKER_COVERAGE block must not be counted), section-header tolerance, auto-discovery invariants, end-to-end exit code. CHANGELOG/HISTORY: - Extended the v0.0.151 entry with two new sub-sections under "Fixed" (pr-review follow-ups) and a new "Tests" section documenting the two regression-test files. Doc counts (auto-validated by `check_doc_counts.py`): - TESTING.md total: 3,827 → 3,860 tests (+33), 29 → 31 files - TESTING.md table: two new rows for the test files - ROADMAP.md: 3,827 → 3,860 - README.md: 3,827 → 3,860 Validation: - `pytest tests/ -q` → 3,846 passed, 14 skipped (+33 net new) - `mypy vera/` → clean - `python scripts/check_walker_coverage.py` → 9 walkers cover all 29 Expr subclasses (clean) - `python scripts/check_doc_counts.py` → consistent across all surfaces Refs #597 #668 Co-Authored-By: Claude <noreply@anthropic.invalid>
All pr-review items now closed in this PRPer direction to pull all 7 deferred findings forward rather than file a follow-up. Landed in Code fixes (4 items, ~25 LOC net change)
Test additions (2 files, 33 new tests, ~551 LOC)
Validation
Score-cardCombined with the earlier
The follow-up issue I'd proposed earlier is no longer needed — everything that came out of the pr-review-toolkit run plus all 3 CodeRabbit rounds is in this PR. 🤖 Generated with Claude Code |
…nctionDef
CR-4 (`scripts/check_walker_coverage.py:95, 104, 120, 231`) —
type-annotation narrowing inconsistency.
The runtime walker loop at line 106 accepts both
`ast.FunctionDef` and `ast.AsyncFunctionDef`:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
but four function signatures + local declarations narrow the
type to `ast.FunctionDef` only:
- `find_walker_functions` return type
- `walkers` list-declaration inside `find_walker_functions`
- `extract_isinstance_classes` parameter type
- `walkers` list-declaration inside `main`
This passes mypy today because Python's `ast` module's type
relationship between `FunctionDef` and `AsyncFunctionDef`
doesn't trigger strict variance checks here, but the
inconsistency is documentation-misleading — a reader looking
at the signatures would conclude async-def walkers aren't
supported, while at runtime they are.
Fix: widen all four annotations to
`ast.FunctionDef | ast.AsyncFunctionDef`. Mechanical change,
no behaviour difference.
Validation
- `python scripts/check_walker_coverage.py` -> clean (9 walkers
cover all 29 Expr subclasses)
- `pytest tests/test_check_walker_coverage_597.py
tests/test_walker_defensive_branches_597.py -q` -> 33 passed
Refs #597 #668
Co-Authored-By: Claude <noreply@anthropic.invalid>
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/check_walker_coverage.py`:
- Around line 129-147: extract_isinstance_classes currently uses
ast.walk(fn_node) which also visits nested FunctionDef/AsyncFunctionDef/ClassDef
bodies and thus incorrectly collects isinstance(..., ast.X) from inner scopes;
replace the walk with a custom ast.NodeVisitor that implements visit_Call to
detect isinstance calls (check sub.func is ast.Name with id "isinstance", has
>=2 args, inspect args[1] for ast.Tuple or single target and collect
ast.Attribute nodes whose value is ast.Name id "ast" into the classes set) and
override visit_FunctionDef, visit_AsyncFunctionDef, and visit_ClassDef to return
without calling generic_visit so nested scopes are skipped, then run that
visitor on fn_node and return the collected classes.
In `@TESTING.md`:
- Line 472: The "Validation Scripts" inventory is out of sync with the addition
of check_walker_coverage.py; update the README text and table to reflect the new
script by changing the count from "Twelve scripts" to "Thirteen scripts" (or
adjust phrasing to avoid a hard count) and insert a table row or list entry for
check_walker_coverage.py with its description (e.g., "Every walker function
covers every Expr subclass via dispatch or checklist comment"); ensure any
referenced totals or indices elsewhere in TESTING.md are similarly updated so
the document is internally consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: b224f93b-932a-45aa-9dfe-03f2462351d3
📒 Files selected for processing (9)
CHANGELOG.mdREADME.mdROADMAP.mdTESTING.mdscripts/check_walker_coverage.pytests/test_check_walker_coverage_597.pytests/test_walker_defensive_branches_597.pyvera/codegen/compilability.pyvera/wasm/inference.py
…s inventory count CR-5 (`scripts/check_walker_coverage.py:129-147`) — make isinstance collection scope-aware. Pre-fix `extract_isinstance_classes` used `ast.walk(fn_node)`, which recursively visits nested `FunctionDef` / `AsyncFunctionDef` / `ClassDef` bodies. No walker has nested helpers today, but a future refactor introducing one (e.g. a local helper with its own `isinstance(x, ast.SomeExpr)` check) would have its inner isinstance counted as outer-walker coverage — silently making the outer walker appear to cover a class it doesn't actually handle. Replaced with `_IsinstanceCollector(ast.NodeVisitor)` that: - Overrides `visit_Call` to detect `isinstance(_, ast.X)` / `isinstance(_, (ast.X, ast.Y))` patterns - Overrides `visit_FunctionDef`, `visit_AsyncFunctionDef`, and `visit_ClassDef` to return without `generic_visit`, pruning nested scopes - Entry is `visit` on each body statement (skipping fn_node itself, which would trigger the FunctionDef pruning) 3 regression tests added in `tests/test_check_walker_coverage_597.py`: - `test_nested_function_isinstance_not_counted` - `test_nested_async_function_isinstance_not_counted` - `test_nested_class_isinstance_not_counted` Each constructs a synthetic walker with a nested def/async-def/class containing `isinstance(x, ast.BinaryExpr)`, and asserts only the outer walker's isinstance (`ast.IntLit`) is collected. CR-6 (`TESTING.md:416`) — Validation Scripts count drift. `Twelve scripts` -> `Thirteen scripts` to reflect `check_walker_coverage.py` added by this PR. The table row for the script was already added in the earlier commit; only the prose count needed bumping. Doc counts auto-updated: - TESTING.md total tests: 3,860 -> 3,863 (+3 CR-5 regression tests) - TESTING.md `test_check_walker_coverage_597.py` row: 12 -> 15 tests, 255 -> 311 lines - ROADMAP.md / README.md: 3,860 -> 3,863 Validation - `python scripts/check_walker_coverage.py` -> 9 walkers cover all 29 Expr subclasses (clean — same as pre-fix) - `pytest tests/test_check_walker_coverage_597.py tests/test_walker_defensive_branches_597.py -v` -> 36 passed - `python scripts/check_doc_counts.py` -> consistent Refs #597 #668 Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
TESTING.md (1)
472-472:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winThe validation scripts inventory in the "Validation Scripts" section is still out of sync.
Line 472 correctly adds
check_walker_coverage.pyto the pre-commit hooks table, but the "Validation Scripts" section (lines 414-432) still states "Twelve scripts" without including the new checker. This inconsistency was previously flagged and remains unresolved.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@TESTING.md` at line 472, Update the "Validation Scripts" section in TESTING.md to reflect the added check_walker_coverage.py: change the count text from "Twelve scripts" to "Thirteen scripts" and add the new script entry (check_walker_coverage.py) to the inventory/list so it matches the pre-commit hooks table; ensure the section header "Validation Scripts" and its descriptive sentence and list/table include the new checker name and any brief description used for other entries so both lists stay consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.pre-commit-config.yaml:
- Line 127: The pre-commit hook's files pattern (the 'files' entry) is too
narrow and should be widened to include all Python under vera so walker markers
outside the listed subpaths still trigger the hook; update the 'files' regex
used in .pre-commit-config.yaml (the files:
'(vera/ast\.py$|vera/wasm/.*\.py$|vera/checker/.*\.py$|vera/codegen/.*\.py$|vera/smt\.py$|scripts/check_walker_coverage\.py$)')
to match vera/**/*.py (or an equivalent regex covering all vera/*.py and
subdirectories) so it aligns with the checker’s walker discovery.
---
Duplicate comments:
In `@TESTING.md`:
- Line 472: Update the "Validation Scripts" section in TESTING.md to reflect the
added check_walker_coverage.py: change the count text from "Twelve scripts" to
"Thirteen scripts" and add the new script entry (check_walker_coverage.py) to
the inventory/list so it matches the pre-commit hooks table; ensure the section
header "Validation Scripts" and its descriptive sentence and list/table include
the new checker name and any brief description used for other entries so both
lists stay consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 81e39273-bfce-44f8-ad4e-76df4b6f2945
⛔ Files ignored due to path filters (5)
docs/index.htmlis excluded by!docs/**docs/index.mdis excluded by!docs/**docs/llms-full.txtis excluded by!docs/**docs/llms.txtis excluded by!docs/**uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (19)
.pre-commit-config.yamlCHANGELOG.mdCONTRIBUTING.mdHISTORY.mdREADME.mdROADMAP.mdTESTING.mdpyproject.tomlscripts/check_walker_coverage.pytests/test_check_walker_coverage_597.pytests/test_walker_defensive_branches_597.pyvera/README.mdvera/__init__.pyvera/checker/expressions.pyvera/codegen/compilability.pyvera/smt.pyvera/wasm/closures.pyvera/wasm/context.pyvera/wasm/inference.py
CR-7 (`.pre-commit-config.yaml:127`) — pre-commit hook trigger
regex narrower than script's auto-discovery glob.
The walker-coverage script auto-discovers via
`(ROOT / "vera").rglob("*.py")` (any Python file under `vera/`
containing the `WALKER_COVERAGE:` marker — fix that already
landed for CR-2). The pre-commit hook's `files:` regex,
however, restricted triggering to a hardcoded list of subpaths
(`vera/ast.py`, `vera/wasm/.*\.py`, `vera/checker/.*\.py`,
`vera/codegen/.*\.py`, `vera/smt.py`, plus the script itself).
All 6 current walker files match the existing regex by
coincidence — verified by cross-checking each discovered path
against the regex. But a future walker landing outside the
listed subpaths (e.g. `vera/transform.py` or `vera/parser/foo.py`)
would be auto-discovered by the script but edits to it wouldn't
trigger the hook — replicating the registry-drift silent-skip
class this PR is meant to close, just at a different layer.
Fix: widen the regex to `(vera/.*\.py$|scripts/check_walker_coverage\.py$)`
so it matches every Python file under `vera/`, plus the
enforcement script itself. Verified:
- All 6 current walker files still match
- Hypothetical future locations (`vera/transform.py`,
`vera/parser/foo.py`, `vera/__init__.py`) now match
- Non-walker files (`tests/test_foo.py`, `README.md`) still
don't match
Skipping the duplicate CR-6 finding (TESTING.md "Twelve scripts"
→ "Thirteen scripts") — already fixed in `c1321c7`; current
state at TESTING.md:416 reads "Thirteen scripts" as required.
Validation
- Regex spot-check on 12 paths (6 actual walkers + 3 hypothetical
future locations + 3 non-walker controls) — all match the
expected behaviour
- `python scripts/check_walker_coverage.py` -> clean (9 walkers
cover all 29 Expr subclasses)
- `python scripts/check_doc_counts.py` -> consistent
Refs #597 #668
Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
♻️ Duplicate comments (1)
TESTING.md (1)
416-432:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAlign the “Thirteen scripts” claim with the table contents.
This section states 13 scripts, but the table currently enumerates 12. Please add the missing
check_walker_coverage.pyrow (or adjust the count).Suggested patch
| `check_doc_counts.py` | Counts cited in TESTING.md, CONTRIBUTING.md, and CLAUDE.md match live codebase | +| `check_walker_coverage.py` | Every walker function covers every `Expr` subclass via dispatch or checklist comment | | `check_licenses.py` | All installed packages have MIT-compatible licenses | | `fix_allowlists.py` | Auto-fix stale allowlist line numbers after Markdown edits |As per coding guidelines
**/*.md: "Review Markdown files for factual accuracy against the codebase, broken links, and outdated information."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@TESTING.md` around lines 416 - 432, The heading in TESTING.md claims "Thirteen scripts" but the table lists only 12; update TESTING.md to either add a table row for the missing script `check_walker_coverage.py` (brief description: validates walker coverage metrics) or change the heading to "Twelve scripts" so they match; locate the table in the shown diff and add the `check_walker_coverage.py` entry (or adjust the count) and ensure the description follows the same style as other rows, referencing the actual script name `check_walker_coverage.py` to keep documentation consistent with the codebase.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@TESTING.md`:
- Around line 416-432: The heading in TESTING.md claims "Thirteen scripts" but
the table lists only 12; update TESTING.md to either add a table row for the
missing script `check_walker_coverage.py` (brief description: validates walker
coverage metrics) or change the heading to "Twelve scripts" so they match;
locate the table in the shown diff and add the `check_walker_coverage.py` entry
(or adjust the count) and ensure the description follows the same style as other
rows, referencing the actual script name `check_walker_coverage.py` to keep
documentation consistent with the codebase.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 320f7319-fc7d-4295-a556-40971f937160
📒 Files selected for processing (6)
.pre-commit-config.yamlREADME.mdROADMAP.mdTESTING.mdscripts/check_walker_coverage.pytests/test_check_walker_coverage_597.py
CR-8 (`TESTING.md:416-432`) — heading-vs-table mismatch. My earlier CR-6 fix in `c1321c7` bumped the prose count from "Twelve scripts" to "Thirteen scripts" but I incorrectly claimed the table row was "already added in an earlier commit". The CHANGELOG-style row I'd added (line 472, in the pre-commit hooks list) was for a different table. The "Validation Scripts" inventory table at lines 418-432 was still at 12 rows. Added the missing row for `check_walker_coverage.py` between `check_doc_counts.py` and `check_licenses.py` (grouping it with the other "doc / cross-tree consistency" checks), with a description matching the style of adjacent rows. Validation - TESTING.md table now lists 13 scripts, matching the heading - `python scripts/check_doc_counts.py` -> consistent Refs #597 #668 Co-Authored-By: Claude <noreply@anthropic.invalid>
|
@coderabbitai You were right, and my earlier reply was wrong. Applied in Validated: |
|
Tip For best results, initiate chat on the files or code changes.
|
Summary
Closes the bug class responsible for every Tier-A fix shipped in v0.0.145..150 (#604, #655, #628, #648, #559). All five had the same shape: a walker function handled N of N+1
Exprsubclasses, the missing case silently fell through, and the enclosing function produced wrong output. This PR makes "did you handle every subclass?" a mechanically checkable contract.# WALKER_COVERAGE:checklist comments on 9 walker functions classifying every one of the 29Exprsubclasses with one of four dispositions: Handled / Intentionally ignored / Cannot occur / MISSING.scripts/check_walker_coverage.py— parses each walker'sisinstance(_, ast.X)calls AND its checklist comment, verifies the union covers everyExprsubclass.walker-coveragepre-commit hook — a newExprsubclass added toast.pynow forces every walker to either handle it or document its disposition before the commit lands.isinstancebranches added across_scan_io_ops(+4),_scan_expr_for_handlers(+5),_infer_expr_wasm_type(+2),_infer_vera_type(+8). Masked today by upstream guards; plugs the gap if any upstream mechanism relaxes.vera/README.md"Walker-completeness convention" section documenting the contract for future contributors.Audit findings
105 mechanical-missing cells across 9 walkers × 29 subclasses, broken down as:
isinstancebranchesTest plan
python scripts/check_walker_coverage.py→ 9 walkers cover all 29Exprsubclasses (clean)pytest tests/ -q→ 3,813 passed, 14 skippedmypy vera/→ cleanwalker-coveragehook) → all greenpython scripts/check_doc_counts.py→ consistent (27 hooks)Closes #597
Summary by CodeRabbit
Chores
Documentation
Tests
Bug Fixes / Robustness