Skip to content

docs: add State handler + where block example to SKILL.md#442

Merged
aallan merged 3 commits into
mainfrom
docs/skill-md-where-block-state
Mar 31, 2026
Merged

docs: add State handler + where block example to SKILL.md#442
aallan merged 3 commits into
mainfrom
docs/skill-md-where-block-state

Conversation

@aallan

@aallan aallan commented Mar 31, 2026

Copy link
Copy Markdown
Owner

Summary

Add a "State handler with a loop helper" subsection to SKILL.md showing the most common State effect pattern: handle[State<Int>] wrapping a where-block helper that has effects(<State<Int>>).

Why this matters

SKILL.md is the sole source of Vera knowledge provided to LLMs during evaluation. This pattern — combining a State handler with a where-block effectful helper — was completely undocumented. The existing where-block examples only show pure functions (is_even/is_odd), and the existing State examples only show inline get/put without helper functions.

Evidence from VeraBench

We A/B tested the improved SKILL.md against the original on two problems that use this pattern:

Problem Original SKILL.md Improved SKILL.md
VB-T5-004 (State accumulator) ❌ check fail ✅ check pass
VB-T5-008 (IO print loop) ❌ check fail ✅ check pass

Both problems went from "can't produce valid Vera syntax at all" to "compiles successfully" with the documentation improvement. The model (Claude Sonnet 4) had no other changes — same prompt, same problem description, only the SKILL.md content differed.

De Bruijn slot ordering is the dominant failure mode in VeraBench (see vera-bench issue 6), but the where-block + effects syntax gap was the top syntax failure — the model couldn't even produce code that parses.

What the example covers

  • Pure outer function with handle[State<Int>] that discharges the effect
  • Where-block helper with effects(<State<Int>>) that calls get/put directly
  • with @Int = @Int.0 clause for updating handler state
  • decreases clause on the loop helper for termination
  • Pure helper function called from within the handler body
  • Visibility rules (no public/private on where-block functions)

Relates to

Test plan

  • The example code passes vera check and vera verify
  • scripts/check_skill_examples.py passes
  • docs/SKILL.md regenerated via scripts/build_site.py

Summary by CodeRabbit

  • Documentation
    • Added a new “State handler with a loop helper” section with examples showing a public-facing state-eliminating function, a where-scoped recursive loop helper that performs get/put and uses resumptions, and a termination hint for the recursive helper.
  • Chores
    • Adjusted example-validation allowlisting and fragment mappings to reflect relocated examples and to change which example blocks are treated as intentionally unparseable.

Add a 'State handler with a loop helper' subsection showing the most
common State pattern: handle[State<Int>] wrapping a where-block helper
that has effects(<State<Int>>).

This pattern was the #1 undocumented syntax that LLMs fail to produce.
VeraBench testing shows:
- Without this example: Claude Sonnet 4 fails check on both VB-T5-004
  (State accumulator) and VB-T5-008 (IO print loop) — can't produce
  valid where-block + effect syntax
- With this example: both problems pass check (100%)

The example covers: pure outer function with handler, effectful where-
block helper with decreases clause, pure helper called from handler
body, with clause for state updates, and visibility rules for where
blocks.

Relates to: aallan/vera-bench#15

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Mar 31, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 11c01cdb-ea3f-4ed1-8858-d3d76f467380

📥 Commits

Reviewing files that changed from the base of the PR and between f8877f1 and 4c02b20.

⛔ Files ignored due to path filters (2)
  • docs/SKILL.md is excluded by !docs/**
  • docs/llms-full.txt is excluded by !docs/**
📒 Files selected for processing (2)
  • SKILL.md
  • scripts/check_skill_examples.py

📝 Walkthrough

Walkthrough

Added a new "State handler with a loop helper" example to SKILL.md showing a top-level pure function eliminated by handle[State<Int>] and a where‑scoped recursive helper that declares effects(<State<Int>>) and uses get/put resumption clauses with an explicit decreases(...) measure. Also updated scripts/check_skill_examples.py allowlist entries to match changed fenced-block line numbers and removed/relocated several allowlisted fragments.

Changes

Cohort / File(s) Summary
Documentation example
SKILL.md
Inserted a new "State handler with a loop helper" snippet: adds example private fn add_value(@int, @int->@int) and public fn sum_with_state(@nat->@int); uses handle[State<Int>](@int = 0) with get(@Unit) -> { resume(@Int.0) } and put(@int) -> { resume(()) } with @int=@Int.0``; includes a where-scoped `sum_loop` helper declaring `effects(<State>)` and a `decreases(...)` clause for recursion termination.
Allowlist / test script
scripts/check_skill_examples.py
Adjusted ALLOWLIST fragments: removed several previously allowlisted fragments, moved and remapped many fragment line-number entries (e.g., "Line comments", "Match arm fragment", "String escapes", "Map/Set common mistakes"), and updated import-syntax allowlist entries to reflect the new SKILL.md fenced-block positions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Suggested labels

docs, ci

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarises the main change: adding a State handler and where-block example to SKILL.md documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/skill-md-where-block-state

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Mar 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.32%. Comparing base (cc51d1b) to head (4c02b20).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #442   +/-   ##
=======================================
  Coverage   90.32%   90.32%           
=======================================
  Files          49       49           
  Lines       19335    19335           
  Branches      220      220           
=======================================
  Hits        17464    17464           
  Misses       1867     1867           
  Partials        4        4           
Flag Coverage Δ
javascript 50.56% <ø> (ø)
python 95.27% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@SKILL.md`:
- Around line 1256-1262: The wording is imprecise: "Pure helper functions (like
add_value) can be called from within the handler body" incorrectly implies
add_value is invoked directly by sum_with_state; update the sentence to state
that pure helpers like add_value can be called from code executed within the
handler's scope (e.g., from the where-function sum_loop) or explicitly "can be
called from the where-function" to reflect that add_value is invoked by sum_loop
rather than directly by the handler body in sum_with_state.
- Line 1249: The call put(add_value(get(()), `@Nat.0`)) passes a Nat where
add_value expects Int; fix by converting the Nat to an Int before calling
add_value—either replace the literal with an Int literal (e.g., `@Int.0`) or wrap
the Nat with the explicit conversion used in this codebase (e.g.,
nat_to_int(`@Nat.0`) or Int::from_nat(`@Nat.0`)), so that add_value(get(()), <Int>)
matches the expected (`@Int`, `@Int` -> `@Int`) signature; update the call site
(add_value/get/put) accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 48df1941-e5e7-46a7-8f0b-e14535c98613

📥 Commits

Reviewing files that changed from the base of the PR and between cc51d1b and 98807ff.

📒 Files selected for processing (1)
  • SKILL.md

Comment thread SKILL.md
Comment thread SKILL.md

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (2)
SKILL.md (2)

1261-1261: ⚠️ Potential issue | 🟡 Minor

Clarify where add_value is called.

Line 1261 states "Pure helper functions (like add_value) can be called from within the handler body", but add_value is actually called from the sum_loop where-function (line 1249), not directly from the handler body (lines 1236–1237). The handler body only calls sum_loop. Consider rephrasing to "can be called from code executed within the handler's scope" or "can be called from the where-function" for precision.

📝 Suggested clarification
-- Pure helper functions (like `add_value`) can be called from within the handler body
+- Pure helper functions (like `add_value`) can be called from the where-function or handler body
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@SKILL.md` at line 1261, Update the phrasing in SKILL.md so it accurately
reflects where add_value is invoked: replace "Pure helper functions (like
`add_value`) can be called from within the handler body" with text that
specifies these helpers can be called from code executed within the handler's
scope (for example, the where-function `sum_loop` that the handler invokes) or
explicitly say "can be called from the where-function (e.g., `sum_loop`) rather
than directly from the handler body"; reference the symbols `add_value` and
`sum_loop` to guide the change.

1249-1249: ⚠️ Potential issue | 🔴 Critical

Type mismatch: Nat passed where Int expected.

Line 1249 calls add_value(get(()), @Nat.0) where get(()) returns @Int and @Nat.0 (sum_loop's second parameter) is a Nat, but add_value requires (@int, @int->@int). Vera requires explicit conversion between Nat and Int—there is no automatic coercion.

🔧 Proposed fix
-      put(add_value(get(()), `@Nat.0`));
+      put(add_value(get(()), nat_to_int(`@Nat.0`)));
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@SKILL.md` at line 1249, The call passes a Nat where add_value expects an Int;
wrap the Nat (sum_loop's second parameter `@Nat.0`) in an explicit conversion to
Int before passing it to add_value (so the second argument to add_value is of
type `@Int`). Locate the call to add_value(get(()), `@Nat.0`) and replace the raw
`@Nat.0` with the appropriate explicit conversion (e.g., Int.from_nat or your
project's nat-to-int converter) so add_value, get, and sum_loop types align.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@SKILL.md`:
- Line 1261: Update the phrasing in SKILL.md so it accurately reflects where
add_value is invoked: replace "Pure helper functions (like `add_value`) can be
called from within the handler body" with text that specifies these helpers can
be called from code executed within the handler's scope (for example, the
where-function `sum_loop` that the handler invokes) or explicitly say "can be
called from the where-function (e.g., `sum_loop`) rather than directly from the
handler body"; reference the symbols `add_value` and `sum_loop` to guide the
change.
- Line 1249: The call passes a Nat where add_value expects an Int; wrap the Nat
(sum_loop's second parameter `@Nat.0`) in an explicit conversion to Int before
passing it to add_value (so the second argument to add_value is of type `@Int`).
Locate the call to add_value(get(()), `@Nat.0`) and replace the raw `@Nat.0` with
the appropriate explicit conversion (e.g., Int.from_nat or your project's
nat-to-int converter) so add_value, get, and sum_loop types align.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c71e424a-3a57-4949-8fdd-544c5fc7be91

📥 Commits

Reviewing files that changed from the base of the PR and between 98807ff and f8877f1.

📒 Files selected for processing (1)
  • SKILL.md

@aallan aallan added the vera-bench The issue or PR originated due to aallan/vera-bench label Mar 31, 2026
The 51-line "State handler with a loop helper" insertion shifted all
code-block line numbers after line 1213, making 19 allowlist entries in
check_skill_examples.py stale. Update them to their new positions, remove
entries whose blocks now parse cleanly, and add the one new entry needed
(Correct: import syntax example at 1681).

Also clarify the "Pure helper functions" key point: add_value is called
from the `where` block helper (sum_loop), not from the handler body
directly.

Regenerate docs/ site assets to include the new SKILL.md section.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan aallan added the documentation Improvements or additions to documentation label Mar 31, 2026
@aallan aallan merged commit 3995d25 into main Mar 31, 2026
19 checks passed
@aallan aallan deleted the docs/skill-md-where-block-state branch March 31, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation vera-bench The issue or PR originated due to aallan/vera-bench

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant