Skip to content

docs(site): redesign veralang.dev homepage; close Agent Score gaps#526

Merged
aallan merged 7 commits into
mainfrom
redesign-index
Apr 24, 2026
Merged

docs(site): redesign veralang.dev homepage; close Agent Score gaps#526
aallan merged 7 commits into
mainfrom
redesign-index

Conversation

@aallan

@aallan aallan commented Apr 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • Full redesign of docs/index.html with an editorial-research aesthetic — tighter typographic hierarchy, two weighty anchors (the "Why?" thesis and the code showcase), VeraBench lifted and given a masthead stat + mascot lockup, reference grid condensed from 17 features to 9, dark "For Agents" section structurally framing the page as a machine-readable specification.
  • A quiet structural device at the top of the page — @reader.0 → humans / @reader.1 → agents — uses Vera's own slot-reference syntax to acknowledge the dual-audience architecture without a banner or pop-up. The agent-facing metadata (alternate links, llms-txt directives, schema.org JSON-LD, inline <script type="text/llms.txt">) is preserved intact.
  • Agent Score moved from 3 failures + 2 warnings to 2 failures + 1 warning via both the HTML redesign and a rewrite of scripts/build_site.py's build_llms_txt() + build_index_md() generators. Three remaining checks tracked for follow-up in veralang.dev: close remaining Agent Score gaps (content negotiation, truncation budget, markdown parity) #525.

Design intents (for the record)

  1. Section reweighting. Why section and code showcase promoted as anchors; VeraBench lifted with a display-type headline stat; reference material condensed (17 features → 9, 6 principles unchanged) so it stops competing with the thesis.

  2. Bilingual reading-path device. The @reader.0 → humans / @reader.1 → agents strip at the top of the page. Acknowledges the dual audience structurally, in Vera's own syntax.

  3. Three-font hierarchy rule.

    • DM Serif Display — Vera asserting something (hero subhead, h2s, thesis callout, VeraBench stat)
    • Inter — Vera explaining something (body, captions, UI labels)
    • JetBrains Mono — machine surfaces (code, @section.* eyebrows, the readpath)

    Old muddy h3 role split into small-caps sans labels so sections read as labelled, not titled.

  4. Palette aligned to Negroni's Vermouth + Campari scales. #FFECD1 matches the hero meerkat's baked cream (so the hero card blends into the page rather than sitting on a darker surface); #FFE2CE becomes the contrast surface for alternating-section rhythm; #FEEAD1 pinned to the VeraBench section to match its chart/mascot baked creams exactly (different shade because those assets were produced separately). The section backgrounds now alternate paper → gradient → dark → paper → paper-hi → paper-lo → paper → dark → paper. No adjacent sections share a tone.

  5. Code showcase stacked, not side-by-side. Annotations sit below each sample with an orange left-rule rather than in a narrow sidebar column, so long lines (e.g. research_topic's URL concat) don't clip.

  6. VeraBench mascot as masthead. Paired with the killer stat at baseline via grid-template-columns: auto 220px, forming a lockup rather than floating in the far right column.

  7. research_topic snippet points at a reachable API. Originally the Wikipedia URL; programmatic Wikipedia fetches return 403 without a custom User-Agent. Swapped to DuckDuckGo's Instant Answer API (https://api.duckduckgo.com/?format=json&q=), which is short, unauthenticated, and returns a JSON envelope the model can summarise. End-to-end tested with a real vera run + Inference.complete.

  8. Agent Score remediations. Page shrunk 52.9K → 44.9K chars (clears the 50K warning boundary). build_llms_txt() gains a ## Homepage section referencing /index.md; build_index_md() rewritten to mirror the HTML section-for-section (grew 4K → 13.5K) — markdown-content-parity miss rate 90% → 21%.

  9. Load-bearing agent metadata preserved. All <link rel="alternate">, rel="llms-txt", rel="llms-full-txt", schema.org JSON-LD, and the bottom-of-body <script type="text/llms.txt"> stay intact.

Agent Score results (local afdocs run)

Check Before After
page-size-html ⚠ 52.9K ✓ 28K effective (44.9K raw, 37% boilerplate)
page-size-markdown ✓ 13K
llms-txt-directive
llms-txt-freshness ✗ 1/2 ✓ 2/2 (on veralang.dev; skipped locally due to hostname mismatch)
markdown-content-parity ✗ 90% missing ✗ 21% missing (4× improvement — remaining gap is interface chrome)
content-negotiation ✗ GitHub Pages limitation; needs a Cloudflare Worker or migration
content-start-position ⚠ inline CSS consumes truncation budget; single-file constraint

Remaining gaps tracked in #525 and added to ROADMAP Phase 3b.

Files changed

  • docs/index.html — redesign (52.9K → 44.9K chars)
  • docs/index.md — generator output, now mirrors HTML section-for-section
  • docs/llms.txt — generator output, now includes Homepage entry
  • docs/sitemap.xml — regenerated (lastmod refresh)
  • docs/github.svg — new, used in footer
  • scripts/build_site.pybuild_llms_txt() + build_index_md() updates
  • CHANGELOG.md — Unreleased entry under new ### Website subsection
  • ROADMAP.mdveralang.dev: close remaining Agent Score gaps (content negotiation, truncation budget, markdown parity) #525 reference under Phase 3b

Test plan

  • python scripts/check_site_assets.py — site assets are in sync with generators
  • Local HTTP server + npx afdocs check http://localhost:8765 --fixes --verbose — all Agent Score improvements confirmed
  • Manual viewport checks at 375px, 768px, 1280px — type hierarchy, section rhythm, code samples, and masthead lockup all render cleanly at every width
  • research_topic code sample run end-to-end with a real ANTHROPIC_API_KEY — DDG → LLM → stdout works; output is a faithful multi-source summary pulling from AbstractText and Infobox in the JSON
  • CI green after push

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Redesigned homepage with expanded content, examples, benchmark info and agent-focused guidance; added a dedicated homepage section for agents.
  • Documentation

    • Roadmap updated with discoverability optimisations; Known Issues now lists CI-ignored vulnerabilities and removal criteria.
  • Bug Fixes

    • Fixed sample query encoding and stale homepage status counts; Agent Score improved (now 2 failures / 1 warning).
  • Tests

    • Updated test expectations to match reduced example block count.
  • Chores

    • CI dependency-audit temporarily ignores an additional vulnerability.

Full redesign of docs/index.html with an editorial-research aesthetic.
Structural moves: bilingual reading-path device at the top of the page
(@reader.0 -> humans / @reader.1 -> agents, using Vera's own slot-reference
syntax to acknowledge the dual audience); "Why?" thesis promoted to a
weighty anchor with a serif-display callout; five-sample code showcase
with commentary stacked below each block (the old side-by-side column
clipped long lines like research_topic's URL concat); VeraBench lifted
above the reference grid with a masthead stat paired to the mascot in a
baseline-aligned lockup; reference grid condensed from 17 features to 9
(typed-stdlib entries merged; Full contracts + Contract-driven testing
merged); dark "For Agents" section framing the page as a machine-readable
specification. Meerkat hero, briefing-mandated font stack (DM Serif
Display + Inter + JetBrains Mono), and single-file HTML all preserved;
no frameworks, no trackers, no analytics.

Visual system:
- Three-font hierarchy: serif for statements the site is making, sans
  for explanations, mono for machine surfaces (code, eyebrows, readpath).
- Cream/brown/orange palette aligned to Negroni's Vermouth + Campari
  scales; #FFECD1 matches the hero meerkat's baked cream, #FEEAD1 pinned
  to the VeraBench section to match its chart/mascot assets, #FFE2CE as
  contrast surface for alternating-section rhythm.

Agent Score (buildwithfern.com/agent-score) moved from 3 failures + 2
warnings to 2 failures + 1 warning:
- page-size-html 52.9K -> 44.9K (clears 50K edge)
- page-size-markdown 13K (clears warning)
- llms-txt-directive: pass (was fail)
- llms-txt-freshness: 2/2 sitemap doc pages covered (was 1/2)
- markdown-content-parity miss rate 90% -> 21% (4x reduction)

scripts/build_site.py:
- build_llms_txt() gains a ## Homepage section referencing /index.md
- build_index_md() rewritten to mirror the HTML section-for-section
  (thesis, five code samples, VeraBench stat + table + caveats, Runs
  Everywhere, install, For Agents, status). Output grew 4,019 -> 13,491
  chars; well under the 50K page-size threshold.

Preserves all load-bearing agent metadata: <link rel="alternate"> entries,
rel="llms-txt" / rel="llms-full-txt" directives, schema.org JSON-LD, and
the inline <script type="text/llms.txt"> hand-off at the bottom of <body>.

Remaining gaps (content-negotiation via GitHub Pages, inline-CSS
truncation budget, final 21% parity) tracked in #525 and added to
ROADMAP Phase 3b.

Refs: #525

Co-Authored-By: Claude <noreply@anthropic.invalid>
@coderabbitai

coderabbitai Bot commented Apr 24, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Rewrites homepage generation (large build_index_md overhaul, adds ## Homepage to build_llms_txt), updates CI to ignore CVE-2026-3219, records ignored CVEs and roadmap entries, reduces validated Vera code blocks on the homepage, and updates changelog/testing metadata.

Changes

Cohort / File(s) Summary
Build script
scripts/build_site.py
Major rewrite of build_index_md(version: str) -> str (expanded landing content, new sections including Vera examples, VeraBench, Runs Everywhere, For Agents, Status; computes and interpolates n_conformance = _count_conformance()). build_llms_txt(version: str) -> str now appends a ## Homepage section. (+235/-59)
CI workflow
.github/workflows/ci.yml
Extends pip-audit invocation with --ignore-vuln CVE-2026-3219 and updates inline comments documenting the temporary ignore and removal condition. (+4/-1)
Documentation records
CHANGELOG.md, ROADMAP.md, KNOWN_ISSUES.md, TESTING.md
Adds Unreleased changelog entries (homepage redesign, Agent Score change, CI ignore), appends roadmap items (#525, #528), records CI‑ignored CVEs with rationale/issue refs, and updates testing metadata (validated code-block counts and test line counts).
Tests
tests/test_html.py
Adjusts TestHtmlCodeSamples.test_vera_block_count expectation from 5 → 4 Vera <pre> blocks and updates assertion comment/message to reflect the redesigned homepage samples. (+5/-3)

Sequence Diagram(s)

(omitted — changes are content/CI/test updates, not multi-component control-flow changes)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Possibly related issues

Possibly related PRs

Suggested labels

docs, tests, ci

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the primary change: a redesign of the veralang.dev homepage with improvements to Agent Score metrics. It is concise, specific, and directly related to the substantial changes across docs/index.html and supporting generators.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch redesign-index

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Apr 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.03%. Comparing base (9e07089) to head (fa87bca).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #526   +/-   ##
=======================================
  Coverage   91.03%   91.03%           
=======================================
  Files          58       58           
  Lines       21964    21964           
  Branches      259      259           
=======================================
  Hits        19995    19995           
  Misses       1962     1962           
  Partials        7        7           
Flag Coverage Δ
javascript 56.83% <ø> (ø)
python 95.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scripts/build_site.py`:
- Around line 421-429: The DuckDuckGo query is interpolated raw into Http.get in
research_topic causing spaces and special characters to break or change the
request; before calling
Http.get(string_concat("https://api.duckduckgo.com/?format=json&q=", `@String.0`))
percent-encode the user query (the `@String.0` value) and use the encoded string
in the concatenation so queries like "Alan Turing" or "C&C" are safe—implement
or call a URL-encoding helper (e.g., encode_query or encodeURIComponent
equivalent) and replace the direct `@String.0` usage when building the URL in
research_topic.
- Around line 575-577: The status paragraph currently hardcodes project facts
(e.g., "six algebraic effects" and "77-program conformance suite"); replace
those literals with values sourced from the site's metadata instead of fixed
text: use the metadata object used elsewhere in build_site.py (e.g.,
site_meta.effects or metadata.effects for the effects list length and
site_meta.conformance_count or metadata.conformance_count for the conformance
suite size) and keep the existing {n_examples}, {RAW}, and {REPO} placeholders;
update the string construction around the "## Status" block to compute
effects_count = len(metadata.effects) and conformance_count =
metadata.conformance_count and interpolate those variables so the generated page
always reflects the repo state.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 85af570f-f5d4-46e8-a01e-9c42acb9bcc5

📥 Commits

Reviewing files that changed from the base of the PR and between 9e07089 and 2d9bc10.

⛔ Files ignored due to path filters (5)
  • docs/github.svg is excluded by !**/*.svg, !docs/**
  • docs/index.html is excluded by !docs/**
  • docs/index.md is excluded by !docs/**
  • docs/llms.txt is excluded by !docs/**
  • docs/sitemap.xml is excluded by !docs/**
📒 Files selected for processing (3)
  • CHANGELOG.md
  • ROADMAP.md
  • scripts/build_site.py

Comment thread scripts/build_site.py
Comment thread scripts/build_site.py Outdated
Two unrelated CI failures on redesign-index, fixed together:

1. tests/test_html.py::test_vera_block_count expected 5 Vera blocks
   in docs/index.html but the redesign consolidated safe_classify
   into classify_sentiment, leaving 4. Test expectation updated
   5 -> 4, comment rewritten, and the CHANGELOG 'five-sample' claim
   corrected to 'four-sample' to match the actual layout.

2. pip-audit flags CVE-2026-3219 in pip 26.0.1 (GHSA-58qw-9mgm-455v)
   an archive-handling hardening bug. Upstream fix merged in
   pypa/pip#13870, milestone pip 26.1 -- not yet released, so
   there is nothing to upgrade to. Add --ignore-vuln CVE-2026-3219
   to the dependency-audit step with a dated comment linking the
   CVE, GHSA, upstream PR, and internal tracking issue #527.
   Threat model (untrusted ambiguous archives) does not apply to
   our CI. The ignore is a bridge; #527 carries the 'remove when
   pip 26.1 ships' action item.

Also:
- Add a 'CI ignores' table to KNOWN_ISSUES.md covering both
  CVE-2026-4539 (pygments, existing) and CVE-2026-3219 (pip, new)
  with explicit removal triggers.
- Add a ### CI bullet to CHANGELOG [Unreleased] recording the
  ignore and linking #527.

Co-Authored-By: Claude <noreply@anthropic.invalid>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@TESTING.md`:
- Line 79: Update TESTING.md to match the changed test_html.py metadata: adjust
the documented HTML block count (the HTML block count entry that refers to
test_html.py) to 4 to match the table entry for test_html.py, and update the
"dependency-audit" command example to the exact command used by the project’s
CI/dependency scanner so it reflects the current PR state; locate the references
to "test_html.py", the HTML block count statement, and the "dependency-audit"
command in TESTING.md and replace their text so they are factual and runnable
against the current codebase.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 024e2421-c99d-4718-9656-ab9f6f674dd8

📥 Commits

Reviewing files that changed from the base of the PR and between 2d9bc10 and 9634f33.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • KNOWN_ISSUES.md
  • TESTING.md
  • tests/test_html.py

Comment thread TESTING.md
aallan and others added 2 commits April 24, 2026 21:23
Addresses two CodeRabbit findings on the v0.0.119 homepage redesign:

1. research_topic sample URL-encodes its input. The DuckDuckGo query
   interpolated the raw @String.0 parameter, so queries with spaces,
   &, %, unicode, etc. would have broken the request. One extra let
   binding (url_encode(@String.0)); verified with vera check. As a
   bonus the sample now demonstrates Vera's url_encode built-in.

2. Correct stale status-paragraph facts. Two were already drifted:
   - 'six algebraic effects (IO, Http, State, Exceptions, Async,
     Inference)' -- missing Random; actual is seven
   - '77-program conformance suite' -- actual is 80
   - HTML '30 worked examples' -- actual is 32 (markdown was already
     using the dynamic {n_examples})
   Fixed in both docs/index.html and build_index_md(); the markdown
   generator now also uses {n_conformance} dynamically.

The structural fix (auto-generate or gate all hardcoded numbers in
docs/index.html via scripts/check_doc_counts.py, same pattern as
TESTING.md/CLAUDE.md already use) is filed as #528 and placed in
ROADMAP Phase 3b alongside #424, #401, #525.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Two stale references in TESTING.md flagged by CodeRabbit, verified
against current code:

1. Line 16 claimed '5 Vera blocks in docs/index.html (5 validated)'
   but the redesign reduced it to 4. Updated 5 -> 4 in both the
   block count and the validated count.

2. Line 422 listed the dependency-audit command without the new
   --ignore-vuln CVE-2026-3219 flag added in the previous commit.
   Updated to match ci.yml exactly, with the tracking reference
   to #527 for context.

Co-Authored-By: Claude <noreply@anthropic.invalid>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@ROADMAP.md`:
- Line 173: The roadmap line incorrectly states "three checks still fail" for
issue `#525`; update the phrasing in ROADMAP.md (the bullet referencing [`#525`] and
the Agent Score items `content-negotiation`, `content-start-position`,
`markdown-content-parity`) to accurately reflect "2 failures + 1 warning" (or
otherwise mirror the exact Agent Score output) and adjust the sentence so it no
longer claims three failures, keeping the existing explanation of each check
intact.

In `@scripts/build_site.py`:
- Around line 501-503: The generated effects bullet in the markdown currently
lists "Algebraic effects — IO, Http, State, Exceptions, Async, Inference" but
the status block states seven effects; update the generator in
scripts/build_site.py so the effects list includes "Random" (i.e., change the
string/array that produces the bullet to include Random so the line becomes "...
IO, Http, State, Exceptions, Async, Inference, Random") ensuring the produced
docs/index.md text and the status count remain consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c9ea47c7-04f5-4b90-bd66-f9932d86410d

📥 Commits

Reviewing files that changed from the base of the PR and between 9634f33 and ffbd147.

⛔ Files ignored due to path filters (2)
  • docs/index.html is excluded by !docs/**
  • docs/index.md is excluded by !docs/**
📒 Files selected for processing (4)
  • CHANGELOG.md
  • ROADMAP.md
  • TESTING.md
  • scripts/build_site.py

Comment thread ROADMAP.md Outdated
Comment thread scripts/build_site.py Outdated
…ffects list

Both legitimate findings, verified against current code:

1. ROADMAP line 173 said 'three checks still fail' but the post-
   redesign Agent Score is 2 failures + 1 warning (per the PR body).
   Rephrased to match exactly: two failures (content-negotiation,
   markdown-content-parity) + one warning (content-start-position).
   Check-by-check explanations preserved.

2. Reference-grid 'Algebraic effects' bullet listed IO through
   Inference and omitted Random. Same drift I caught earlier in the
   Status paragraph but missed in the Key Features bullet. Fixed in
   both docs/index.html (line 479) and the markdown generator in
   scripts/build_site.py (line 503). docs/index.md regenerated.

Adds another data point for #528 -- the effects list appears in
two places on the page and both had drifted. Gating via
check_doc_counts.py would have caught this on the first commit.

Co-Authored-By: Claude <noreply@anthropic.invalid>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scripts/build_site.py`:
- Around line 473-486: The homepage Markdown text currently claims a
"60-problem" benchmark and repeats "60 problems" in the paragraph/table; update
those occurrences to the correct "50 problems" to match the roadmap/PR and
ensure consistency (search for the literal strings "60-problem" and "60
problems" in the generated content block around the VeraBench table and the
paragraph referencing "VeraBench v0.0.7"). Leave the rest of the wording, table,
and release links (e.g., "VeraBench v0.0.7" and "Vera v0.0.108") intact unless
those release tags also need changing; run the site build to verify the updated
Markdown renders correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: cd749053-d47a-4192-a1f5-af0778782991

📥 Commits

Reviewing files that changed from the base of the PR and between ffbd147 and 6daca73.

⛔ Files ignored due to path filters (2)
  • docs/index.html is excluded by !docs/**
  • docs/index.md is excluded by !docs/**
📒 Files selected for processing (2)
  • ROADMAP.md
  • scripts/build_site.py

Comment thread scripts/build_site.py Outdated
The page was internally inconsistent: docs/index.html line 412 said
'60-problem benchmark' in the hero paragraph but line 452 caveat
already said '50 problems'. The markdown generator mirrored the hero
(60) on line 473 and drifted the caveat from 50 to 60 on line 486,
compounding the mismatch.

VeraBench currently has 60 problems, but this entire section shows
v0.0.7 results (per the explicit 'Results from VeraBench v0.0.7' link)
and at v0.0.7 the suite was 50 problems. Aligning all four sites
to 50 makes the description consistent with the results shown.

Another data point for #528: the same number in two sibling paragraphs
diverged silently. Gated via check_doc_counts.py this couldn't happen.

Co-Authored-By: Claude <noreply@anthropic.invalid>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scripts/build_site.py`:
- Around line 499-509: The effects list in build_llms_txt omits the "Random"
effect causing inconsistent output versus build_index_md; update the effects
enumeration inside the build_llms_txt function (the list/string that currently
enumerates six effects) to include "Random" so both build_llms_txt and
build_index_md list the same seven effects (IO, Http, State, Exceptions, Async,
Inference, Random).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 723340eb-50f6-4962-b8fb-31d746651ad3

📥 Commits

Reviewing files that changed from the base of the PR and between 6daca73 and cac8c3c.

⛔ Files ignored due to path filters (2)
  • docs/index.html is excluded by !docs/**
  • docs/index.md is excluded by !docs/**
📒 Files selected for processing (1)
  • scripts/build_site.py

Comment thread scripts/build_site.py
The rabbit caught two more stale effects lists in build_llms_txt:

1. Line 79 intro paragraph: '(IO, Http, State, Exceptions, Async,
   Inference)' -> added Random
2. Line 121 Chapter 7 description: same drift. Verified against the
   spec -- spec/07-effects.md section 7.7.4 does cover Random, so
   the description should include it too.

Also fixed a regression I introduced in an earlier commit in this
PR: making {n_conformance} dynamic in the status paragraph left
the hardcoded article 'A' in place, producing 'A 80-program
conformance suite' (ungrammatical). The HTML had 'An 80-program'
correctly. Changed the generator article 'A' -> 'An' since the
current count (80) starts with a vowel sound; needs revisiting
if the count drops into a consonant-sound range (e.g. 100, 200).

docs/llms.txt and docs/index.md regenerated.

Every pass through this review cycle is uncovering more instances
of the same 'effects list in N places, M of them drifted' pattern.
#528 is looking more and more like the right move.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan aallan merged commit 9cbb977 into main Apr 24, 2026
19 checks passed
@aallan aallan deleted the redesign-index branch April 24, 2026 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant