Skip to content

Replace line-numbered allowlists with inline HTML-comment fence annotations #538

@aallan

Description

@aallan

Summary

Replace the line-number-keyed ALLOWLIST dicts in scripts/check_skill_examples.py, scripts/check_spec_examples.py, and scripts/check_faq_examples.py with HTML-comment annotations placed immediately before each Vera fence that should be skipped.

The recurring pain

Every PR that edits SKILL.md hits the line-number-shift problem. Inserting one paragraph in the middle of SKILL.md shifts every allowlist entry below it; scripts/fix_allowlists.py exists to auto-renumber, but it has its own bugs:

  1. Silent duplicate keys: fix_allowlists.py --fix has shipped duplicate dict keys (same line number, two reasons) multiple times. Saved to memory as feedback_spec_allowlist.md. Has surfaced again in PR Release v0.0.121: nested closures + ADT capture work (#514) + #527 CVE cleanup #536 review cycles.
  2. Stale entries are missed: when a fence is removed, the allowlist entry for it points at the wrong block but fix_allowlists.py has no way to know, so it remaps to a different fence and the false allowlist hides a real failure.
  3. Cross-PR conflict source: line-numbered allowlists conflict on every concurrent PR that touches the relevant doc.

Proposed fix

Inline HTML-comment annotations in the source markdown:

<!-- vera:skip-parse category="fragment" reason="bare type expression" -->
```vera
List<Result<User, Error>>
```

Each pipeline stage gets its own directive name (vera:skip-parse, vera:skip-check, vera:skip-verify) — clearer than overloading one name with a stage attribute, and composes naturally when multiple directives stack on one fence.

The HTML-comment form is preferred over an info-string variant (vera-fragment) because:

  • It composes — arbitrary key="value" attributes (category, reason, issue link) without polluting the language tag.
  • It preserves GitHub syntax highlighting (the language tag remains plain vera).
  • It matches Sybil and pytest-codeblocks skip directives, so a future migration to Sybil composes naturally.
  • It's self-documenting: the labelled comment with a reason field signals intent in a way a bare suffix does not.

Migration

One-shot script processing each allowlisted document in descending line-number order so insertions don't shift unprocessed targets. Sketch (from the consolidation brief):

# scripts/migrate_allowlists.py — run once, then delete.
import re, pathlib
from check_spec_examples import ALLOWLIST_PARSE  # plus check / verify variants

ALL: dict[str, list[tuple[int, str, str, str]]] = {}
for src, kind in [(ALLOWLIST_PARSE, "parse"), ...]:
    for (path, lineno), (category, reason) in src.items():
        ALL.setdefault(path, []).append((lineno, kind, category, reason))

FENCE = re.compile(r'^(\s*)```vera\b')
for path, entries in ALL.items():
    entries.sort(key=lambda e: -e[0])  # descending so inserts don't shift
    p = pathlib.Path(path)
    lines = p.read_text().splitlines(keepends=True)
    for lineno, kind, category, reason in entries:
        i = lineno - 1
        if not FENCE.match(lines[i]):
            for d in (-1, 1, -2, 2):
                if 0 <= i + d < len(lines) and FENCE.match(lines[i + d]):
                    i += d; break
            else:
                print(f"MISS {path}:{lineno} — investigate"); continue
        indent = FENCE.match(lines[i]).group(1)
        r = reason.replace('"', '\\"')
        ann = f'{indent}<!-- vera:skip-{kind} category="{category}" reason="{r}" -->\n'
        lines.insert(i, ann)
    p.write_text("".join(lines))

After migration:

  • check_*_examples.py look at the comment immediately before each fence rather than consult a dict
  • scripts/fix_allowlists.py is deleted entirely
  • The dict files (currently in the check scripts) are deleted
  • scripts/check_skill_examples.py and friends shrink

Acceptance criteria

  • grep -nE 'ALLOWLIST_(PARSE|CHECK|VERIFY)' scripts/ returns no matches
  • A sample edit inserting 50 lines into SKILL.md does not require any allowlist update
  • scripts/fix_allowlists.py no longer exists in the repo

Why now

PR #536 alone hit the line-number-shift problem multiple times and the silent-duplicate-keys bug at least once. This is sustained recurring pain, not a one-off. Estimated effort: half a day for the migration script and careful diff review.

Source

Distilled from the doc-consolidation brief (Phase 3) — a working brief evaluated in #528, cherry-picking the elements proportionate to actual project pain.

Metadata

Metadata

Assignees

No one assigned

    Labels

    refactorCode refactoring and structural improvementstoolingIssue around tooling built for the language (e.g. package managers, IDE plug-ins)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions