Skip to content

ci(fork-sync): MODULE_ATTESTATIONS manifest for src/agents/* (#2437)#2446

Merged
alexey-pelykh merged 1 commit intomainfrom
ci/2437-module-attestations
Apr 21, 2026
Merged

ci(fork-sync): MODULE_ATTESTATIONS manifest for src/agents/* (#2437)#2446
alexey-pelykh merged 1 commit intomainfrom
ci/2437-module-attestations

Conversation

@alexey-pelykh
Copy link
Copy Markdown

Summary

Implements H9 of ADR 0005 — semantic-stub detection that H7 (throwing-stub AST gate) and H8 (test-side mock baseline) cannot see. Every runtime export in src/agents/ depth-1 now carries an attestation declaring its implementation status (live / stub / partial / deprecated). Structural consistency enforced by scripts/check-attestations.mjs; semantic correctness enforced by CODEOWNERS review.

Scope

  • 93 attested modules, 319 attestations across src/agents/* depth-1
  • 11 entries classified "partial" — functions that were gutted to return hardcoded defaults (e.g., lookupContextTokens returns 200000, resolveThinkingDefault returns undefined). They work in the narrow case of "return the default"; callers tolerate that.
  • No current "stub" or "deprecated" entries — nothing's currently broken in src/agents/
  • Rest are auto-classified "live" (real implementations)

Design decisions (per user "proceed as issue describes")

Open question Decision
Category set 4 categories including "partial" — real fork code has functions that work partially post-gutting; "partial" prevents the false binary
Placement Inline, top of file, after imports — co-location prevents drift
CODEOWNERS /src/agents/ @alexey-pelykh (solo maintainer)
Tree-shakability as const on plain object, never imported by production — zero bundle cost

Enforced invariants

  1. Every runtime export has an attestation entry (missing fails CI)
  2. Every entry corresponds to a current export (stale fails CI)
  3. "live" attestation on a throwing-stub-shape function fails
  4. "stub" attestation on a function with non-test callers fails

Runtime exports counted: export function, export const = () => ..., export const = function..., export class, export default (if function). Types, interfaces, enums, non-function data constants, and re-exports are not counted.

Changes

  • scripts/check-attestations.mjs (new, 526 LOC): the gate. Includes --self-test mode with 10 fixtures covering all 4 invariants, expression-body arrow enumeration, category validation, non-function-export exclusion. Fail messages guide the author to the right fix.
  • scripts/generate-attestations.mjs (new, 261 LOC): one-shot bootstrap tool that walks attested modules, enumerates exports via AST, auto-classifies using H7 calibration signals, writes MODULE_ATTESTATIONS blocks. Idempotent; remains as a maintenance utility for extending scope to new directories later.
  • scripts/lib/throwing-stub-shape.mjs (new, 210 LOC): shared AST classification helpers (looksLikeThrowingStub, classifyThrowingStubShape, and the four calibration-signal primitives). Previously duplicated across 3 scripts; now single source of truth. Net -373 LOC across callers, +210 LOC in lib.
  • scripts/check-throwing-stub-callers.mjs: refactored to use shared lib (its 10 self-tests still pass unchanged)
  • .github/workflows/ci.yml: new attestation-gate job (self-test before main check, mirroring throwing-stub-callers-gate convention)
  • .github/CODEOWNERS (new): /src/agents/@alexey-pelykh
  • CONTRIBUTING.md: new § Module attestations with category table, 5-point "when this fires", 4-step "how to update"
  • src/agents/*.ts (83 files): MODULE_ATTESTATIONS block added after imports

Verification

Command Result
node scripts/check-attestations.mjs 93 modules, 319 attestations, PASS
node scripts/check-attestations.mjs --self-test 10/10 PASS
node scripts/check-throwing-stub-callers.mjs 0 violations, PASS
node scripts/check-throwing-stub-callers.mjs --self-test 10/10 PASS (refactor preserved behavior)
node scripts/check-stub-debt.mjs both counters PASS (12, 134)
pnpm check clean

Fresh-context validation (session 2df6a80c)

Initial pass found 1 blocking issue + 3 non-blocking observations:

  • Blocking: Expression-body arrow coverage gap — ts.isBlock(init.body) filter silently excluded export const foo = () => expr from enumeration. 13 runtime exports uncovered, including 5 semantic stubs with active production callers. This was the exact failure mode H9 was designed to catch per the script's own docstring. Fix applied: dropped the filter; both scripts now treat expression-body arrows identically to block-body arrows. All 13 gaps closed with appropriate classifications.
  • Non-blocking: No self-test (→ added in polish), duplicated AST logic (→ extracted to shared lib in polish), .mocks.shared.ts suffix missing (→ added).

Re-validated: CLEAN.

Polish iterations (session 9a3fc434, 3 iterations)

  1. Extracted shared throwing-stub-shape.mjs lib; ~-373/+210 LOC net; single source of truth for calibration signals
  2. Added --self-test with 10 fixtures; wired into CI before main gate
  3. Aligned test-file filter; removed 2 dead MODULE_ATTESTATIONS blocks from test-harness files

Dry-run probes (AC 8)

Probe Expected Actual
Delete an attestation entry FAIL "has no MODULE_ATTESTATIONS entry"
Add a stale attestation FAIL "stale attestation"
Attest "live" on a throwing-stub shape FAIL "matches throwing-stub pattern"
Attest "stub" on a function with live callers FAIL "has N non-test caller(s)"

PR #2360's regression was equivalent to probe #3. The gate would have caught it at PR time.

Cascade impact

H9 delivered. Unblocks Phase 3 (#2441 composite gate) — H7/H8/H9 all landed.

Follow-up candidates (out of scope, deferred)

  • Expand attestation scope to src/middleware/ and src/gateway/ (tracked via ADR 0005 H9 description)
  • Explore richer attestation metadata (remediation issue links, expiry dates) if churn warrants
  • 6-month post-ship review of "partial" classification accuracy

Test plan

  • Gate passes on current main
  • Self-tests (10/10)
  • Existing gates (H5, H7, H8) still pass
  • pnpm check clean
  • Expression-body arrows enumerated
  • Probes confirm all 4 invariants fire
  • Re-validated CLEAN after fixes

Closes #2437.

Implements H9 of ADR 0005 — semantic-stub detection that H7 (throwing-stub
AST gate) and H8 (test-side mock baseline) cannot see.

Every runtime export in src/agents/ depth-1 now carries an attestation
declaring the implementation status: "live" / "stub" / "partial" /
"deprecated". The attestation-gate CI job enforces structural invariants;
CODEOWNERS enforces semantic review.

Design decisions (per user direction to proceed as issue describes):
- 4-category set including "partial" for legitimately-gutted-to-default
  functions that work in the narrow case
- Inline placement (top of file, after imports) to prevent drift
- Plain object with `as const` for tree-shakability (zero bundle cost)
- CODEOWNERS /src/agents/ → @alexey-pelykh (solo maintainer project)
- Initial scope: src/agents/* depth-1. Expand to src/middleware/, etc.,
  in follow-up issues if the pattern proves valuable

Rollout:
- 93 attested modules, 319 attestations across src/agents/* depth-1
- Auto-classified "live" by default; 11 semantic-stub exports manually
  classified "partial" (functions returning hardcoded defaults post-
  gutting — e.g., lookupContextTokens returns constant 200000,
  resolveThinkingDefault returns undefined)
- No current "stub" or "deprecated" entries (nothing currently broken)

Gate invariants:
1. Every runtime export has an attestation entry (missing fails CI)
2. Every entry corresponds to a current export (stale fails CI)
3. "live" attestation on a throwing-stub-shape function fails
4. "stub" attestation on a function with non-test callers fails

Enforced via scripts/check-attestations.mjs (AST-based classifier with
10 self-test fixtures covering all 4 invariants, expression-body arrow
enumeration, category validation, and non-function exclusion).

Polish iterations (3 total):
1. Extracted ~330 LOC of duplicated AST classification into new
   scripts/lib/throwing-stub-shape.mjs shared by check-attestations,
   generate-attestations, and check-throwing-stub-callers. Single site
   for the four calibration signals; net -373 LOC across 3 scripts,
   +210 LOC shared lib
2. Added --self-test mode with 10 fixtures + wired into CI (symmetric
   with throwing-stub-callers-gate convention). Regressions in the
   classifier itself now caught before the main gate runs
3. Aligned test-file filter across both attestation scripts via
   isTestLikeTypeScriptFile; removed 2 dead MODULE_ATTESTATIONS blocks
   from test-harness files that slipped past the naive filter

Validation (fresh-context subprocess, session 2df6a80c) initially
found one blocking issue (expression-body arrows silently excluded by
isBlock filter) + 3 non-blocking. All addressed; re-validated CLEAN.
Polish (session 9a3fc434): CLEAN after 3 iterations.

Dry-run verification:
- Probe 1 (delete attestation): FAIL "has no MODULE_ATTESTATIONS entry"
- Probe 2 (add stale attestation): FAIL "stale attestation"
- Probe 3 (live on throwing-stub): FAIL "attested 'live' but matches
  throwing-stub pattern"
- Probe 4 (stub with live callers): FAIL "has 9 non-test caller(s)"

AC 8 (PR #2360 dry-run): time-traveling to PR #2360's state is
impractical; verified the equivalent case — a "live" attestation on a
throwing-stub-shape function fails, and an unattested new export fails.
These are the precise failure modes PR #2360 exhibited.

HQ ADR 0005 H9 description updated in parallel (hq commit pending) to
reflect the as-shipped scope and 6-month sunset review.

scripts/generate-attestations.mjs is a one-shot bootstrap tool that
remains as a maintenance utility for extending the attestation scope
to new directories (idempotent; skips modules with existing blocks).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(fork-sync): MODULE_ATTESTATIONS manifest for src/agents/* (Option 6)

1 participant