Skip to content

refactor(loader): emit JSON Schema from loader declarations (one-source convergence) (n8p.6)#94

Merged
brandon-fryslie merged 4 commits into
mainfrom
refactor/n8p.6-schema-emitter
Jun 9, 2026
Merged

refactor(loader): emit JSON Schema from loader declarations (one-source convergence) (n8p.6)#94
brandon-fryslie merged 4 commits into
mainfrom
refactor/n8p.6-schema-emitter

Conversation

@brandon-fryslie

Copy link
Copy Markdown
Contributor

What

The final child of epic n8p (declarative loader schema + one combinator engine). Makes the loader schema declarations the single source for both the runtime validator AND the editor-facing JSON Schema — one declaration, two interpreters [LAW:one-source-of-truth].

The gap this closes

The config grammar had two authors:

  • dsl-types.tsts-json-schema-generator → committed schema
  • the loader validators (src/config/loader/*)

They described different grammars: the type-derived schema knew structure + enums but not exactly-one-present for cache/actions, and it omitted the cells sugar. config-schema.test.ts even encoded the divergence as a deliberate "shape vs meaning" boundary.

How

Each loader schema now carries a json facet beside parse, authored from the same source constants (an enum spec's allowed feeds both parse's membership check and json's enum, so they cannot drift). validateConfig composes the validate* fns; the new emitConfigSchema composes the symmetric *Json() fns over the same module-private declarations.

  • validate-core: FieldSpec / TaggedArm / PresentArm gain json; objectJson / recordJson / taggedUnionJson / oneOfPresentJson / withConst emit-twins mirror the validate interpreters.
  • each module exports its *Json() emitter beside its validator. Layout breaks the node recursion with a $ref (emit's analogue of the lazy parse seam) and emits both authoring surfaces (row sugar + recursive root, cells included).
  • build wiring: gen-schema/check-schema move to tsx scripts over emitConfigSchema; the ts-json-schema-generator devDep is dropped; the committed artifact is regenerated from the emitter.

The emitted schema is more faithful than the type-derived one — it now captures cells and the exactly-one-present constraints. What a JSON Schema still can't express (cross-field refinements, palette membership, duration format, cross-refs) stays a loader-only semantic check — the same complementary boundary, now with a smaller gap.

Fold-in (per fold-small-followups)

Corrected stale CLAUDE.md merge wording: layout is LayoutRowInput[] sugar over the canonical root tree (not a 2D string[][]), and actions/helpers merge by-name.

Verification

  • typecheck, lint, full build (tsx gen + tsdown) ✓
  • check:schema byte-diff guard ✓
  • config-schema.test.ts behavioral parity (13) ✓ — schema accepts every GOOD, rejects every BAD_STRUCTURAL, dangling-ref boundary holds
  • 119-parity bar (dsl-loader + dsl-merge + config-resolution) ✓ — validator untouched
  • 259-test sweep across all affected loader/dsl/segment/layout areas ✓
  • extended ad-hoc corpus (cells, every cache variant, every action source, nested trees) accept/reject correctly ✓

Closes n8p.6 → epic n8p is now 7/7.

…ce convergence) (n8p.6)

The config grammar had two authors: dsl-types.ts → ts-json-schema-generator →
committed schema, AND the loader validators. The two described different
grammars (the schema knew structure+enums but not exactly-one-present for
cache/actions, and omitted the cells sugar) — a [LAW:one-source-of-truth] gap
config-schema.test even encoded as shape-vs-meaning.

Resolve it with one declaration, two interpreters: each loader schema now
carries a 'json' facet beside 'parse', authored from the same source constants
(an enum spec's allowed feeds both the membership check and the schema enum).
validateConfig composes the validate* fns; the new emitConfigSchema composes
the symmetric *Json() fns over the SAME module-private declarations.

- validate-core: FieldSpec/TaggedArm/PresentArm gain 'json'; objectJson /
  recordJson / taggedUnionJson / oneOfPresentJson / withConst emit-twins
- each module exports its *Json() emitter beside its validator; layout breaks
  the node recursion with a $ref (emit's analogue of the lazy parse seam) and
  emits both authoring surfaces (row sugar + recursive root, cells included)
- gen-schema/check-schema move to tsx scripts over emitConfigSchema; drop the
  ts-json-schema-generator devDep; regenerate the committed artifact
- the emitted schema is MORE faithful than the type-derived one (captures cells
  + exactly-one-present); config-schema behavioral parity (13) holds, validator
  untouched (119 parity green)

Fold-in: correct CLAUDE.md merge wording (layout is LayoutRowInput[] sugar over
the canonical root tree, not a 2D string[][]; actions/helpers merge by-name).

Closes n8p.6 — epic n8p (declarative loader schema) now 7/7.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Z.ai Coding Agent Review

This PR replaces ts-json-schema-generator with a dual-interpreter approach: the same declarative loader schemas (GLOBALS_SCHEMA, VARIABLE_SCHEMA, CACHE_SCHEMA, SET_ARMS, layout grammar, etc.) now feed both the runtime validator (via parse facets) and the JSON Schema emitter (via json facets). This is a strong [LAW:one-source-of-truth] improvement — the old schema derived from dsl-types.ts and could drift from the loader (it was missing cells nodes, for example). The new approach closes that gap.

No must-change issues found. Observations for the author's attention:

  1. [LAW:types-are-the-program]JsonNode = Readonly<Record<string, unknown>> is very loose. The withConst function (validate-core.ts:604-616) and setArm (actions.ts:296) cast JsonNode to { properties, required } via as, trusting that objectJson always produces that shape. The type doesn't enforce it. A full JSON Schema AST would be impractically large, so this is a pragmatic choice — but if withConst ever receives a non-object node (e.g. an anyOf), it would silently produce garbage.

  2. The input variable arm hand-authors its json (variables.ts:209-219) instead of deriving from objectJson(fieldMap). This is the one drift point where the schema shape is not mechanically derived from the field map. The comment explains why (cross-field invariant between type and default), and the hand-authored shape matches the parse, but adding a new field to input's parse would require remembering to update the hand-authored json too. Consider a note or assertion tying the two.

  3. Schema size doubled (~673 → ~1393 lines) because CacheDecl is now inlined in every variable kind instead of $ref-shared. This is fine for editor validation but worth being aware of.

  4. cells node kind is now correctly included in the LayoutNode schema definition — a real accuracy fix over the old schema. The existing config-schema.test.ts doesn't have a GOOD config that exercises cells through ajv; adding one would pin the new arm.

✅ Approved

The n8p.6 schema emitter replaced the ts-json-schema-generator code path with
emitConfigSchema, so the devDep was removed from package.json but the lockfile
still pinned it. CI installs with --frozen-lockfile, which rejects the mismatch.
Regenerate the lockfile to match.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Z.ai Coding Agent Review

This PR replaces ts-json-schema-generator with a hand-rolled schema emitter that derives JSON Schema from the same declarative loader schemas the runtime validator interprets. Each FieldSpec, PresentArm, and TaggedArm now carries a json facet alongside its parse function, and module-level *Json() functions compose these into the full editor-facing schema. This is a strong [LAW:one-source-of-truth] improvement — the schema and validator now read the same declarations.

Must-change: The input variable arm at src/config/loader/variables.ts:209-218 hand-authors its json object separately from its parse logic, while every other variable arm derives json from field specs via arm(). This creates a second source of truth for the input variable's shape that can drift from the parse logic. The fix is to define INPUT_FIELDS and use arm("input", INPUT_FIELDS) like all other kinds — a custom default field spec can handle the cross-field (default-matches-type) invariant since parse receives the full raw record.

Pre-existing / non-blocking observations:

  • The committed schema artifact roughly doubled in size (~673 → ~1393 lines) because the shared CacheDecl definition is now inlined per-variable-kind rather than referenced via $ref. This is mechanically generated so not a LAW issue, but schema consumers now download a larger file. A future $ref extraction pass over the emitted tree could reduce this if it becomes a concern.
  • The test in test/config-schema.test.ts provides good [LAW:verifiable-goals] coverage: it validates the committed artifact against Ajv, asserts the schema/loader boundary (structurally-bad configs fail both; semantically-bad configs pass schema but fail loader), and the check:schema guard prevents drift. The cells layout node kind (previously missing from the type-derived schema) is now correctly covered.

❌ Request Changes

Comment thread src/config/loader/variables.ts Outdated
The input variable arm was the one place 'parse' and 'json' were authored as two
independent representations of the same shape — a [LAW:one-source-of-truth] drift
gap inside the very change that eliminates such gaps elsewhere. input is special
only for its default-matches-type cross-field invariant; since a FieldSpec
receives the whole record, that invariant moves into a custom 'default' spec that
reads its sibling raw.type (without re-reporting a bad type — the 'type' spec owns
that error). input is now one field map like every other arm, and arm() derives
both interpreters from it. Emitted schema is byte-identical; validator behavior
unchanged (119 parity green).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Z.ai Coding Agent Review

The PR replaces ts-json-schema-generator with a custom schema emitter derived from the same declarative loader schemas (FieldSpec maps, TaggedUnionSchema, OneOfPresentSchema, RecordSchema) that the runtime validator interprets. Each FieldSpec now carries a json facet alongside parse, and emit helpers (objectJson, recordJson, withConst, taggedUnionJson, oneOfPresentJson) project those declarations into JSON Schema mechanically. The input variable arm was refactored from a hand-authored parse closure into a field map (INPUT_FIELDS) using inputDefaultSpec for its cross-field invariant — closing the last spot where parse and schema were authored independently. This is a genuine [LAW:one-source-of-truth] improvement: the old schema derived from TypeScript types (which knew structure but not exactly-one-present for cache/actions, and omitted cells sugar), while the new one derives from the actual validation declarations.

Must-change (1): The layout node schema regression — layoutNodeJson() emits arms with no required fields because layout field specs use required: false for parse-time error recovery. The parse functions always return fallback values (never undefined), so changing to required: true is safe and restores the schema's structural enforcement.

Pre-existing note: JsonNode = Readonly<Record<string, unknown>> is very loose — it admits any object, not just valid JSON Schema fragments. This is a pragmatic tradeoff (a full JSON Schema type would be extremely complex), and the check:schema guard + config-schema.test.ts catch structural drift, but the type doesn't enforce correctness at compile time. Not a regression; worth noting for future tightening.

❌ Request Changes

Comment thread src/config/loader/layout.ts
… strength

The emitted layout-node arms carried no 'required' array, so the schema accepted
{root:{kind:'container'}} (and even {root:{}}) — weaker than both the old schema
and the loader, which reports a missing direction/name/children and throws. The
cause: the layout field specs were 'required: false' for parse-time recovery, but
'required' now has two readers — fields() (fail-if-undefined) and objectJson()
(schema required[]). Those specs' parse never returns undefined (they recover to
''/'vertical'/[]), so 'required: true' is a no-op for fields() while correctly
emitting the field as mandatory. One value, honestly serving both interpreters
[LAW:types-are-the-program].

Adds two regression cases (bare container/segment node) asserting schema AND
loader both reject — locking the lockstep the PR establishes.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Z.ai Coding Agent Review

This PR replaces ts-json-schema-generator with a dual-interpreter pattern: each declaration (FieldSpec, PresentArm, TaggedArm) now carries both a parse facet (runtime validation) and a json facet (schema emission), composed by mechanical emit helpers (objectJson, recordJson, taggedUnionJson, oneOfPresentJson). This directly enforces [LAW:one-source-of-truth] — the editor-facing schema and the runtime validator now derive from the same declarations, closing real drift (the old type-derived schema missed the cells sugar and exactly-one-present constraints).

No must-change items found. The diff is clean:

  • All FieldSpec/PresentArm/TaggedArm instances have json facets (enforced by TypeScript since the interfaces now require readonly json: JsonNode).
  • The input arm was correctly refactored to use arm() with inputDefaultSpec() (reads sibling raw.type to resolve the cross-field default invariant), closing the last hand-authored parse/schema split.
  • The required: false → true changes on layout fields (cellsSegmentsSpec, segmentNameSpec, directionSpec, childrenSpec, literalSpec) correctly reflect their semantic mandatory nature in the emitted schema — the parse recovers (so fields() treats required: true as a no-op), but the schema now correctly marks them required. The new test cases for bare container/segment nodes validate this.
  • refine() correctly stays outside the schema path — it wraps ArmParse (not FieldSpec), and the schema derives from the field map directly. Cross-field refinements (min<max, by≠0) are correctly excluded from the schema as unexpressible.
  • The cells node kind is now included in the LayoutNode definition, and the recursive $ref for children correctly mirrors the lazy thunk.

Observations (not must-change):

  • The generated schema grew from 673 to 1406 lines because the cache anyOf is inlined per variable kind instead of shared via $ref. This is cosmetic — the source of truth is still CACHE_SCHEMA, and the output is auto-generated.
  • JsonNode = Readonly<Record<string, unknown>> is deliberately loose to avoid a parallel typed AST. The tradeoff is documented.
  • withConst uses type assertions (as Record<string, unknown>), which is fragile in principle but correct in practice since it's only called with objectJson output.

✅ Approved

@brandon-fryslie brandon-fryslie dismissed stale reviews from github-actions[bot] and github-actions[bot] June 9, 2026 12:01

Findings addressed and resolved in later commits (input-arm one-source fix 70b3e6e, layout-required fix 7844ffe); superseded by the clean re-review on 7844ffe.

@brandon-fryslie brandon-fryslie merged commit 1b82790 into main Jun 9, 2026
7 checks passed
@brandon-fryslie brandon-fryslie deleted the refactor/n8p.6-schema-emitter branch June 9, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants