refactor(loader): emit JSON Schema from loader declarations (one-source convergence) (n8p.6)#94
Conversation
…ce convergence) (n8p.6) The config grammar had two authors: dsl-types.ts → ts-json-schema-generator → committed schema, AND the loader validators. The two described different grammars (the schema knew structure+enums but not exactly-one-present for cache/actions, and omitted the cells sugar) — a [LAW:one-source-of-truth] gap config-schema.test even encoded as shape-vs-meaning. Resolve it with one declaration, two interpreters: each loader schema now carries a 'json' facet beside 'parse', authored from the same source constants (an enum spec's allowed feeds both the membership check and the schema enum). validateConfig composes the validate* fns; the new emitConfigSchema composes the symmetric *Json() fns over the SAME module-private declarations. - validate-core: FieldSpec/TaggedArm/PresentArm gain 'json'; objectJson / recordJson / taggedUnionJson / oneOfPresentJson / withConst emit-twins - each module exports its *Json() emitter beside its validator; layout breaks the node recursion with a $ref (emit's analogue of the lazy parse seam) and emits both authoring surfaces (row sugar + recursive root, cells included) - gen-schema/check-schema move to tsx scripts over emitConfigSchema; drop the ts-json-schema-generator devDep; regenerate the committed artifact - the emitted schema is MORE faithful than the type-derived one (captures cells + exactly-one-present); config-schema behavioral parity (13) holds, validator untouched (119 parity green) Fold-in: correct CLAUDE.md merge wording (layout is LayoutRowInput[] sugar over the canonical root tree, not a 2D string[][]; actions/helpers merge by-name). Closes n8p.6 — epic n8p (declarative loader schema) now 7/7.
There was a problem hiding this comment.
Z.ai Coding Agent Review
This PR replaces ts-json-schema-generator with a dual-interpreter approach: the same declarative loader schemas (GLOBALS_SCHEMA, VARIABLE_SCHEMA, CACHE_SCHEMA, SET_ARMS, layout grammar, etc.) now feed both the runtime validator (via parse facets) and the JSON Schema emitter (via json facets). This is a strong [LAW:one-source-of-truth] improvement — the old schema derived from dsl-types.ts and could drift from the loader (it was missing cells nodes, for example). The new approach closes that gap.
No must-change issues found. Observations for the author's attention:
-
[LAW:types-are-the-program]—JsonNode = Readonly<Record<string, unknown>>is very loose. ThewithConstfunction (validate-core.ts:604-616) andsetArm(actions.ts:296) castJsonNodeto{ properties, required }viaas, trusting thatobjectJsonalways produces that shape. The type doesn't enforce it. A full JSON Schema AST would be impractically large, so this is a pragmatic choice — but ifwithConstever receives a non-object node (e.g. ananyOf), it would silently produce garbage. -
The
inputvariable arm hand-authors itsjson(variables.ts:209-219) instead of deriving fromobjectJson(fieldMap). This is the one drift point where the schema shape is not mechanically derived from the field map. The comment explains why (cross-field invariant betweentypeanddefault), and the hand-authored shape matches the parse, but adding a new field toinput's parse would require remembering to update the hand-authored json too. Consider a note or assertion tying the two. -
Schema size doubled (~673 → ~1393 lines) because
CacheDeclis now inlined in every variable kind instead of$ref-shared. This is fine for editor validation but worth being aware of. -
cellsnode kind is now correctly included in theLayoutNodeschema definition — a real accuracy fix over the old schema. The existingconfig-schema.test.tsdoesn't have a GOOD config that exercisescellsthrough ajv; adding one would pin the new arm.
✅ Approved
The n8p.6 schema emitter replaced the ts-json-schema-generator code path with emitConfigSchema, so the devDep was removed from package.json but the lockfile still pinned it. CI installs with --frozen-lockfile, which rejects the mismatch. Regenerate the lockfile to match.
There was a problem hiding this comment.
Z.ai Coding Agent Review
This PR replaces ts-json-schema-generator with a hand-rolled schema emitter that derives JSON Schema from the same declarative loader schemas the runtime validator interprets. Each FieldSpec, PresentArm, and TaggedArm now carries a json facet alongside its parse function, and module-level *Json() functions compose these into the full editor-facing schema. This is a strong [LAW:one-source-of-truth] improvement — the schema and validator now read the same declarations.
Must-change: The input variable arm at src/config/loader/variables.ts:209-218 hand-authors its json object separately from its parse logic, while every other variable arm derives json from field specs via arm(). This creates a second source of truth for the input variable's shape that can drift from the parse logic. The fix is to define INPUT_FIELDS and use arm("input", INPUT_FIELDS) like all other kinds — a custom default field spec can handle the cross-field (default-matches-type) invariant since parse receives the full raw record.
Pre-existing / non-blocking observations:
- The committed schema artifact roughly doubled in size (~673 → ~1393 lines) because the shared
CacheDecldefinition is now inlined per-variable-kind rather than referenced via$ref. This is mechanically generated so not a LAW issue, but schema consumers now download a larger file. A future$refextraction pass over the emitted tree could reduce this if it becomes a concern. - The test in
test/config-schema.test.tsprovides good [LAW:verifiable-goals] coverage: it validates the committed artifact against Ajv, asserts the schema/loader boundary (structurally-bad configs fail both; semantically-bad configs pass schema but fail loader), and thecheck:schemaguard prevents drift. Thecellslayout node kind (previously missing from the type-derived schema) is now correctly covered.
❌ Request Changes
The input variable arm was the one place 'parse' and 'json' were authored as two independent representations of the same shape — a [LAW:one-source-of-truth] drift gap inside the very change that eliminates such gaps elsewhere. input is special only for its default-matches-type cross-field invariant; since a FieldSpec receives the whole record, that invariant moves into a custom 'default' spec that reads its sibling raw.type (without re-reporting a bad type — the 'type' spec owns that error). input is now one field map like every other arm, and arm() derives both interpreters from it. Emitted schema is byte-identical; validator behavior unchanged (119 parity green).
There was a problem hiding this comment.
Z.ai Coding Agent Review
The PR replaces ts-json-schema-generator with a custom schema emitter derived from the same declarative loader schemas (FieldSpec maps, TaggedUnionSchema, OneOfPresentSchema, RecordSchema) that the runtime validator interprets. Each FieldSpec now carries a json facet alongside parse, and emit helpers (objectJson, recordJson, withConst, taggedUnionJson, oneOfPresentJson) project those declarations into JSON Schema mechanically. The input variable arm was refactored from a hand-authored parse closure into a field map (INPUT_FIELDS) using inputDefaultSpec for its cross-field invariant — closing the last spot where parse and schema were authored independently. This is a genuine [LAW:one-source-of-truth] improvement: the old schema derived from TypeScript types (which knew structure but not exactly-one-present for cache/actions, and omitted cells sugar), while the new one derives from the actual validation declarations.
Must-change (1): The layout node schema regression — layoutNodeJson() emits arms with no required fields because layout field specs use required: false for parse-time error recovery. The parse functions always return fallback values (never undefined), so changing to required: true is safe and restores the schema's structural enforcement.
Pre-existing note: JsonNode = Readonly<Record<string, unknown>> is very loose — it admits any object, not just valid JSON Schema fragments. This is a pragmatic tradeoff (a full JSON Schema type would be extremely complex), and the check:schema guard + config-schema.test.ts catch structural drift, but the type doesn't enforce correctness at compile time. Not a regression; worth noting for future tightening.
❌ Request Changes
… strength
The emitted layout-node arms carried no 'required' array, so the schema accepted
{root:{kind:'container'}} (and even {root:{}}) — weaker than both the old schema
and the loader, which reports a missing direction/name/children and throws. The
cause: the layout field specs were 'required: false' for parse-time recovery, but
'required' now has two readers — fields() (fail-if-undefined) and objectJson()
(schema required[]). Those specs' parse never returns undefined (they recover to
''/'vertical'/[]), so 'required: true' is a no-op for fields() while correctly
emitting the field as mandatory. One value, honestly serving both interpreters
[LAW:types-are-the-program].
Adds two regression cases (bare container/segment node) asserting schema AND
loader both reject — locking the lockstep the PR establishes.
There was a problem hiding this comment.
Z.ai Coding Agent Review
This PR replaces ts-json-schema-generator with a dual-interpreter pattern: each declaration (FieldSpec, PresentArm, TaggedArm) now carries both a parse facet (runtime validation) and a json facet (schema emission), composed by mechanical emit helpers (objectJson, recordJson, taggedUnionJson, oneOfPresentJson). This directly enforces [LAW:one-source-of-truth] — the editor-facing schema and the runtime validator now derive from the same declarations, closing real drift (the old type-derived schema missed the cells sugar and exactly-one-present constraints).
No must-change items found. The diff is clean:
- All
FieldSpec/PresentArm/TaggedArminstances havejsonfacets (enforced by TypeScript since the interfaces now requirereadonly json: JsonNode). - The
inputarm was correctly refactored to usearm()withinputDefaultSpec()(reads siblingraw.typeto resolve the cross-field default invariant), closing the last hand-authored parse/schema split. - The
required: false → truechanges on layout fields (cellsSegmentsSpec,segmentNameSpec,directionSpec,childrenSpec,literalSpec) correctly reflect their semantic mandatory nature in the emitted schema — the parse recovers (sofields()treatsrequired: trueas a no-op), but the schema now correctly marks them required. The new test cases for bare container/segment nodes validate this. refine()correctly stays outside the schema path — it wrapsArmParse(notFieldSpec), and the schema derives from the field map directly. Cross-field refinements (min<max, by≠0) are correctly excluded from the schema as unexpressible.- The
cellsnode kind is now included in theLayoutNodedefinition, and the recursive$refforchildrencorrectly mirrors thelazythunk.
Observations (not must-change):
- The generated schema grew from 673 to 1406 lines because the cache
anyOfis inlined per variable kind instead of shared via$ref. This is cosmetic — the source of truth is stillCACHE_SCHEMA, and the output is auto-generated. JsonNode = Readonly<Record<string, unknown>>is deliberately loose to avoid a parallel typed AST. The tradeoff is documented.withConstuses type assertions (as Record<string, unknown>), which is fragile in principle but correct in practice since it's only called withobjectJsonoutput.
✅ Approved
What
The final child of epic n8p (declarative loader schema + one combinator engine). Makes the loader schema declarations the single source for both the runtime validator AND the editor-facing JSON Schema — one declaration, two interpreters
[LAW:one-source-of-truth].The gap this closes
The config grammar had two authors:
dsl-types.ts→ts-json-schema-generator→ committed schemasrc/config/loader/*)They described different grammars: the type-derived schema knew structure + enums but not exactly-one-present for cache/actions, and it omitted the
cellssugar.config-schema.test.tseven encoded the divergence as a deliberate "shape vs meaning" boundary.How
Each loader schema now carries a
jsonfacet besideparse, authored from the same source constants (an enum spec'sallowedfeeds bothparse's membership check andjson'senum, so they cannot drift).validateConfigcomposes thevalidate*fns; the newemitConfigSchemacomposes the symmetric*Json()fns over the same module-private declarations.validate-core:FieldSpec/TaggedArm/PresentArmgainjson;objectJson/recordJson/taggedUnionJson/oneOfPresentJson/withConstemit-twins mirror the validate interpreters.*Json()emitter beside its validator. Layout breaks the node recursion with a$ref(emit's analogue of thelazyparse seam) and emits both authoring surfaces (row sugar + recursiveroot,cellsincluded).gen-schema/check-schemamove totsxscripts overemitConfigSchema; thets-json-schema-generatordevDep is dropped; the committed artifact is regenerated from the emitter.The emitted schema is more faithful than the type-derived one — it now captures
cellsand the exactly-one-present constraints. What a JSON Schema still can't express (cross-field refinements, palette membership, duration format, cross-refs) stays a loader-only semantic check — the same complementary boundary, now with a smaller gap.Fold-in (per fold-small-followups)
Corrected stale
CLAUDE.mdmerge wording:layoutisLayoutRowInput[]sugar over the canonicalroottree (not a 2Dstring[][]), andactions/helpersmerge by-name.Verification
typecheck,lint, fullbuild(tsx gen + tsdown) ✓check:schemabyte-diff guard ✓config-schema.test.tsbehavioral parity (13) ✓ — schema accepts every GOOD, rejects every BAD_STRUCTURAL, dangling-ref boundary holdsdsl-loader+dsl-merge+config-resolution) ✓ — validator untouchedCloses n8p.6 → epic n8p is now 7/7.