[Schema Consistency] Schema Consistency Check — 2026-03-03 (Custom Jobs Compiler Coverage Gaps) #19383
Replies: 4 comments
-
|
🚀 The smoke test agent has arrived! I, the humble Copilot smoke test bot, have descended upon this discussion like a well-intentioned robot. My tests ran, my assertions passed (mostly), and I emerged victorious!
The machines are learning. The pipelines are running. The coffee is still hot. Smoke test run §22618955604 reporting for duty!
|
Beta Was this translation helpful? Give feedback.
-
|
💥 WHOOSH! 🦸 THE SMOKE TEST AGENT WAS HERE! 💥 KRAK-A-BOOM! The mighty Claude smoke test agent swooped in, ran all systems checks, and emerged victorious! 🔥 ZAP! — GitHub MCP: NOMINAL This agent will return... in 12 hours! ⏱️
|
Beta Was this translation helpful? Give feedback.
-
|
/plan |
Beta Was this translation helpful? Give feedback.
-
|
This discussion was automatically closed because it expired on 2026-03-04T10:09:06.706Z.
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Audit of the
jobs:sub-object reveals thatbuildCustomJobs()inpkg/workflow/compiler_jobs.goextracts only 8 of the 17 schema-defined job properties, silently discarding the rest. Users who rely on schema autocomplete to write custom job configurations will see their settings accepted at parse time but silently dropped at compile time with no warning or error.Strategy used: Strategy-19 — Custom Jobs Compiler Coverage Gap Analysis (variant of Strategy-1 Hybrid Semantic Field Coverage, applied to
jobs:sub-object specifically)Run: §22617702420 · 2026-03-03
Summary
runs-ontype handlingcache-memoryscope validation,strategyschema usability, docs gapCritical Issues
HIGH-1 — Custom jobs: 7 schema-defined job fields silently dropped by
buildCustomJobs()File:
pkg/workflow/compiler_jobs.go·buildCustomJobs()starting at line ~424The
jobs:frontmatter schema (main_workflow_schema.json,.properties.jobs.additionalProperties.properties) defines 17 standard job properties. ThebuildCustomJobs()compiler function only extracts 8 of them. The remaining 7 are silently dropped — no error, no warning, no hint.JobstructbuildCustomJobs()needsruns-on(string)ifpermissionsoutputsuseswithsecretsstepsname(display name)DisplayName)timeout-minutesTimeoutMinutes int)concurrencyConcurrency string)containerContainer string)servicesServices string)envEnv map[string]string)continue-on-errorstrategyNote:
containerandservicesdo work at the top-level workflow scope (extracted viaextractTopLevelYAMLSectionincompiler_orchestrator_workflow.go:236and:365), but NOT when used inside a named custom job entry. This creates a confusing asymmetry.Affected code — what is extracted vs what exists in Job struct
Job.DisplayNamerendering code exists inpkg/workflow/jobs.go:231-232:...but it is never populated for custom jobs.
Impact: Users who add
timeout-minutes,env,container,services,concurrency,name, orcontinue-on-errorto their custom job definitions get no indication these are unsupported — the schema validates them, the compiler silently discards them.HIGH-2 — Custom jobs
runs-onarray and object forms silently droppedFile:
pkg/workflow/compiler_jobs.go:465-469The schema defines
runs-onfor custom jobs as aoneOf [string, array, object]:The compiler only handles the
stringvariant:When
runs-onis an array (e.g.,[self-hosted, linux, large]), the type assertionrunsOn.(string)fails,job.RunsOnremains empty, and the compiled job is emitted without aruns-onfield — which is invalid GitHub Actions YAML.Contrast: The top-level
runs-onusesextractTopLevelYAMLSection(compiler_orchestrator_workflow.go:234) which passes the raw YAML through correctly for all three forms.Impact: Custom jobs using runner arrays (common for self-hosted or larger runners) produce malformed compiled output with no error.
Schema Improvements Needed
MEDIUM-1 —
cache-memoryscopeenum value not validated in codeFile:
pkg/workflow/cache.go:106-110The schema defines
scopeas an enum:The parser code accepts any string with no validation:
Later logic only tests
if scope == "repo"(cache.go:406), so any invalid value (e.g.,"organization","global") silently falls through to"workflow"behavior. Users get no feedback that their scope setting is invalid.Impact: Invalid scope values pass schema validation (if schema validation is skipped) and are silently treated as
"workflow", creating confusing debugging scenarios.MEDIUM-2 —
jobs.strategyschema is structurally unusableFile:
pkg/parser/schemas/main_workflow_schema.json,.properties.jobs.additionalProperties.properties.strategyThis defines
strategyas an empty object —additionalProperties: falsewith noproperties— meaning the only valid value is{}. Any real GitHub Actions matrix strategy (e.g.,matrix: {os: [ubuntu, windows]}) would fail schema validation.The schema includes the field to signal "this is a GitHub Actions job," but the definition actively blocks valid usage and provides no useful structure. The compiler also does not extract or pass through
strategyat all.Options:
strategyfrom the schema (honest: not supported in custom jobs)matrix/include/exclude/fail-fast/max-parallelproperties (implement support)additionalProperties: trueto allow pass-through without explicit support (pragmatic workaround)Documentation Gaps
MEDIUM-3 — Custom jobs section over-promises support
File:
docs/src/content/docs/reference/frontmatter.md:609The custom jobs section says:
This refers to steps (not jobs), but the phrasing is adjacent to the job definition examples and could mislead users into thinking all GitHub Actions job properties work in custom jobs. There is no documentation of:
timeout-minutes,env,container,services,concurrency,name,continue-on-error)runs-ononly supports string formSuggested addition: A table or note listing supported vs unsupported job fields, similar to the runner support table at line 465.
Recommendations
Fix
buildCustomJobs()to extract the missing fields —env,timeout-minutes,concurrency,name/DisplayNameare straightforward additions since theJobstruct already has the fields and the rendering code injobs.goalready handles them.containerandservicesrequire YAML string conversion (same pattern as top-level extraction).continue-on-errorneeds a newJobstruct field.Fix
runs-onarray/object handling inbuildCustomJobs()— Usego-yaml.Marshalon the raw value (matching the top-level approach) instead of a type-asserting string cast. Prevents malformed output for self-hosted runner arrays.Add
scopeenum validation inparseCacheMemoryEntry()— After parsing the scope string, validate it against["workflow", "repo"]and return an error for invalid values.Fix or remove
strategyschema definition — Either define it properly with all GitHub Actionsstrategysub-fields, or remove it to avoid false promises.Update custom jobs documentation — Add a clear field support table documenting which job-level properties are supported.
Strategy Performance
jobs:sub-object)References:
Beta Was this translation helpful? Give feedback.
All reactions