Keep agent prompts in versioned YAML files with a typed output schema, so you edit a prompt without recompiling and a linter catches a broken skill before it ships.

Last tested: June 2026. See Changelog at the bottom.

If this saves you from hardcoded prompt strings, follow @renezander030 — production notes on running LLM agents outside a demo.

Working implementation: github.com/renezander030/draftcat (Go, MIT) — the skill loader, output_schema validator, and draftcat validate linter below all ship in the repo.

The problem: your agent's prompts are string literals compiled into the binary. Editing one means a code review, a rebuild, and a redeploy. There is no schema on what the model returns, so a malformed response surfaces three steps later as a nil-map panic. And nothing tells you a prompt references a variable you never pass.

Move each skill into a YAML file: name, description, role, prompt (with {{variables}}), and an output_schema the runtime enforces on the model's JSON. Then lint the whole set.

The rule of thumb: a skill is data, not code. A prompt change is a YAML edit and a restart, not a recompile. Every skill carries the schema that validates its own output, and validate is the gate that runs in CI before any of it reaches the model.

git clone https://github.com/renezander030/draftcat && cd draftcat
go build -o draftcat .
./draftcat validate --strict     # lint config.yaml + every skills/*.yaml

--strict turns every warning into a failure, so a CI job fails on an orphaned skill or an undeclared {{var}}, not just on a hard error.

One file per skill in skills/. This is the entire schema:

name: triage-lead
description: Classify inbound CRM leads by intent and urgency
role: classifier
prompt: |
  You are a lead triage specialist. Classify this contact's intent and urgency.

  Contact:
  {{contact}}

  {{#source}}Source: {{source}}{{/source}}
  {{#conversation}}Recent messages: {{conversation}}{{/conversation}}

  Respond with ONLY valid JSON.
output_schema:
  intent:           {type: string, enum: [buying, inquiry, support, spam]}
  urgency:          {type: string, enum: [hot, warm, cold]}
  reason:           {type: string}
  suggested_action: {type: string, enum: [immediate_followup, nurture_sequence, manual_review, discard]}
  score:            {type: number, min: 0, max: 100}

• {{contact}} is a required substitution; {{#source}}...{{/source}} is a section that renders only when source is set (Mustache-style).
• role (classifier / drafter / voice-analyst) maps a skill to a model tier in config, so cheap skills run on a cheap model.
• output_schema is the contract the runtime enforces on the model's reply.

Glob the directory, unmarshal each file, register by name. A missing directory is fine; an unparseable file is logged and skipped, not fatal — one broken file never takes down the rest.

type SkillDef struct {
	Name         string                 `yaml:"name"`
	Description  string                 `yaml:"description"`
	Role         string                 `yaml:"role"`
	Prompt       string                 `yaml:"prompt"`
	OutputSchema map[string]interface{} `yaml:"output_schema"`
}

func LoadSkills(dir string) (*SkillRegistry, error) {
	reg := &SkillRegistry{skills: make(map[string]*SkillDef)}
	files, _ := filepath.Glob(filepath.Join(dir, "*.yaml"))
	for _, f := range files {
		data, err := os.ReadFile(f)
		if err != nil {
			log.Printf("[skills] failed to read %s: %v", f, err)
			continue
		}
		var skill SkillDef
		if err := yaml.Unmarshal(data, &skill); err != nil {
			log.Printf("[skills] failed to parse %s: %v", f, err)
			continue
		}
		reg.skills[skill.Name] = &skill
		log.Printf("[skills] loaded: %s (%s)", skill.Name, skill.Description)
	}
	return reg, nil
}

Because skills load from disk at startup, editing triage-lead.yaml and restarting changes the prompt with no rebuild. The prompt is config, not a compiled string.

The point of the schema: the model returns text, you get back a typed map or a hard error. The validator strips markdown fences, parses JSON, then checks each declared field for presence, type, numeric bounds, and enum membership.

func validateOutput(text string, schema map[string]interface{}) (map[string]interface{}, error) {
	// strip ```json fences, then:
	var parsed map[string]interface{}
	if err := json.Unmarshal([]byte(cleaned), &parsed); err != nil {
		return nil, fmt.Errorf("output is not valid JSON: %w\nRaw: %s", err, text)
	}
	for key, schemaDef := range schema {
		val, exists := parsed[key]
		if !exists {
			return nil, fmt.Errorf("missing required field: %s", key)
		}
		def := schemaDef.(map[string]interface{})
		switch def["type"] {
		case "int", "number":
			num := toFloat64(val)
			if num == nil {
				return nil, fmt.Errorf("field %s: expected number, got %T", key, val)
			}
			// min / max bounds checked here
		case "string":
			if _, ok := val.(string); !ok {
				return nil, fmt.Errorf("field %s: expected string, got %T", key, val)
			}
		case "bool":
			if _, ok := val.(bool); !ok {
				return nil, fmt.Errorf("field %s: expected bool, got %T", key, val)
			}
		}
		if rawEnum, ok := def["enum"]; ok {
			if !enumContains(rawEnum.([]interface{}), val) {
				return nil, fmt.Errorf("field %s: value %v not in allowed set", key, val)
			}
		}
	}
	return parsed, nil
}

A model that answers "urgency": "maybe" fails the enum check before the value ever reaches a downstream step. The same function runs in production and in the fixture test runner, so a test green means the real path is green.

draftcat validate walks config.yaml and every skills/*.yaml and reports structured findings. Each is error or warn with a path. --strict fails on any finding.

$ ./draftcat validate --strict
WARN  provider.api_key_env: env var OPENROUTER_API_KEY is empty (engine will refuse to start at runtime)
WARN  pipelines[2:lead-triage].steps[1:classify].vars: skill "triage-lead" references {{contact}}, not in vars and not a known upstream data key
WARN  skills/draft-followup: loaded but not referenced by any pipeline
WARN  skills/group-learning-items: loaded but not referenced by any pipeline

0 error(s), 17 warning(s)

Exit code is 0 on a clean run, 1 on any error, and 1 on any warning under --strict. Drop it straight into CI:

- run: go build -o draftcat .
- run: ./draftcat validate --strict     # fail the build on any skill/config finding

These are the messages the validator emits. Paste-into-Google ready:

missing required 'name' — a skill file has no name:. The registry keys on name, so a nameless skill is unreferenceable.

missing 'prompt' — a skill with a schema but no prompt. Nothing to send the model.

duplicate skill name (also defined in another file) — two files declare the same name:. The second silently overwrites the first at load time; the linter makes it an error.

output_schema.<field>: unsupported type "X" (validator handles int|number|bool|string) — you wrote type: array. The runtime validator only knows four types; anything else is unenforced, so it warns.

output_schema.<field>: missing 'type' — a field def with neither type nor enum. It validates nothing.

skills/<name>: loaded but not referenced by any pipeline — a dead skill. Not an error, but usually a sign of a rename you half-finished.

skill "X" references {{var}}, not in vars and not a known upstream data key — the prompt interpolates a variable no step provides. This is the one that silently ships an empty {{contact}} to the model in production.

1. validate exits 1 with 0 error(s) > you ran --strict and have warnings. Read them or drop --strict.
2. Skill not taking effect after an edit > the engine loaded skills at startup; restart the process.
3. duplicate skill name > two files, same name:. Rename one; the filename does not have to match name.
4. Model output rejected at runtime > validateOutput failed. The error names the field and reason (type / enum / missing).
5. Prompt renders with a literal {{contact}} > that variable was never passed; the validate warning predicted it.

• Vs string literals in Go: editing a prompt is a YAML diff and a restart, reviewable on its own, no rebuild.
• Vs a prompt-management SaaS: the skills live in your repo, version with your code, and need no network call to resolve. The schema and linter are 200 lines, not a vendor dependency.
• Vs an untyped prompt file: the output_schema makes the model's contract executable. A drifted response fails at the boundary, not three steps downstream.

This is Production AI Automation Notes #12. The series covers approval gates, token budgets, SQLite dedup, prompt-injection defense, deterministic step pipelines, and fixture testing — the discipline of running LLM agents in production.

• #1 Agent Approval Gates — proposed actions, schema validation, audit log
• #6 Prompt-Injection Defense — input sanitization + output schema validation
• #11 Pipeline Fixture Testing — dry-run pipelines from JSON fixtures; zero API calls in CI

Reference implementation: draftcat (Go, MIT). Follow @renezander030 for new entries.

How do you manage agent prompts? Drop a comment with: where prompts live (code / YAML / DB / SaaS), whether you validate model output against a schema, and the last time a prompt change broke something downstream.

• Initial publish. Covers the skill YAML format, the loader, output_schema runtime enforcement, the validate --strict linter, verbatim error strings, and debug flow.
• Skipped gates: hardware matrix and model-picks table (not hardware- or model-bound — this is a config/validation pattern).