A governed learning loop for production voice agents: harvest call Learning-Items, group them, propose a workflow/prompt diff, gate it behind two human approvals, then publish an auto-versioned workflow.

Last tested: June 2026. See Changelog at the bottom.

If this saves you setup time, follow @renezander030 — production AI agent governance in Go.

Reference implementation (MIT, Go): github.com/renezander030/draftcat — build with -tags voice.

A voice agent that edits its own prompts unsupervised is a liability: one bad call summary and the next 200 callers hear the regression. But never updating the prompt means every gap a caller exposes stays open. The fix is not autonomy and it is not a frozen prompt. It is a loop with a human in the middle of the diff.

This is how draftcat closes that loop for a DACH (German/Austrian/Swiss) phone-screening agent running on Dograh: the agent flags what it didn't know during calls, a scheduled pipeline clusters those flags, an LLM proposes a minimal workflow change, and a human approves the diff before it is committed and again before it ships to prod.

• Two approval gates, not zero and not one. Gate the diff (is this the right change?) and the deploy (did staging prove it?). One gate lets an approved-but-broken change reach callers.
• Propose minimal diffs, never rewrites. The prompt that proposes the update is instructed to touch only the highest-priority cluster. A model asked to "improve the workflow" rewrites the whole graph and you lose review tractability.
• The git repo is the source of truth, not the LLM output. The committed workflow JSON is the exact file the publish step reads. The model proposes; git records; Dograh versions.

One scheduled guardrail pipeline (hourly) that harvests Learning-Items and walks the 7-step review. Calls themselves run on a separate 5-minute pipeline. Build draftcat with the voice tag; without it the lean binary is unchanged.

# Build with the voice plugin (adds 5 webhook receivers + 7 pipeline actions)
go build -tags voice -o draftcat
go test  -tags voice ./...

# config.yaml — voice block
voice:
  enabled: true
  listen_addr: 127.0.0.1:8080
  auth:
    method: bearer
    token_env: DRAFTCAT_VOICE_TOKEN   # must match Dograh's BEARER_TOKEN credential
  dograh:
    base_url: https://dograh.client.internal
    staging_url: https://dograh-staging.client.internal
    api_key_env: DOGRAH_API_KEY
budgets:
  per_day_calls: 200
  per_day_call_minutes: 1500

While a call runs, the voice orchestrator posts a Learning-Item the moment the agent hits a gap — a question it couldn't answer, a price it didn't know, a phrase it misheard. Draftcat owns governance; the orchestrator just posts JSON.

curl -sS -X POST http://127.0.0.1:8080/voice/learning \
  -H "Authorization: Bearer $DRAFTCAT_VOICE_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{
    "session_id": "call-8842",
    "description": "Caller asked about VPC pricing twice; agent had no answer",
    "category": "knowledge_gap",
    "severity": "medium"
  }'
# {"ok": true}

This persists to the voice_learnings table. Nothing acts on it yet — that is the point. Raw flags accumulate; a human decides when and what to change.

Reviewing 100 individual Learning-Items is operator-hostile. The first AI step groups items that describe the same gap. The skill is a plain YAML file (prompt + output schema + role):

# skills/group-learning-items.yaml
name: group-learning-items
description: Cluster new voice-AI Learning-Items by topic so the operator reviews groups, not individual rows
role: voice-analyst
prompt: |
  You are reviewing {{voice_learning_count}} new Learning-Items emitted by the
  voice agent over the last review window. Cluster items that describe the
  same underlying gap (e.g. "agent did not know our VPC pricing", "caller
  asked about VPC pricing twice"). Return a compact summary for the operator.

  Learning-Items:
  {{voice_learnings}}

  Respond with ONLY valid JSON:
  - groups_json: JSON string of an array of {topic, learning_ids, count, suggestion}
  - top_priority: single-sentence pointer to the most impactful group
  - groups_count: integer
output_schema:
  groups_json:    {type: string}
  top_priority:   {type: string}
  groups_count:   {type: int}

The operator now reviews "VPC pricing came up 14 times" — one group — instead of 14 rows.

After the operator approves the grouping, a second AI step proposes a change. The prompt explicitly forbids a full rewrite:

# skills/propose-prompt-update.yaml (prompt body)
Based on the grouped Learning-Items below and the operator's approval to act,
propose a minimal, surgical change to the voice workflow definition. Do not
rewrite the whole graph. Touch only the agentNode prompts or branching edges
that fix the highest-priority group.

It emits the full updated proposed_definition ({nodes, edges}), the target workflow_id / workflow_uuid, and a git commit_message. Output is schema-validated before anything downstream sees it.

This is the whole loop as a deterministic pipeline. Note the two approval steps — diff first, deploy second — and that the git commit sits between them so the committed file is exactly what gets published.

# fixtures/voice-dach-screener/guardrail.yaml
pipelines:
  - name: voice-guardrail-review
    schedule: 1h
    steps:
      - {name: harvest-learnings, type: deterministic, action: voice_learnings_new, vars: {limit: "100"}}
      - {name: group-similar,     type: ai,            skill: group-learning-items}
      - {name: review-grouping,   type: approval,      mode: hitl, channel: telegram}   # gate 1: is this the right gap?
      - {name: propose-update,    type: ai,            skill: propose-prompt-update}
      - {name: approve-diff,      type: approval,      mode: hitl, channel: telegram}   # gate 2: is this the right change?
      - name: commit
        type: deterministic
        action: git_commit_workflow_update
        vars: {path: workflows/dach-screener.json, content_var: proposed_definition, message_var: commit_message, repo_dir: /var/lib/dograh-workflows}
      - {name: smoke,          type: deterministic, action: dograh_staging_smoke, vars: {workflow_uuid_var: workflow_uuid}}
      - {name: approve-deploy, type: approval,      mode: hitl, channel: telegram}      # gate 3: did staging prove it?
      - name: publish
        type: deterministic
        action: dograh_prod_publish
        vars: {workflow_id_var: workflow_id, definition_path: /var/lib/dograh-workflows/workflows/dach-screener.json}
      - {name: notify-done, type: deterministic, action: notify}

Each approval step pauses the run and pings Telegram. Nothing past a gate executes until a human responds. The dograh_prod_publish PUT auto-versions on Dograh's side, so prior workflow versions are preserved and a rollback is a re-publish of the previous git commit.

• Why not autonomous self-editing? A self-editing agent can regress 200 callers between two evals. The blast radius of a voice prompt is every subsequent call; the diff has to be cheap to review and impossible to ship unreviewed.
• Why not eval-only (no prompt change)? Then every gap a caller exposes stays open. Evals tell you that something is wrong; this loop produces the change and ships it safely.
• Why git + Dograh versioning instead of just overwriting the prompt? Because "what changed, who approved it, when" has to survive an audit (Art. 22 / automated-decision provenance for EU callers). The commit message + the auto-versioned workflow give you that for free.

undefined: voiceRoutes / receivers do nothing — you built without the tag. Voice endpoints only exist under go build -tags voice. The lean binary has no /voice/* routes by design.

401 Unauthorized on every /voice/* POST — the Authorization: Bearer token does not match. voice.auth.token_env in draftcat and Dograh's BEARER_TOKEN webhook credential must be the same value.

Dograh refuses the webhook — Dograh requires HTTPS webhooks. Run draftcat behind Caddy or nginx for TLS; 127.0.0.1:8080 is the upstream, not the public URL.

pre_call_context times out — the pre-call enrichment has a sub-300ms p95 budget. If your account lookup is slower, return a partial context rather than blocking the greeting.

With draftcat running and a bearer token set, fake a completed call and watch it flow to operator approval:

export TOKEN="$DRAFTCAT_VOICE_TOKEN"
SID="smoke-$(date +%s)"

curl -sS -X POST http://127.0.0.1:8080/voice/session_start \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "{\"id\":\"$SID\",\"caller_phone\":\"+491234567890\",\"workflow\":\"smoke\"}"

curl -sS -X POST http://127.0.0.1:8080/voice/session_end \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "{\"session_id\":\"$SID\",\"outcome\":{\"decision\":\"qualified\",\"score\":85},\"cost_cents\":1200}"

Pass criteria: both POSTs return {"ok": true}, and the next run of the voice-dach-screener pipeline harvests the session and pings Telegram for approval before any downstream write.

• One approval gate instead of two. Approving the diff is not the same as approving the deploy. Without the staging smoke + second gate, an approved-but-broken change reaches callers.
• Letting the model rewrite the whole workflow graph. You lose the ability to review the diff. Constrain the proposal prompt to the highest-priority cluster only.
• Skipping git commit and PUTing the LLM output straight to prod. Then your source of truth is a model response you can't git revert. Commit first; publish the committed file.

This is Production AI Automation Notes #14 — a series on shipping AI agents outside demos. Reference implementation: draftcat (Go, MIT).

Related entries:

• #1: Agent Approval Gates — the JSON-schema + human-review + deterministic-dispatch pattern this loop is built on.
• #11: Test LLM agent pipelines against fixtures, zero API calls
• #12: LLM skills as YAML — prompt + output_schema + role — the skill format the two AI steps above use.
• #13: Inbound agent webhook auth in Go — constant-time bearer token — the bearer check behind the /voice/* receivers.

Follow @renezander030 for the next entry.

• draftcat docs/voice.md, skills/*.yaml, fixtures/voice-dach-screener/ (this gist's reference implementation)
• Dograh — open-source voice-AI orchestrator; webhook nodes + PUT /api/v1/workflow/{id} auto-versioning

Running a self-improving loop on a different voice stack (Pipecat-direct, Vapi, Retell, LiveKit Agents)? Comment with: orchestrator, how you capture Learning-Items, how many approval gates you run, and the worst regression a missing gate ever shipped. I'll fold the patterns back in.

• Initial publish. Self-improving voice agent loop with human-approved prompt diffs, backed by draftcat's -tags voice plugin (5 receivers, 7 pipeline actions, 3 skills, 7-step guardrail).
• Deliberate gate skips: no hardware×model table (gate 13 — not hardware-bound); no companion repo (draftcat is the reference repo, consistent with #11–#13); no failure-diagnosis tree beyond the common-failures section (gate 20 — the 7-step pipeline order already documents the flow).