Skip to content

v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"])#1192

Merged
garrytan merged 5 commits into
masterfrom
garrytan/tehran
May 19, 2026
Merged

v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"])#1192
garrytan merged 5 commits into
masterfrom
garrytan/tehran

Conversation

@garrytan

Copy link
Copy Markdown
Owner

Summary

  • Shell-job inherit: ["database_url"] allowlist validated pre-enqueue in both submit surfaces (gbrain jobs submit shell CLI + submit_job MCP op). Worker resolves the value from its own loadConfig() at child-spawn time; persisted minion_jobs.data row stores names only.
  • env: { GBRAIN_DATABASE_URL | DATABASE_URL | GBRAIN_DIRECT_DATABASE_URL } rejected pre-enqueue with a paste-ready hint pointing at inherit:.
  • Inline-secret cmd/argv scan rejects cmd: "GBRAIN_DATABASE_URL=... gbrain sync" patterns (codex H1 — caught a real bypass pre-merge).
  • gbrain doctor home_dir_in_worktree warns when ~/.gbrain lives inside a git worktree (Conductor topology). Handles .git as directory or file.
  • ~/.gbrain/.gitignore retroactive via every saveConfig() + gbrain post-upgrade so existing users get the safety belt without re-running gbrain init.
  • New canonical guide: docs/guides/agent-to-gbrain.md with two-domain framing for downstream agent authors (MCP ops via OAuth vs localOnly admin ops via shell-job inherit:).

Closes #1137. Credit @WinterMute — their doc PR made the env-stripping gap visible enough to fix in code.

Test Coverage

53 new test cases across 5 files:

CODE PATHS                                              USER FLOWS
[+] shell-inherit.ts (NEW)                              [+] inherit: ["database_url"] flow
  ├── INHERITABLE record                                  ├── [★★★] CLI submit + worker resolution (E2E)
  │   ├── [★★★] read fn returns config.database_url       └── [★★★] submit_job op path + audit lift
  │   └── [★★★] shadow-key reverse lookup
[+] shell-validate.ts (NEW)
  ├── validateShellJobParams (T1 + T3 + F1 + H1 + H3)   [+] Defense-in-depth
  │   ├── [★★★] cmd XOR argv + cwd + env shape           └── [★★★] handler revalidates pre-existing rows
  │   ├── [★★★] inherit closed enum + unknown rejected
  │   ├── [★★★] env shadow rejection (T3)
  │   ├── [★★★] fail-fast on missing config (F1)
  │   ├── [★★★] cmd/argv inline-secret scan (H1)
  │   ├── [★★★] explicit shadow message (F3)
  │   └── [★★★] T1 regression guard (no persistence before throw)
[+] shell.ts (extended)
  ├── ShellJobParams.inherit field                      [+] Doctor + init
  ├── buildChildEnv resolution                            ├── [★★★] ~/.gbrain inside dir-style worktree
  └── handler defense-in-depth                            ├── [★★★] ~/.gbrain inside .git-as-file worktree
[+] config.ts:ensureGitignore (NEW)                       ├── [★★★] walk terminates at $HOME
  ├── [★★★] idempotent write                              └── [★★★] GBRAIN_HOME override outside worktree
  ├── [★★★] no clobber on user customization
  └── [★★★] honors GBRAIN_HOME
[+] doctor.ts:home_dir_in_worktree (NEW)

COVERAGE: 100% on new code (53 tests across 5 files)
QUALITY: ★★★:53

Tests: 0 → 53 new

Plan Completion

All 9 planned tasks complete. Plan: ~/.claude/plans/system-instruction-you-are-working-cozy-zebra.md.

  • T1 — shell-validate.ts pre-enqueue validator
  • T2 — shell-inherit.ts single-source INHERITABLE record
  • T3 — env-shadow rejection (pre-enqueue)
  • T4 — two-domain docs (minions-shell-jobs.md + agent-to-gbrain.md + UPGRADING)
  • T5 — ensureGitignore retroactive via saveConfig + runPostUpgrade
  • T6 — home_dir_in_worktree doctor check
  • T7 — submit_job audit-log lift
  • T8 — 53 unit + E2E tests
  • T9 — VERSION + CHANGELOG + migration + CLAUDE.md + llms.txt

Codex pre-landing review added H1 (cmd/argv inline scan) + H3 (GBRAIN_DIRECT_DATABASE_URL shadow) — both shipped with tests.

Pre-Landing Review

/plan-eng-review cleared the plan with 5 cross-model tensions resolved before implementation. Pre-landing codex caught H1 (cmd/argv inline-secret bypass — fixed) + H3 (missing direct-URL shadow — fixed) + H2 (output-side leakage — documented honestly in CHANGELOG + minions-shell-jobs.md#secrets). M1 (MinionQueue.add direct-call bypass) and M2 (distributed-worker config mismatch) are documented as known trade-offs aligned with the single-uid trust model.

Honest scope — what does NOT close in this PR

  • Output-side leakage (codex H2): result.stdout_tail / result.stderr_tail / error_text can carry the URL if the script prints it. Script author's responsibility. CHANGELOG + minions-shell-jobs.md#secrets say this explicitly. Future PR may add output-redaction.
  • API-key inheritance (anthropic/openai/groq/voyage): deferred. Needs GBrainConfig + buildGatewayConfig refactor.
  • Encrypted vault at rest: /cso audit cut as theater for single-uid topology. Revisit if backup/screenshare threat becomes concrete.

Verification Results

  • bun run verify — green (typecheck + 11 shell pre-checks)
  • bun run test — 7509 pass / 0 fail (full unit suite post-merge)
  • bun test test/minions-shell-validate.test.ts test/minions-shell-inherit.test.ts test/config-ensure-gitignore.test.ts test/doctor-home-dir-in-worktree.test.ts53/53 pass
  • E2E with real Postgres — all passing (one pre-existing flake on voyage-multimodal.test.ts hits the real Voyage API; unchanged from master, unrelated to this branch)

TODOS

No TODOS.md changes — out-of-scope follow-ups are inline in the plan file under "Out-of-scope follow-ups" (API-key inheritance, output-redaction, vault revisit).

Test plan

  • Submit shell job with inherit: ["database_url"] → child receives URL, row + audit log show names only
  • Submit shell job with env: { GBRAIN_DATABASE_URL: ... } → rejected pre-enqueue with hint
  • Submit shell job with cmd: "GBRAIN_DATABASE_URL=... gbrain sync" → rejected pre-enqueue (H1 fix)
  • Submit shell job with env: { GBRAIN_DIRECT_DATABASE_URL: ... } → rejected (H3 fix)
  • gbrain doctor warns when ~/.gbrain inside a worktree
  • gbrain post-upgrade lays down ~/.gbrain/.gitignore retroactively

🤖 Generated with Claude Code

garrytan and others added 5 commits May 18, 2026 21:45
…"database_url"])

Replaces PR #1137's plaintext-config / plaintext-env workarounds with code.
Shell-job params gain `inherit: ["database_url"]`, validated pre-enqueue in
both the CLI (`gbrain jobs submit`) and `submit_job` MCP op handler. Worker
resolves the value from its own loadConfig() at child-spawn time; the
persisted `minion_jobs.data` row stores only the name. Plain
`env: { GBRAIN_DATABASE_URL: ... }` / `env: { DATABASE_URL: ... }` /
`env: { GBRAIN_DIRECT_DATABASE_URL: ... }` are rejected pre-enqueue with a
paste-ready hint pointing at `inherit:`.

Codex pre-landing review caught two bypasses + one missing shadow name:
- H1: cmd/argv inline-secret regex scan (cmd:"GBRAIN_DATABASE_URL=... gbrain
  sync" was a clean bypass — fixed)
- H3: GBRAIN_DIRECT_DATABASE_URL added to shadowKeys
- H2: honest docs about output-side leakage (stdout_tail/stderr_tail can still
  carry the value if the script prints it; that's the script author's
  responsibility, not gbrain's)

Also: gbrain doctor learns home_dir_in_worktree (warns when ~/.gbrain lives
inside a git worktree); ~/.gbrain/.gitignore retroactive via saveConfig +
post-upgrade.

New canonical guide: docs/guides/agent-to-gbrain.md (two-domain framing for
downstream agent authors: MCP ops via OAuth vs localOnly admin ops via
shell-job inherit:).

Closes #1137. Tests: +53 new (21 validator + 12 inherit-record + 6
ensureGitignore + 5 doctor + 2 PGLite E2E + 7 codex-driven H1/H3 cases).

Credit: @WinterMute filed PR #1137 which made the env-stripping gap visible
enough to fix in code. Thank you.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Master added PR #1164 (v0.36.3.0 — dynamic embedding column selection) since
my last push. Merged into the branch with conflict resolution on VERSION /
package.json / CHANGELOG (kept v0.36.5.0 on top; v0.36.3.0 entry from master
lands at line 98).

Also: master's new check:privacy gate caught references to the agent-name in
my changes. Swept CHANGELOG.md, skills/migrations/v0.36.5.0.md, and
shell-validate.ts to use @garrytan-agents / PR #1137 attribution per CLAUDE.md.

Verify + 7590-test unit suite green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Master added PR #1193 (v0.36.4.0 — brain-health-100, autonomous remediation
via doctor --remediate + Minions) and PR #1201 (docs drift audit) since the
last merge. My branch stays at v0.36.5.0; CHANGELOG sequence is now
0.36.5.0 (mine) → 0.36.4.0 (master) → 0.36.3.0 → 0.36.2.0.

Resolved: VERSION + package.json (kept 0.36.5.0); CHANGELOG (mine on top,
strip markers, all entries preserved). All source files auto-merged cleanly.

verify green, llms regenerated.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
User feedback: "agent spawning minions should have agency to do what it wants
with secrets and pass only the ones that it needs. don't be a security nazi
please."

Replaces the closed INHERITABLE enum (database_url only) with three small
helpers in shell-inherit.ts:

- INHERIT_NAME_RE: snake_case shape guard. Rejects __proto__, leading
  underscore, uppercase, path-traversal. Prototype-pollution defense.
- deriveEnvKey(name): config-key → child-env-key. Uppercase by default with
  one override: database_url → GBRAIN_DATABASE_URL.
- resolveInheritValue(cfg, name): value lookup with Object.hasOwn.

inherit: now accepts any snake_case config-key the worker has. Agent picks
what it needs per-job (database_url, anthropic_api_key, voyage_api_key, or
any custom field). Validator does NOT police WHICH keys — single-uid trust
model treats agent as peer of worker.

Drops the v0.36.5.0-RC rules that were paternalistic for the actual threat
model:
- closed-enum check
- env-shadow rejection
- cmd/argv inline-secret scan

Keeps the parts that defend real problems:
- pre-enqueue validation (closes the persistence-before-throw window)
- snake_case regex (prototype-pollution + audit-log readability)
- fail-fast on missing config value (UX guardrail, not security)

Tests: shell-validate (existing rules + new free-form + prototype-pollution
defense + T1 regression guard) and shell-inherit (regex matrix, deriveEnvKey
per-name, resolveInheritValue with hasOwn defense). E2E case now exercises
inherit:["anthropic_api_key"] to prove genuinely free-form.

Docs and CHANGELOG rewritten to reflect the open design + the design-arc
story (closed → cut → free-form). Migration file too.

7653 unit tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Honest defense for the documented output-side leakage. When a script prints
an inherited secret, the value lands plaintext in
result.stdout_tail / result.stderr_tail / error_text. v0.36.5.0 adds:

- `redact_secrets: true` ShellJobParams field
- `--redact-secrets` CLI convenience flag on `gbrain jobs submit shell`
- shell-redact.ts: pure `redactSecretsInText(text, secrets)` helper
  (string-mode replaceAll; regex metachars in values stay literal)
- Handler post-processes both tails before throw/return, so the persisted
  row carries `<REDACTED:name>` tokens instead of values

Only inherit-resolved values are scrubbed. env: values are not (those are
the agent's "fine in the row" channel by design). Heuristic — defeats
accidental `echo "$GBRAIN_DATABASE_URL"`, not adversarial encode-then-print.
Default false for back-compat.

Tests:
- test/minions-shell-redact.test.ts (9 cases): pure-function behavior,
  regex-metachar safety, multi-secret independent redaction, substring
  overlap, empty-input/map edge cases
- test/minions-shell-validate.test.ts: +4 cases for redact_secrets shape
- test/e2e/minions-shell-pglite.test.ts: +2 cases proving redact_secrets:
  true scrubs persisted row AND redact_secrets:false preserves plaintext
  (back-compat regression guard)

Docs + CHANGELOG + migration file + CLAUDE.md updated.

7667 unit tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@garrytan garrytan merged commit e227965 into master May 19, 2026
7 checks passed
brandonlipman added a commit to brandonlipman/gbrain that referenced this pull request May 29, 2026
* upstream/master:
  v0.37.0.0 feat(skillpack): registry cathedral — third-party publish + install + 10/10 quality bar (garrytan#1208)
  v0.36.6.0 feat: cross-modal search wave (text↔image + unified column + LLM intent) (garrytan#1165)
  v0.36.5.0 feat: secure DATABASE_URL access for shell jobs (inherit: ["database_url"]) (garrytan#1192)
  v0.36.4.0 feat: brain-health-100 — autonomous remediation via doctor --remediate + Minions (garrytan#1193)
  fix(docs): comprehensive drift audit — contradictions, broken links, stale refs (garrytan#1201)
  v0.36.3.0 feat: dynamic embedding column selection for search (garrytan#1164)
  v0.36.2.0 feat: ZeroEntropy as default + zero-based README rewrite (garrytan#1136)
  v0.36.1.1 fix-wave: community PR triage + 28 atomic fixes (garrytan#1182)
  v0.36.1.0 Hindsight calibration wave: brain learns how you tend to be wrong (garrytan#1139)
  v0.36.0.0 feat(skillpack): scaffold + reference + harvest (retire managed-block install) (garrytan#1130)
  v0.35.8.0 feat(cycle): phantom-page redirect inside extract_facts (garrytan#1138)
  v0.35.7.0 feat: temporal trajectory + founder scorecard (Phases 2-4) (garrytan#1131)
  v0.35.6.0 feat(search): floor-ratio gate for metadata boost stages (closes garrytan#1091) (garrytan#1129)
  v0.35.5.1 fix(doctor): stop counting clean supervisor exits as crashes (garrytan#1108)
  v0.35.5.0 fix wave: bootstrap + orphans + think MCP + worktree + walker (garrytan#1111)
  v0.35.4.0 fix(doctor,entities): supervisor crash classification + bare-name resolver + 58x perf + stub guard observability (garrytan#1085)
  v0.35.3.1 feat(eval): temporal-aware contradiction probe + verdict enum (garrytan#1052)
  v0.35.3.0 fix wave: extract_facts items + git --no-recurse-submodules placement (garrytan#1053)

# Conflicts:
#	src/core/postgres-engine.ts
#	test/schema-bootstrap-coverage.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant