Skip to content

v0.19.1 — smoke-test skillpack (post-restart health + auto-fix)#369

Merged
garrytan merged 5 commits intomasterfrom
feat/smoke-test-skillpack
Apr 24, 2026
Merged

v0.19.1 — smoke-test skillpack (post-restart health + auto-fix)#369
garrytan merged 5 commits intomasterfrom
feat/smoke-test-skillpack

Conversation

@garrytan
Copy link
Copy Markdown
Owner

@garrytan garrytan commented Apr 23, 2026

Summary

Adds gbrain smoke-test — a post-container-restart health check + auto-fix tool for gbrain + OpenClaw. Runs 8 built-in checks (Bun, gbrain CLI, doctor, worker, OpenClaw Codex Zod CJS, gateway, embedding API key, brain repo) with auto-fix, plus a user-extensible drop-in mechanism at ~/.gbrain/smoke-tests.d/*.sh.

Motivating bug: OpenClaw's Codex plugin ships a broken zod@4 on every container restart (missing core.cjs → every Codex ACP session fails). The workaround (npm install zod@4 --force) now runs automatically on every bootstrap via gbrain smoke-test.

Shipping as a patch (v0.19.1), not a minor, because the scope is one new CLI command + one skill — honest to size. Broader doctor-consolidation plans (v0.20 candidate) are tracked separately.

What's in the diff

smoke-test feature (commit 12a395b):

  • scripts/smoke-test.sh — 204-line bash runner with test-first/fix-after/re-test loop
  • skills/smoke-test/SKILL.md — contract, built-in test catalog, drop-in authoring pattern, known-issue database
  • src/cli.tsgbrain smoke-test dispatch (added to CLI_ONLY)
  • skills/manifest.json — skillpack registration

Merge + CI fixes (commits 67fc285, 89a567e, a67e159):

  • Merge master (resolved src/cli.ts conflict keeping all CLI_ONLY additions from both sides: smoke-test + master's skillpack/routing-eval/skillify)
  • skills/RESOLVER.md — smoke-test registered under Operational so resolver_health check passes
  • skills/smoke-test/SKILL.md — added ## Anti-Patterns and ## Output Format sections required by test/skills-conformance.test.ts
  • llms-full.txt — regenerated via bun run build:llms to match the new RESOLVER row (drift guard now passes)

Test Coverage

[+] scripts/smoke-test.sh (NEW, 204 lines bash, 8 checks)
  └── [GAP] No automated test — bash script, external side effects (bun install, npm install zod, gbrain jobs work).
            Project convention does not unit-test shell scripts. Verified manually + by CI import.

[+] src/cli.ts (smoke-test dispatch)
  └── [★ TESTED] Dispatch pattern covered by existing CLI_ONLY smoke tests.

[+] skills/manifest.json, skills/smoke-test/SKILL.md, skills/RESOLVER.md
  └── [★★ TESTED] test/skills-conformance.test.ts + test/resolver.test.ts + test/check-resolvable.test.ts

[+] llms-full.txt
  └── [★★ TESTED] test/build-llms.test.ts drift guard

COVERAGE: 4/6 paths tested (67%) — 2 gaps are both in the bash script (intentional, shell side effects)

Tests: 136 test files (unchanged — this PR adds no new test files; existing conformance + resolver + build-llms + skillpack-check tests exercise the skill file additions).

CI Status

Last CI run surfaced 8 failures, all addressed in commits 89a567e + a67e159:

  • gbrain skillpack-check (doctor exit 1 cascade) → fixed by RESOLVER wiring
  • RESOLVER.md > every manifest skill is reachable → fixed
  • checkResolvable — all manifest skills reachable → fixed
  • skills conformance > smoke-test > Anti-Patterns + Output Format → fixed
  • 1× Tier 1 E2E gbrain doctor exits 0 on healthy DB → fixed (same RESOLVER root cause)
  • build-llms > committed files match generator output → fixed by regen

Pre-Landing Review

Documented concerns for a future v0.20 consolidation pass (not blockers for this patch):

  • Shell script output uses emoji (✅❌🔧⏭️) — deviates from CLAUDE.md no-emoji rule
  • curl | bash auto-install of Bun on missing-runtime path — supply-chain pattern inside a health check
  • Hardcoded container paths (/data/gbrain, /app/node_modules/openclaw/...) — brittle to OpenClaw layout changes
  • Duplicates gbrain doctor for test 3 (shells out + greps)
  • No tests on 204 lines of bash

These are tracked in the v0.20 consolidation plan (doctor-as-single-health-surface, discussed separately).

Plan Completion

This PR ships the original smoke-test feature as-is (v0.19.1). The broader consolidation plan at ~/.claude/plans/lets-look-at-what-swirling-finch.md (folding smoke-test + skillpack-check + check-backlinks + lint + orphans + check-resolvable into gbrain doctor with severity + category tags) was not approved and is deferred to a later release.

TODOS

No TODO items completed by this PR. (Open items in TODOS.md are BrainBench Cat 5 + Cat 6, unrelated.)

Test plan

  • bun test test/skills-conformance.test.ts test/resolver.test.ts test/check-resolvable.test.ts test/skillpack-check.test.ts test/build-llms.test.ts — all 191 tests pass
  • bun test test/e2e/doctor-progress.test.ts test/progress.test.ts — all 17 run, 5 env-skipped, 0 fail
  • gbrain doctor exits 0 locally (resolver_health: OK, all 29 skills conformant)
  • gbrain skillpack-check returns healthy: true with empty actions

Other locally-failing tests (dream, orphans, progress CLI — all 5-10s subprocess timeouts) are env-dependent on DATABASE_URL; they skip gracefully in CI.

🤖 Generated with Claude Code

root and others added 5 commits April 23, 2026 17:32
Adds `gbrain smoke-test` CLI command that runs 8 health checks after
container restart, auto-fixes known issues, and reports results.

Built-in tests:
  1. Bun runtime (auto-install if missing)
  2. GBrain CLI loads (auto-reinstall deps)
  3. GBrain database connection (doctor health score)
  4. GBrain worker process (auto-start)
  5. OpenClaw Codex plugin Zod CJS (auto-reinstall broken zod@4)
  6. OpenClaw gateway responding
  7. Embedding API key present
  8. Brain repo exists

User-extensible: drop scripts in ~/.gbrain/smoke-tests.d/*.sh

Includes SKILL.md with full documentation, pattern for adding tests,
and known-issue database (e.g. Zod core.cjs publish bug).

Designed to run from OpenClaw bootstrap hooks so every container
restart automatically verifies and repairs the environment.
Resolves src/cli.ts CLI_ONLY set conflict: keeps all additions from both
sides (smoke-test from this branch; skillpack, routing-eval, skillify
from master).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes the 7 failing unit tests + 1 failing Tier 1 E2E:

- `skills/RESOLVER.md`: add smoke-test under Operational (mirrors
  skillpack-check placement). Fixes resolver_health check failure which
  cascaded into skillpack-check tests, doctor exit code, and the E2E
  'gbrain doctor exits 0 on healthy DB' assertion.

- `skills/smoke-test/SKILL.md`: add `## Anti-Patterns` and
  `## Output Format` sections required by skills-conformance.test.ts.

Root cause: PR #369 added skills/smoke-test/ to the manifest but never
wired it into RESOLVER.md and never added the sections the conformance
test requires for every manifest entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
build-llms drift guard (test/build-llms.test.ts:58) failed because
llms-full.txt inlines skills/RESOLVER.md and the last commit added a
smoke-test trigger row there. Regenerated via `bun run build:llms`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@garrytan garrytan changed the title feat: smoke-test skillpack — post-restart health checks + auto-fix v0.19.1 — smoke-test skillpack (post-restart health + auto-fix) Apr 24, 2026
@garrytan garrytan merged commit 246cd8b into master Apr 24, 2026
4 checks passed
garrytan added a commit that referenced this pull request Apr 24, 2026
Pulls upstream v0.19.1 (#369): smoke-test skillpack (post-restart health
+ auto-fix). New `smoke-test` command added to CLI_ONLY.

Conflicts resolved:
- VERSION — kept 0.21.0; upstream is 0.19.1
- package.json — v0.21.0 wins
- CHANGELOG.md — v0.21.0 preserved above upstream's v0.19.1
- src/cli.ts — CLI_ONLY merged: upstream's `smoke-test` + this branch's `providers`

Build clean: 0.21.0 binary runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
garrytan added a commit that referenced this pull request Apr 24, 2026
Master shipped its own v0.19.1 (smoke-test skillpack, PR #369) and then
v0.20.0 (BrainBench extraction to sibling repo, PR #195) while this
branch was in review. Bumping our release from v0.19.1 to v0.20.1
follows the CLAUDE.md rule: "VERSION must be higher than master's."

Resolved conflicts:

- VERSION: 0.19.1 (ours) + 0.20.0 (master) → 0.20.1
- package.json: same bump applied to the version field
- CHANGELOG.md: our queue-resilience entry renamed from v0.19.1 to
  v0.20.1 (6 inline refs updated across the body: numbers-that-matter
  table, "To take advantage" block, pre-v0.20.1 code reference,
  adversarial-review mention, and the v0.19.2 → v0.20.2 deferral
  reference for composite indexes). Entry stays at the top of the
  file, followed by master's v0.20.0 (BrainBench) and v0.19.1
  (smoke-test skillpack). Sequence is now 0.20.1 → 0.20.0 → 0.19.1 →
  0.19.0 → 0.18.2 → ...

Rebuilt binary reports gbrain 0.20.1.

Pre-merge verification carried forward:
- 143 minions + 13 doctor unit tests pass
- typecheck clean
- 189 E2E tests pass against real Postgres 16 + pgvector
- All 3 smoke cases pass (basic, --sigkill-rescue, --wedge-rescue)
- queue_health doctor check fires correctly on a forged stalled-forever job

No source changes — conflict resolution was version-label surgery only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Qodo-Free-For-OSS
Copy link
Copy Markdown

Hi, When bun is missing, the smoke-test script automatically executes a remote install script (curl ... | bash), creating a remote code execution path during bootstrap runs.

Severity: action required | Category: security

How to fix: Require explicit opt-in install

Agent prompt to fix - you can give this to your LLM of choice:

Issue description

The smoke-test script runs curl -fsSL https://bun.sh/install | bash automatically if bun is not found. This is a classic supply-chain/RCE footgun.

Issue Context

Smoke-test is intended for automated post-restart bootstrap, so this can execute without interactive review.

Fix Focus Areas

  • scripts/smoke-test.sh[58-72]

Implementation directions

  • Remove auto-install entirely, or gate it behind an explicit env var / flag (e.g., GBRAIN_SMOKE_ALLOW_BUN_INSTALL=1).
  • If keeping install: add integrity verification (pinned version + checksum) and fail closed when verification cannot be performed.
  • Log clearly when install is skipped due to missing opt-in.

We noticed a couple of other issues in this PR as well - happy to share if helpful.


Qodo code review - free for open-source.

ChenyqThu pushed a commit to ChenyqThu/jarvis-knowledge-os-v2 that referenced this pull request Apr 27, 2026
Merge upstream/master (commit 11abb24, gbrain v0.20.4) into KOS v2 fork.

Six upstream commits land:
- v0.19.0 check-resolvable OpenClaw fallback (garrytan#326)
- v0.19.1 smoke-test skillpack (garrytan#369)
- v0.20.0 BrainBench extracted to sibling repo (garrytan#195)
- v0.20.2 jobs supervisor (garrytan#364) — Postgres-only, PGLite skips
- v0.20.3 queue resilience + queue_health doctor (garrytan#379) — Postgres-only
- v0.20.4 minion-orchestrator skill consolidation (garrytan#381)

Conflicts resolved (2 real, 5 auto):
- .gitignore: union both fork (.omc/, kos-jarvis log globs) and upstream
  (eval/data/world-v1/world.html, amara-life-v1 cache) entries.
- skills/manifest.json: append upstream's smoke-test skill plus retain
  the 9 kos-jarvis fork skills (39 total).
- CLAUDE.md / README.md / package.json (0.20.4) / skills/RESOLVER.md /
  src/cli.ts (mode 0755) auto-merged cleanly.

Fork-local patches preserved (verified post-merge):
- src/core/pglite-schema.ts:65 — idx_pages_source_id commented out
  (upstream garrytan#370 still open, fix retained).
- src/core/pglite-engine.ts:87 — pg_switch_wal() before close()
  (WAL durability patch, no upstream issue filed yet).
- src/cli.ts mode 100755 — bun shim executable bit.

Issue garrytan#332 (v0_13_0 process.execPath) fixed upstream in v0.19.0 ...
running gbrain apply-migrations --yes will clear the partial-ledger
remainder that has been stuck in doctor since the v0.13 sync.

v0.20's headline features (jobs supervisor, queue_health, wedge-rescue,
backpressure-audit) are Postgres-only and skip on our PGLite engine.
Sync is preventive ... keeps the fork mergeable rather than buying new
runtime capability.

Pre-merge baseline (HEAD 170876f):
- pages 1988, chunks 3750 (100% embedded), links 8522, timeline 10881
- doctor health 60/100 (failed: minions_migration partial 0.13.0)
- brain_score 86/100

Rollback: git tag pre-sync-v0.20-1777105378
PGLite snapshot: ~/.gbrain/brain.pglite.pre-sync-v0.20-1777105391 (416M)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants