Skip to content

chore: audit tests and onboarding docs for obsolete LLM config references #2482

@alexey-pelykh

Description

@alexey-pelykh

Context

RemoteClaw is middleware that delegates LLM execution to CLI agents (Claude Code, Gemini, Codex, OpenCode). The pre-fork embedded execution path was gutted during early milestones. Scattered test files and documentation pages still reference obsolete LLM-level concepts (provider selection, model catalogs, thinking-level config, mocks of the removed embedded runner).

Problem

Orphan tests

runEmbeddedPiAgent is the primary function of the gutted embedded execution engine. Verification:

# Production imports of the function:
rg "import.*runEmbeddedPiAgent|from.*pi-embedded" src/ --type ts | grep -v "\.test\." | grep -v "\.cases\."
# No hits. Zero production imports.

# Tests that mock or reference it:
rg "runEmbeddedPiAgent" src/ --type ts

Known files to audit:

  • src/commands/agent.test.ts — thinking merge tests (expectDefaultThinkLevel, "prefers per-model thinking over global thinkingDefault", "defaults thinking to adaptive for Anthropic Claude 4.6 models"). Orphan. (May be removed in a concurrent thinking-gut PR — coordinate.)
  • src/auto-reply/reply.block-streaming.test.ts — entire file may be orphan (runs against runEmbeddedPiAgent mock). Audit and confirm.
  • src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.tsmixed: contains both real agent-runner exercises AND runEmbeddedPiAgentMock assertions. Orphan sections only; keep the live parts.
  • src/auto-reply/reply.triggers.trigger-handling.filters-usage-summary-current-model-provider.cases.ts — contains expect(runEmbeddedPiAgentMock).not.toHaveBeenCalled() tautologies. Delete tautological assertions, keep whatever real coverage exists.

Onboarding and help-text

The onboarding flow and help pages must not prompt the user for LLM provider selection, model choice, or LLM API keys — those are CLI-agent concerns. The user only configures: channel tokens (Telegram, Slack, WhatsApp, etc.), CLI auth env vars (provider-level env vars consumed by the CLI subprocess), and workspace/session paths.

Scope to audit:

  • docs/ directory
  • README.md at repo root
  • src/**/* help strings, onboarding prompts, wizard flows, CLI --help output
  • Translations (if any): search for the same concepts in docs/**/*-fr.md, -de.md, etc.

Grep patterns (case-insensitive):

rg -i "provider.*select|model.*picker|api.?key.*(anthropic|openai|google)" docs/ src/
rg -i "choose.*(provider|model)" docs/ src/
rg -i "thinking.?level|thinking.?default" docs/ src/

Tasks

Phase 1: Audit + clean tests

  • For each file in the "Known files to audit" list above, classify every test/assertion: (a) exercises live production code, or (b) orphan (asserts against runEmbeddedPiAgent mock or similar gutted paths)
  • Delete confirmed orphan test files entirely (where all tests are orphan)
  • For mixed files (agent-runner.runreplyagent.e2e.test.ts), extract the live parts and delete the runEmbeddedPiAgent-mock sections
  • Delete tautological assertions (expect(runEmbeddedPiAgentMock).not.toHaveBeenCalled() is vacuously true since the function is imported nowhere)
  • Verify: rg "runEmbeddedPiAgent" src/ --type ts returns zero hits (or only intentional stub-detection references)

Phase 2: Audit onboarding + docs

  • Grep docs/ and src/ for each pattern in the "Grep patterns" section above
  • For each match, classify:
    • LEGITIMATE — references channel-bound config (Telegram tokens, Slack OAuth, etc.) or CLI-auth env-var setup (e.g., "export ANTHROPIC_API_KEY before running Claude CLI")
    • OBSOLETE — references gutted embedded-execution features (e.g., "pick a model provider during setup")
    • STALE — references product strategy or features no longer present
  • Update or remove OBSOLETE and STALE matches
  • Preserve LEGITIMATE matches unchanged

Phase 3: Verification

  • Run full test suite; confirm test count drops (orphans removed) with no new skips
  • Spot-check onboarding end-to-end: a user running the setup wizard for a fresh install should see NO prompts for LLM provider, LLM model selection, or LLM API keys. Channel tokens and CLI auth env-var instructions are expected and correct.

Acceptance Criteria

  • rg "runEmbeddedPiAgent" src/ returns zero hits, or hits only in a single intentional stub-detection file with clear commentary
  • Grep of docs/ and onboarding sources for obsolete LLM concepts returns no OBSOLETE or STALE matches (LEGITIMATE channel/auth matches are fine)
  • tsc --noEmit clean
  • Test suite passes with a smaller test count (orphans removed) and no new skipped tests

Metadata

Metadata

Assignees

Labels

gutRemoving dead upstream subsystems

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions