Skip to content

fix(doctor): quiet tool policy removal audits#88100

Merged
steipete merged 4 commits into
openclaw:mainfrom
giodl73-repo:fix-87798-doctor-tool-policy-audit
May 31, 2026
Merged

fix(doctor): quiet tool policy removal audits#88100
steipete merged 4 commits into
openclaw:mainfrom
giodl73-repo:fix-87798-doctor-tool-policy-audit

Conversation

@giodl73-repo

@giodl73-repo giodl73-repo commented May 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Keep runtime tool-policy removal audits at the normal info level by default.
  • Lower diagnostic-only doctor tool-schema probes to debug so expected profile filtering does not pollute normal openclaw doctor output.
  • Keep actionable sandbox policy block audits at info.
  • Add regression coverage for the default audit level and the doctor-specific debug path.

Fixes #87798.

Validation

  • node scripts/run-vitest.mjs src/agents/tool-policy-pipeline.test.ts src/flows/doctor-core-checks.runtime.test.ts src/flows/doctor-core-checks.runtime-errors.test.ts src/commands/doctor/shared/active-tool-schema-warnings.test.ts
  • node scripts/run-vitest.mjs ui/src/ui/views/chat.test.ts src/agents/agent-bundle-mcp-runtime.test.ts
  • node scripts/test-projects.mjs test/vitest/vitest.gateway-methods.config.ts
  • OPENCLAW_VITEST_INCLUDE_FILE=/tmp/agent-chat-include.json node scripts/test-projects.mjs test/vitest/vitest.gateway-server.config.ts
  • pnpm lint --threads=8
  • git diff --check
  • node node_modules/oxfmt/bin/oxfmt --check ... on touched files
  • pnpm exec oxlint ... on touched files
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main

CI

Surface Area

  • No new config key, CLI flag, environment variable, migration, or compatibility surface.
  • Adds an internal per-call audit verbosity option for doctor probes only.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: XS maintainer Maintainer-authored PR labels May 29, 2026
@clawsweeper

clawsweeper Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 31, 2026, 7:30 PM ET / 23:30 UTC.

Summary
The branch adds an internal tool-policy audit log-level override, passes debug from doctor tool-schema probes, and adds tests that removal audits stay off the normal logger.

PR surface: Source +17, Tests +40. Total +57 across 9 files.

Reproducibility: yes. Current main source shows removal audits emitted at info level from the shared policy pipeline, and the linked report gives a concrete doctor run with tools.profile set to coding; I did not rerun doctor because this review was read-only.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • No automated repair is needed; the clean PR is ready for maintainer review or merge handling under the protected maintainer label.

Security
Cleared: The diff only changes internal TypeScript logging/test paths and does not touch dependencies, workflows, secrets, package resolution, or executable supply-chain surfaces.

Review details

Best possible solution:

Land the narrow doctor-only debug override after normal maintainer review so routine doctor probes stay quiet while actionable runtime and sandbox policy audits remain visible.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main source shows removal audits emitted at info level from the shared policy pipeline, and the linked report gives a concrete doctor run with tools.profile set to coding; I did not rerun doctor because this review was read-only.

Is this the best way to solve the issue?

Yes. The branch is the best narrow fix because it adds an internal diagnostic log-level override only for doctor probe paths instead of globally lowering all policy-removal audits or suppressing actionable sandbox policy blocks.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against db7aff884361.

Label changes

Label justifications:

  • P2: This is a normal-priority doctor output bug fix with limited blast radius and focused regression coverage.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix Crabbox proof of a real doctor command path with exit 0 and an explicit assertion that the normal-level removal audit was absent.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix Crabbox proof of a real doctor command path with exit 0 and an explicit assertion that the normal-level removal audit was absent.
Evidence reviewed

PR surface:

Source +17, Tests +40. Total +57 across 9 files.

View PR surface stats
Area Files Added Removed Net
Source 6 34 17 +17
Tests 3 43 3 +40
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 9 77 20 +57

What I checked:

  • Repository policy read: Read the full root AGENTS.md and the scoped src/agents/AGENTS.md; the review applied the OpenClaw guidance for full PR review, scoped agents guidance, protected-label handling, and avoiding changelog findings. (AGENTS.md:1, db7aff884361)
  • Current main behavior: Current main's auditToolPolicyFilter always emits removal audits through toolPolicyAuditLogger.info, which matches the linked issue's noisy normal-level doctor output. (src/agents/tool-policy-audit.ts:179, db7aff884361)
  • PR implementation: The PR head adds ToolPolicyAuditLogLevel and routes removal-audit messages to debug only when the new logLevel parameter is set to debug; default behavior remains info. (src/agents/tool-policy-audit.ts:196, d7063e7d578b)
  • Doctor-only wiring: The PR passes toolPolicyAuditLogLevel: "debug" from doctor runtime schema checks, while sibling runtime callers such as gateway tool resolution and embedded agent attempts still omit the override and keep info-level audits. (src/flows/doctor-core-checks.runtime.ts:686, d7063e7d578b)
  • Regression coverage: The new pipeline test asserts debug is called and info is not called for diagnostic-only policy probes, while adjacent tests continue asserting ordinary allow/deny removals use info. (src/agents/tool-policy-pipeline.test.ts:327, d7063e7d578b)
  • Logging contract: The logging subsystem defaults normal console output to info and only includes debug when verbose or debug logging is enabled, so moving these audits to debug keeps them out of normal doctor output. (src/logging/console.ts:29, db7aff884361)

Likely related people:

  • Vincent Koc: Local history and blame show recent work on the tool-policy audit pipeline and doctor runtime schema checks that this PR adjusts. (role: recent area contributor; confidence: high; commits: edc0a22179a7, dc05f598bb5a; files: src/agents/tool-policy-audit.ts, src/agents/tool-policy-pipeline.ts, src/flows/doctor-core-checks.runtime.ts)
  • Peter Steinberger: The timeline shows Peter assigned and force-pushed the current branch, and the final two PR commits scope the quieting to doctor paths. (role: recent PR follow-up owner; confidence: medium; commits: 276b158f364a, d7063e7d578b; files: src/flows/doctor-core-checks.runtime.ts, src/commands/doctor/shared/active-tool-schema-warnings.ts, src/agents/agent-tools.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels May 29, 2026
@giodl73-repo

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Removed the CHANGELOG.md entry per the rank-up feedback. Net PR diff is now only src/agents/tool-policy-audit.ts and src/agents/tool-policy-pipeline.test.ts. Local follow-up checks after the cleanup: git diff --check passed; oxfmt --check passed on touched source/test files.

@clawsweeper

clawsweeper Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels May 29, 2026
@steipete steipete self-assigned this May 31, 2026
@steipete steipete force-pushed the fix-87798-doctor-tool-policy-audit branch from 7a67af5 to ac457b5 Compare May 31, 2026 22:46
@steipete steipete force-pushed the fix-87798-doctor-tool-policy-audit branch from ac457b5 to d7063e7 Compare May 31, 2026 23:25
@openclaw-barnacle openclaw-barnacle Bot added commands Command implementations size: S and removed size: XS labels May 31, 2026
@openclaw-barnacle openclaw-barnacle Bot added the app: web-ui App: web-ui label May 31, 2026
@steipete

Copy link
Copy Markdown
Contributor

Land-ready proof for b7af1f56ec1638298fd01258c098e2d079f1851d:

  • Scoped the fix so normal runtime policy-removal audits stay at info, while doctor tool-schema probes pass the audit path as debug.
  • Rebased onto current origin/main; earlier gateway/UI/lint drift fixes are now either upstream or covered by this branch's test-only repair commits.
  • Local proof:
    • node scripts/run-vitest.mjs src/agents/tool-policy-pipeline.test.ts src/flows/doctor-core-checks.runtime.test.ts src/flows/doctor-core-checks.runtime-errors.test.ts src/commands/doctor/shared/active-tool-schema-warnings.test.ts
    • node scripts/run-vitest.mjs ui/src/ui/views/chat.test.ts src/agents/agent-bundle-mcp-runtime.test.ts
    • node scripts/test-projects.mjs test/vitest/vitest.gateway-methods.config.ts
    • OPENCLAW_VITEST_INCLUDE_FILE=/tmp/agent-chat-include.json node scripts/test-projects.mjs test/vitest/vitest.gateway-server.config.ts
    • pnpm lint --threads=8
    • git diff --check
    • touched-file oxfmt --check and oxlint
    • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main clean
  • CI green: https://github.com/openclaw/openclaw/actions/runs/26727664397
  • Real behavior proof green: https://github.com/openclaw/openclaw/actions/runs/26727667473

1 similar comment
@steipete

Copy link
Copy Markdown
Contributor

Land-ready proof for b7af1f56ec1638298fd01258c098e2d079f1851d:

  • Scoped the fix so normal runtime policy-removal audits stay at info, while doctor tool-schema probes pass the audit path as debug.
  • Rebased onto current origin/main; earlier gateway/UI/lint drift fixes are now either upstream or covered by this branch's test-only repair commits.
  • Local proof:
    • node scripts/run-vitest.mjs src/agents/tool-policy-pipeline.test.ts src/flows/doctor-core-checks.runtime.test.ts src/flows/doctor-core-checks.runtime-errors.test.ts src/commands/doctor/shared/active-tool-schema-warnings.test.ts
    • node scripts/run-vitest.mjs ui/src/ui/views/chat.test.ts src/agents/agent-bundle-mcp-runtime.test.ts
    • node scripts/test-projects.mjs test/vitest/vitest.gateway-methods.config.ts
    • OPENCLAW_VITEST_INCLUDE_FILE=/tmp/agent-chat-include.json node scripts/test-projects.mjs test/vitest/vitest.gateway-server.config.ts
    • pnpm lint --threads=8
    • git diff --check
    • touched-file oxfmt --check and oxlint
    • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main clean
  • CI green: https://github.com/openclaw/openclaw/actions/runs/26727664397
  • Real behavior proof green: https://github.com/openclaw/openclaw/actions/runs/26727667473

@steipete steipete merged commit bf777b9 into openclaw:main May 31, 2026
172 of 173 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling app: web-ui App: web-ui commands Command implementations maintainer Maintainer-authored PR P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: doctor emits repeated tool-policy removal audit lines at normal console level

2 participants