Skip to content

Policy: add tool posture conformance checks#85482

Merged
giodl73-repo merged 11 commits into
openclaw:mainfrom
giodl73-repo:policy-tool-posture-conformance
May 23, 2026
Merged

Policy: add tool posture conformance checks#85482
giodl73-repo merged 11 commits into
openclaw:mainfrom
giodl73-repo:policy-tool-posture-conformance

Conversation

@giodl73-repo

@giodl73-repo giodl73-repo commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

#85096 is merged; this PR is now rebased directly on main. This PR adds config-only policy conformance checks for global and per-agent tool posture. The goal is policy conformance over existing OpenClaw config, not runtime tool enforcement and not a second policy engine.

For anyone new to the Policy plugin, the CLI surface is provided by the bundled Policy plugin:

openclaw plugins enable policy
openclaw policy check
openclaw policy check --json
openclaw policy check --severity-min error
openclaw doctor --lint

openclaw policy check runs only the policy checks and emits evidence, findings, and attestation hashes. The same findings also flow into openclaw doctor --lint when the Policy plugin is enabled.

This PR adds the following policy.jsonc syntax:

{
  "tools": {
    "profiles": { "allow": ["messaging", "minimal"] },
    "fs": { "requireWorkspaceOnly": true },
    "exec": {
      "allowSecurity": ["deny", "allowlist"],
      "requireAsk": ["always"],
      "allowHosts": ["sandbox"]
    },
    "elevated": { "allow": false },
    "denyTools": ["group:runtime", "group:fs"]
  }
}

The checks observe tools.profile, tools.allow, tools.alsoAllow, tools.deny, tools.fs.workspaceOnly, tools.exec.security, tools.exec.ask, tools.exec.host, tools.elevated.enabled, and per-agent agents.list[].tools.* overrides. Tool posture policy does not read runtime/operator approval state such as exec-approvals.json, and it does not enforce tool calls at runtime.

The tool-group membership used by these checks is local to the Policy plugin. This does not add a public openclaw/plugin-sdk/* subpath, package export, SDK docs row, or generated SDK baseline change.

Verification

  • git fetch fork policy-tool-posture-conformance
  • git rebase fork/policy-tool-posture-conformance
  • git log --show-signature -1 --format=fuller (signed merge head 2a4ee05; local keyring lacks GitHub public key, prior author commit ae17007 has good signature)
  • node scripts/run-vitest.mjs extensions/policy/src/doctor/register.test.ts extensions/policy/src/cli.test.ts -- --reporter=dot (149 passed)
  • pnpm exec oxfmt --check --threads=1 extensions/policy/src/tool-policy-conformance.ts extensions/policy/src/policy-state.ts extensions/policy/src/doctor/register.ts extensions/policy/src/doctor/register.test.ts docs/cli/policy.md docs/plugins/reference/policy.md src/config/types.tools.ts
  • pnpm docs:check-mdx docs/cli/policy.md docs/plugins/reference/policy.md
  • git diff --check
  • /mnt/c/src/claws-hapi/.agents/skills/autoreview/scripts/autoreview --mode branch (clean: no accepted/actionable findings on 2a4ee05)

Real behavior proof

Behavior addressed: Policy-only conformance for configured global/per-agent tool posture without runtime enforcement, including alsoAllow posture in evidence and attestation.
Real environment tested: WSL checkout at /root/src/openclaw-policy-tools-conformance on Ubuntu-24.04.
Exact steps or command run after this patch: direct policyCheckCommand({ cwd, json: true, severityMin: "error" }) against temp OpenClaw configs under /tmp/policy-proof-85482-refresh, plus focused policy tests for alsoAllow attestation behavior.
Evidence after fix: A nonconforming tool config produced profile, fs workspace-only, exec security/ask/host, elevated, and required-deny findings; the accepted posture produced ok: true with zero findings; a global/per-agent alsoAllow change is now recorded in toolPosture evidence and changes the accepted attestation.
Observed result after fix: Policy reports and clears expected findings for tool profile, filesystem workspace-only posture, exec security/ask/host posture, elevated mode, required deny tools, inherited global posture, alias normalization, group coverage, sandbox-sensitive omitted exec defaults, and alsoAllow attestation drift.
What was not tested: Broad CI and runtime tool execution; this PR is config conformance only and intentionally does not add runtime enforcement.

Redacted CLI evidence

Nonconforming tool posture:

{
  "exitCode": 1,
  "ok": false,
  "checksRun": 37,
  "findings": [
    {
      "checkId": "policy/tools-profile-unapproved",
      "message": "global tools config uses unapproved tool profile 'coding'.",
      "ocPath": "oc://openclaw.config/tools/profile"
    },
    {
      "checkId": "policy/tools-fs-workspace-only-required",
      "message": "global tools config does not require workspace-only filesystem tools.",
      "ocPath": "oc://openclaw.config/tools/fs/workspaceOnly"
    },
    {
      "checkId": "policy/tools-exec-security-unapproved",
      "message": "global tools config uses unapproved exec security 'full'.",
      "ocPath": "oc://openclaw.config/tools/exec/security"
    },
    {
      "checkId": "policy/tools-exec-ask-unapproved",
      "message": "global tools config uses unapproved exec ask 'off'.",
      "ocPath": "oc://openclaw.config/tools/exec/ask"
    },
    {
      "checkId": "policy/tools-exec-host-unapproved",
      "message": "global tools config uses unapproved exec host 'gateway'.",
      "ocPath": "oc://openclaw.config/tools/exec/host"
    },
    {
      "checkId": "policy/tools-elevated-enabled",
      "message": "global tools config permits elevated tool mode.",
      "ocPath": "oc://openclaw.config/tools/elevated/enabled"
    },
    {
      "checkId": "policy/tools-required-deny-missing",
      "message": "global tools config does not deny required tool 'exec'.",
      "ocPath": "oc://openclaw.config/tools/deny"
    }
  ]
}

Accepted tool posture:

{
  "exitCode": 0,
  "ok": true,
  "checksRun": 37,
  "findings": []
}

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation channel: discord Channel integration: discord channel: slack Channel integration: slack channel: whatsapp-web Channel integration: whatsapp-web extensions: diagnostics-otel Extension: diagnostics-otel extensions: memory-lancedb Extension: memory-lancedb channel: twitch Channel integration: twitch extensions: acpx extensions: codex extensions: amazon-bedrock extensions: anthropic-vertex extensions: diffs extensions: policy size: XL maintainer Maintainer-authored PR labels May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

Latest ClawSweeper review: 2026-05-23 21:23 UTC / May 23, 2026, 5:23 PM ET.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

PR Surface
Source +834, Tests +604, Docs +53. Total +1491 across 7 files.

View PR surface stats
Area Files Added Removed Net
Source 4 841 7 +834
Tests 1 604 0 +604
Docs 2 58 5 +53
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 7 1503 12 +1491

Summary
The PR adds Policy plugin config-only conformance checks for global and per-agent tool posture, with evidence, attestation inputs, tests, docs, and tool config comments.

Reproducibility: not applicable. as a bug reproduction; this is a feature PR for new Policy conformance behavior. The PR body provides real-output proof for the new checks, and this review inspected the source and patch rather than executing the read-only checkout.

PR rating
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Summary: The PR has strong real-output proof and no discrete blocking finding, but merge confidence is capped by the unresolved Policy contract/security review.

Rank-up moves:

  • Get maintainer approval for the new tool-posture Policy contract and local group-map boundary.
  • Decide whether a drift guard is required for Policy tool-group expansion before merge.
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Sufficient (live_output): The PR body includes redacted live output showing both failing nonconforming tool posture and a passing accepted posture after the patch.

Risk before merge

  • The new policy syntax, finding IDs, toolPosture evidence shape, and attestation inputs become operator-facing once merged, so a wrong contract can create false compliance or unexpected drift failures.
  • The Policy-local static tool-group map can drift from core tool-group semantics unless maintainers accept that boundary or require a guard.
  • Enabled Policy workspaces may see new openclaw doctor --lint findings or attestation hash changes after upgrade.

Maintainer options:

  1. Confirm Policy contract before merge (recommended)
    Maintainers can explicitly accept the new syntax, check IDs, evidence fields, local group map, and attestation behavior as the operator contract.
  2. Add a tool-group drift guard
    If maintainers do not want a hand-maintained local map, add a focused guard that fails when Policy group expansion diverges from the core tool catalog.
  3. Pause for Policy stack sequencing
    Maintainers can hold this until the related Policy posture and docs PRs are reviewed as one coherent operator-facing stack.

Next step before merge
The protected maintainer label and security-sensitive Policy contract make this a maintainer decision item, not an automated repair candidate.

Security
Needs attention: The diff is security-sensitive because it changes Policy audit evidence and attestation semantics for configured tool posture.

Review details

Best possible solution:

Land only after maintainer approval of the Policy tool-posture contract, with any chosen drift guard or stack sequencing handled before merge.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction; this is a feature PR for new Policy conformance behavior. The PR body provides real-output proof for the new checks, and this review inspected the source and patch rather than executing the read-only checkout.

Is this the best way to solve the issue?

Unclear until maintainer review confirms the contract. The implementation is plausibly scoped to the bundled Policy plugin and config-only, but the local tool-group map and attestation semantics need explicit acceptance or a drift guard.

Label justifications:

  • P2: This is a normal-priority Policy plugin feature with limited runtime blast radius but meaningful operator-contract review needs.
  • merge-risk: 🚨 compatibility: The PR introduces new policy syntax, finding IDs, evidence, and attestation inputs that enabled operators may depend on after merge.
  • merge-risk: 🚨 security-boundary: The checks describe tool security posture and can affect compliance/audit confidence if the contract or group map is wrong.
  • merge-risk: 🚨 automation: Policy watch, doctor lint, and attestation-hash workflows can start reporting new findings or drift after the change.
  • rating: 🦐 gold shrimp: Current PR rating is 🦐 gold shrimp because proof is 🦞 diamond lobster, patch quality is 🦐 gold shrimp, and The PR has strong real-output proof and no discrete blocking finding, but merge confidence is capped by the unresolved Policy contract/security review.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes redacted live output showing both failing nonconforming tool posture and a passing accepted posture after the patch.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted live output showing both failing nonconforming tool posture and a passing accepted posture after the patch.

Security concerns:

  • [medium] Confirm the Policy-local tool-group map — extensions/policy/src/tool-policy-conformance.ts:1
    The PR adds a static Policy-local group map while current core groups are generated from the core tool catalog; if those diverge, Policy can report false compliance or false drift for security-relevant tool posture.
    Confidence: 0.78

What I checked:

  • Current main lacks this Policy tool-posture surface: On current main, POLICY_CHECK_IDS includes existing Policy checks through agent workspace and secret/auth/tool-metadata checks, but it does not include the new policy/tools-* posture checks proposed by the PR. (extensions/policy/src/doctor/register.ts:51, 05c6e7a55391)
  • Current evidence collection has no toolPosture evidence: Current collectPolicyEvidence only conditionally includes gateway exposure, agent workspace, secrets, auth profiles, and TOOLS.md evidence; the PR adds toolPosture as a new evidence family. (extensions/policy/src/policy-state.ts:195, 05c6e7a55391)
  • PR diff adds operator-facing Policy checks: The patch adds supported posture fields, registers seven new policy/tools-* health checks, conditionally collects toolPosture evidence, and feeds those findings into Policy evaluation. (extensions/policy/src/doctor/register.ts:78, 2a4ee05eaba1)
  • Tool-group drift is a real contract question: Current core tool groups are generated from CORE_TOOL_DEFINITIONS, while the PR adds a Policy-local static POLICY_TOOL_GROUPS map; that supports the maintainer-review question about accepting the local map or adding a drift guard. (src/agents/tool-catalog.ts:338, 05c6e7a55391)
  • Real behavior proof is present in the PR body: The PR body includes redacted live output showing nonconforming posture findings and an accepted posture with zero findings, plus a stated command-handler run against temp OpenClaw configs. (2a4ee05eaba1)
  • Maintainer handling is explicitly requested: The provided GitHub context shows the PR has the protected maintainer label, and the author's latest comment asks @galiniliev to review the narrow Policy contract question.

Likely related people:

  • steipete: Local git log/git blame on the current Policy files attributes the current main snapshot to recent commits by Peter Steinberger, including the Policy files touched by this PR. (role: recent area contributor; confidence: medium; commits: d9f73cfe3349, e510042870cf; files: extensions/policy/src/doctor/register.ts, extensions/policy/src/policy-state.ts, extensions/policy/src/doctor/register.test.ts)
  • giodl73-repo: The provided related-item context shows this contributor authored recently merged Policy conformance PRs that this PR builds on, including Gateway exposure and agent workspace conformance. (role: Policy conformance stack contributor; confidence: high; commits: dcc5e45b5006, a94f3444a085, 87bb37325b4a; files: extensions/policy/src/doctor/register.ts, extensions/policy/src/policy-state.ts, extensions/policy/src/doctor/register.test.ts)
  • galiniliev: The author explicitly asked this reviewer to look at the narrow operator-facing Policy contract question; I did not find local code-history proof for this path in the shallow checkout. (role: requested contract reviewer; confidence: low; files: extensions/policy/src/doctor/register.ts, extensions/policy/src/policy-state.ts, extensions/policy/src/tool-policy-conformance.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 05c6e7a55391.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@socket-security

socket-security Bot commented May 22, 2026

Copy link
Copy Markdown

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request

@giodl73-repo giodl73-repo force-pushed the policy-tool-posture-conformance branch from 845ee6c to 21990bb Compare May 22, 2026 19:39
@openclaw-barnacle openclaw-barnacle Bot removed channel: discord Channel integration: discord channel: slack Channel integration: slack channel: whatsapp-web Channel integration: whatsapp-web extensions: diagnostics-otel Extension: diagnostics-otel extensions: memory-lancedb Extension: memory-lancedb channel: twitch Channel integration: twitch extensions: acpx labels May 22, 2026
@giodl73-repo

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@giodl73-repo giodl73-repo force-pushed the policy-tool-posture-conformance branch from 47accbb to 1343d77 Compare May 23, 2026 16:33
@github-actions github-actions Bot removed the dependencies-changed PR changes dependency-related files label May 23, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the scripts Repository scripts label May 23, 2026
@giodl73-repo giodl73-repo force-pushed the policy-tool-posture-conformance branch from 1343d77 to 7a33874 Compare May 23, 2026 16:39
@giodl73-repo

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@giodl73-repo giodl73-repo force-pushed the policy-tool-posture-conformance branch from 7a33874 to 6f956cd Compare May 23, 2026 17:15
@giodl73-repo giodl73-repo force-pushed the policy-tool-posture-conformance branch from 6f956cd to 87bb373 Compare May 23, 2026 17:31
@giodl73-repo

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@giodl73-repo

Copy link
Copy Markdown
Contributor Author

@galiniliev when you review this one, the contract question is intentionally narrow:

  • This is config-only Policy conformance, not runtime tool enforcement.
  • The new syntax/check IDs/evidence cover tool profile, fs workspaceOnly, exec security/ask/host, elevated, denyTools, and alsoAllow.
  • Tool groups are local to the Policy plugin in this PR, so this does not add a new public SDK subpath/export.

ClawSweeper has no discrete code finding on the current head, but it is asking for maintainer confirmation of that operator-facing policy contract and whether a separate drift guard for tool-group expansion is required before merge.

@galiniliev galiniliev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current head 2a4ee05eaba1764ddaf4d58382422c0764396082.

No blocking findings. I’m comfortable with this as a config-only Policy conformance contract: the new tools.* policy syntax observes configured global/per-agent posture, records alsoAllow in toolPosture evidence for attestation drift, and stays inside the bundled Policy plugin without adding a new public SDK subpath or core dependency. The Policy-local tool group map is acceptable for this operator-facing check surface; if we later want it to be canonical across runtime and Policy, that can be a follow-up guard/refactor rather than a blocker here.

What I checked:

  • extensions/policy/src/doctor/register.ts for check registration, policy shape validation, finding IDs, and conformance logic.
  • extensions/policy/src/policy-state.ts and extensions/policy/src/tool-policy-conformance.ts for evidence shape, inherited agent/global posture, alsoAllow, elevated, fs, exec, and deny group handling.
  • Adjacent runtime contracts in src/agents/tool-policy-shared.ts, src/agents/tool-fs-policy.ts, src/agents/sandbox-tool-policy.ts, src/agents/exec-defaults.ts, src/agents/bash-tools.exec-runtime.ts, and src/auto-reply/reply/reply-elevated.ts.
  • Docs updates in docs/cli/policy.md and docs/plugins/reference/policy.md.

Verification:

  • node scripts/run-vitest.mjs extensions/policy/src/doctor/register.test.ts extensions/policy/src/cli.test.ts -- --reporter=dot in a detached PR worktree: 149 passed.
  • Live PR checks at this head show the relevant CI/check lanes green, including docs, lint, prod types, test types, dependency/boundary checks, and real behavior proof.

I did not run a runtime tool-execution E2E because this PR deliberately does not enforce runtime tool calls; the covered behavior is config conformance and attestation evidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation extensions: policy maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: XL status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants