Skip to content

Add tool-policy diagnostics and conformance reporting #2136

@vaddisrinivas

Description

@vaddisrinivas

Problem

When a runtime/profile hides tools or installs policy, operators need proof of what is visible, what is blocked, and whether any raw write-capable tool bypass remains. Today this requires reading logs and reconstructing tool state manually.

Generic use case

Users and maintainers can inspect a profile/runtime and see the active tool inventory, policy mode, hidden raw tools, MCP allowlists, audit health, and recent denial reasons.

Managed-runtime use case

A compiled runtime contract can show conformance: source hash, policy hash, generated capability tools, hidden raw write tools, adapter map, and audit/approval health.

Proposed shape

  • Add core diagnostics RPC that returns active policy/tool visibility state.
  • Add conformance checks for raw write-capable tools, MCP allowlists, and audit availability.
  • Add lightweight UI surface under settings/developer or diagnostics.
  • Keep output redacted and support copyable support bundle.

Acceptance criteria

  • RPC reports visible tools with permission/category metadata.
  • RPC reports policy mode and recent policy denials.
  • Tests cover conformance pass/fail for hidden vs exposed write tools.
  • UI renders diagnostics without exposing secrets.

Alignment

This is generic observability for OpenHuman policy work. It helps contract-driven runtimes, Composio debugging, MCP debugging, and security reviews.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions