Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

@greptile review

Greptile Summary

This PR fixes bench runs incorrectly calling live AWS/EKS and Hermes endpoints by adding a _filter_tools hook to ConnectedInvestigationAgent and overriding it in BenchInvestigationAgent to allowlist only tools whose origin_module starts with the bench-package prefix — so production tools never reach the bench LLM loop.

• app/agent/investigation.py: adds _filter_tools(tools) → tools (identity default) wired between _get_available_tools and _build_connected_tool_context, ensuring the filtered set also drives tool_schemas, seed calls, and state fields — zero production behavior change.
• tests/benchmarks/cloudopsbench/bench_agent.py: overrides _filter_tools with an origin_module-prefix allowlist (ALLOWED_TOOL_MODULE_PREFIXES ClassVar), handles the empty-origin_module edge case with a WARNING log, and documents the trailing-dot exclusion in comments and tests.
• app/tools/registry.py: removes bench-specific examples from docstrings to keep production code neutral about consumers.

Confidence Score: 5/5

Safe to merge — the hook is an identity no-op on the base class, so production investigations are unaffected; the bench subclass now consistently restricts the tool set to replay-only tools throughout the entire agent run.

The base-class _filter_tools returns its input unchanged, so every existing production code path continues without modification. The bench override is self-contained in the test tree, well-tested with five new unit tests, and applies uniformly to tool schemas, seed calls, and all parallel execution. No migration or schema changes are involved.

No files require special attention.

Important Files Changed

Sequence Diagram

sequenceDiagram
    participant Run as run()
    participant GAT as _get_available_tools()
    participant FT as _filter_tools()
    participant BCT as _build_connected_tool_context()
    participant LLM as LLM loop

    Run->>GAT: resolved_integrations
    GAT-->>Run: all available tools (prod + bench)
    Run->>FT: all available tools
    note over FT: Base class: identity (production)<br/>BenchAgent: keep origin_module<br/>startswith(ALLOWED_TOOL_MODULE_PREFIXES)
    FT-->>Run: filtered tools
    Run->>BCT: filtered tools
    BCT-->>Run: available_sources, available_action_names
    Run->>LLM: tool_schemas(filtered tools), seed_calls(filtered tools), _run_parallel(filtered tools)

_{Reviews (3): Last reviewed commit: "fixed greptile notes" | Re-trigger Greptile}

@greptile review

🐸 Rebase? Handled. Conflicts? Squashed. CI? Vibing. @YauhenBichel touched the untouchable and lived. 🫡

👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome.