Agent Persona Exploration - 2026-04-10 #25574
Closed
Replies: 1 comment
-
|
This discussion has been marked as outdated by Agent Persona Explorer. A newer discussion is available at Discussion #25749. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Exploration of the agentic-workflows custom agent behavior across 7 representative automation scenarios drawn from 5 software worker personas. The agent was simulated using
general-purposeagents loaded with gh-aw conventions (theagentic-workflowstype is not directly invokable as a task agent).Persona Overview
Key Findings
safe-outputs. The pattern appears deeply embedded in the agent's behavior.weekly on monday around 09:00with auto-addedworkflow_dispatch, following gh-aw best practices exactly.bash: ["command:*"]allowlists rather thanbash: true, which is the security-first pattern.noopalways included — every workflow included anoopsafe-output path, preventing the most common workflow failure mode.concurrencyblock, which is a production-readiness gap for rapid-push scenarios.Top Patterns
pull_request: types: [opened, synchronize, reopened]withpaths:filters — used in 3/7 scenarioscache-memoryfor state persistencestrict: truein 3/7 explicitly;network.allowedscoped togo,node,github,api.osv.devas appropriateclose-older-issues: true/hide-older-comments: trueused in recurring reports to prevent accumulation<details>/<summary>blocks in all report templates for scannable issue/comment outputView Scenario Scores (all 7)
View High Quality Responses (Score ≥ 24)
BE1 — DB Migration Safety Review (25/25)
Perfect score. Standout behaviors: used
submit-pull-request-review(not justadd-comment) enabling branch-protection integration; dual data sources (git diff + GitHub MCP) for cross-verification; migration-framework sniffing before applying rollback checks; three-way review verdict (APPROVE / COMMENT / REQUEST_CHANGES).DevOps1 — Deployment Incident Responder (24/25)
Pre-fetch step materialized log error signals before the agent started, reducing MCP round-trips. Correct use of
workflow_runtrigger scoped to named deploy workflows. Belt-and-suspenders dedup: bothskip-if-matchin frontmatter AND agent-side SHA-level deduplication search.BE2 — Weekly CVE Scanner (24/25)
Dual-source enrichment (GHSA + OSV) with explicit CVSS threshold (
≥9.0 = CRITICAL,7.0–8.9 = HIGH). Per-CVE deduplication via issue search before eachcreate-issuecall — correctly identified thatskip-if-matchhandles workflow-level gating but can't do per-finding dedup.View Areas for Improvement
Missing
concurrencyblocks in PR workflows (FE1, QA1)When multiple pushes arrive rapidly on the same PR, visual regression (FE1) and coverage (QA1) workflows can race on the same
cache-memorykey. The recommended fix is:This is documented in
create-agentic-workflow.mdexamples but not applied consistently.Python/bash variable handoff in QA1
The coverage tracker prompt mixed shell variable interpolation (
$OVERALL_PCT) inside Python heredocs. The pattern works but is ambiguous — the agent must correctly infer the bash→Python data boundary. A clearer pattern would write shell values to a temp file, thenjson.load()from Python, avoiding interpolation entirely.GitHub MCP
list_pull_requestslabel filtering caveat (PM1)The weekly digest workflow assumes server-side label filtering in the MCP tool, which may not be supported. A production run may silently return all PRs and require client-side filtering. This is a gh-aw MCP documentation gap.
Recommendations
Add
concurrencyblock template to.github/aw/create-agentic-workflow.md— Include a standard example for PR-triggered workflows showinggroup: "workflow-$\{\{ github.ref }}" / cancel-in-progress: true. This single addition would prevent a common production-readiness gap across all PR automation scenarios.Document MCP tool filtering limitations in
.github/aw/github-mcp-server.md— Explicitly note thatlist_pull_requestsand similar list tools may not support server-side label/date filtering; instruct agents to always fetch and client-side filter. Prevents silent data gaps in PM and reporting workflows.Add Python/bash data handoff pattern to
.github/aw/github-agentic-workflows.md— Include a canonical pattern for shell→Python data passing (write to temp JSON file,json.load()in Python) to replace ad-hoc variable interpolation in multi-language analysis workflows.References:
/tmp/gh-aw/cache-memory/agent-persona-explorer-2026-04-10.jsonBeta Was this translation helpful? Give feedback.
All reactions