feat: enhance pr-comment-responder with memory patterns and agent routing#23
Conversation
…attern Fixes #2 Add Phase 4: Copilot Follow-up Handling to address Copilot's unique response pattern where it creates follow-up PRs instead of replying inline to review threads. Changes: - Add Phase 4 workflow with polling for Copilot's issue comment responses - Add follow-up PR detection and closure logic for false positives - Add reply intent categorization (no action/acknowledged/needs discussion) - Add Copilot Interactions table to output format - Add Copilot-specific guidelines, quality checks, and edge cases - Implement idempotency checks (PR state, existing reviews) - Add polling strategy (60s timeout, 5s interval, 12 attempts) User Stories Implemented: - US-1: Detect Copilot response in issue comments - US-2: Identify follow-up PRs created by Copilot - US-3: Close unnecessary follow-up PRs - US-4: Report Copilot interaction results Functional Requirements Implemented: - FR-1 through FR-7 as specified in PRD 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes #2 Transform pr-comment-responder from a monolithic workflow to a thin coordination layer that delegates to specialized agents, following the orchestrator pattern. Key changes: - Add Phase 2 Comment Triage with agent routing table - Add Phase 3 Agent Delegation with example prompts for: - analyst: technical merit evaluation - critic: assumption validation - high-level-advisor: strategic decisions - implementer: code fixes - qa: regression verification - Add Phase 4 Response Synthesis based on agent feedback - Move bot-specific handling to Phase 5 with memory retrieval - Add Agent Workflow table to output format - Add Handoff Protocol table matching orchestrator style - Remove embedded evaluation logic (delegate to agents) - Maintain GitHub API interaction knowledge - Update all three platforms (Claude, Copilot CLI, VS Code) Benefits: - Reuses existing agent expertise (analyst, critic, etc.) - Reduces duplication of evaluation logic - Follows orchestrator workflow pattern - Token-efficient via delegation rather than embedded logic - Easier to maintain and extend 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes #2 Transform pr-comment-responder from duplicating workflow logic to a thin triage layer that delegates to orchestrator with path recommendations. Key changes: - Remove duplicated workflow execution (orchestrator handles this) - Add clear triage decision tree matching standard paths: - Quick Fix: implementer → qa (token-efficient) - Standard: analyst → architect → planner → critic → implementer → qa - Strategic: independent-thinker → high-level-advisor → task-generator - Quick fixes go directly to implementer (skip orchestrator overhead) - Standard/Strategic paths delegate to orchestrator with: - Classification and rationale - Code context and initial assessment - Workflow path recommendation - Retain bot-specific handling (Copilot follow-up, CodeRabbit commands) - Consistent across all three platforms Triage signals: - Quick Fix: "Can explain fix in one sentence" - Standard: "Need to investigate first" - Strategic: "Question is whether, not how" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Apply lessons from Weinberg, Schrag, and Coding Horror articles to optimize orchestrator resource usage across all three platforms (claude, copilot-cli, vs-code-agents). Key improvements: - Add Phase 0: Triage with exit early criteria before orchestrating - Add Complexity Assessment (Trivial/Simple/Standard/Complex) to right-size agent chains - Add Value Checkpoint with Schrag's Principle after Phase 2 - Add Weinberg's Law of the Hammer quote to prevent over-orchestration - Add Pattern Applied section to output format (Weinberg's Consulting Secret) - Fix malformed markdown code fences in vs-code-agents version Philosophy: "The cheapest orchestration is the one that doesn't happen." Not every task needs every agent. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… clarity Address underutilization of key agents and strengthen memory as a triage asset. ## pr-comment-responder enhancements ### Memory as a Strength - Add comprehensive memory protocol for bot false positive patterns - Store reviewer preferences (bots AND humans have patterns) - Track triage decisions (comment → path → outcome) for accuracy improvement - Store successful rebuttals for confidence in declining suggestions - Query domain-specific patterns for faster routing ### Domain-Specific Routing (skip orchestrator when appropriate) - `.github/workflows/*` → devops (CI/CD domain expertise) - `.githooks/*` → devops + security (infrastructure + security) - `**/Auth/**`, `*.env*` → security (critical path) - "WHETHER to do X" questions → independent-thinker (challenge assumptions) ### Independent-thinker Utilization - Route "WHETHER" questions to independent-thinker BEFORE strategic path - Provides unfiltered analysis of reviewer concerns - Intellectually honest evaluation of tradeoffs ## Orchestrator enhancements ### Planner vs Task-Generator Clarity - planner: Epic/Feature → Milestones (high-level breakdown) - task-generator: PRD/Milestone → Atomic tasks (implementation-ready) - Workflow: roadmap → planner → task-generator → implementer/qa/devops ### Routing Heuristics Updates - "Challenge assumptions" now routes to independent-thinker (not critic) - "Plan validation" explicitly routes to critic - Clear distinction between Epic→Milestones and Milestones→Tasks All three platforms (claude, copilot-cli, vs-code-agents) synced. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR transforms the pr-comment-responder agent from a direct implementation agent into a thin triage layer that classifies PR comments and delegates to appropriate agents. It also enhances the orchestrator with upfront triage to avoid unnecessary orchestration overhead, clarifies the distinction between planner and task-generator agents, and adds comprehensive memory protocols for learning from bot patterns and reviewer preferences.
Key changes:
- Repositioned pr-comment-responder as a triage specialist with three workflow paths (Quick Fix, Standard, Strategic)
- Added Phase 0 Triage to orchestrator with Weinberg/Schrag principles to prevent over-orchestration
- Introduced domain-specific routing (devops for CI/CD, security for auth files, independent-thinker for "whether" questions)
- Clarified planner vs task-generator roles across all platforms
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| vs-code-agents/pr-comment-responder.agent.md | Converted to triage specialist with workflow path classification, memory protocol, and bot-specific handling patterns |
| vs-code-agents/orchestrator.agent.md | Added Phase 0 Triage, complexity assessment, planner vs task-generator distinction, and Weinberg/Schrag principles |
| copilot-cli/pr-comment-responder.agent.md | Same triage specialist changes as VS Code version, adapted for copilot CLI syntax |
| copilot-cli/orchestrator.agent.md | Same orchestrator enhancements as VS Code version, adapted for copilot CLI syntax |
| claude/pr-comment-responder.md | Same triage specialist changes as VS Code version, adapted for Claude Task() API |
| claude/orchestrator.md | Same orchestrator enhancements as VS Code version, adapted for Claude Task() API |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Caution Review failedThe pull request is closed. 📝 WalkthroughWalkthroughRestructures multi-agent orchestration and PR comment handling documentation across 5 files to implement a triage-first model with mandatory phased workflows, complexity assessment, and explicit delegation rules instead of direct agent responses. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related issues
Possibly related PRs
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (6)
Comment |
|
Caution Review failedThe pull request is closed. 📝 WalkthroughWalkthroughDocumentation update across six agent guide files introducing orchestration phases (Triage, Value Checkpoint, Complexity Assessment), enhanced routing heuristics, mandatory agent rules, Phase 4 bot-specific handling with Copilot follow-up support, memory protocols, conflict resolution, and standardized output formats. Changes
Sequence Diagram(s)No sequence diagrams generated. Changes are documentation-only updates describing workflow frameworks without introducing new functional control flows requiring visualization. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20–30 minutes
Possibly related issues
Possibly related PRs
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (6)
Comment |
Pull Request
Summary
Enhance pr-comment-responder to leverage persistent memory for bot patterns, add domain-specific agent routing, and clarify planner vs task-generator roles across all three platforms.
Changes
Type of Change
Testing
Agent Review
Security Review
.agents/security/)Files requiring security review:
None - agent documentation only.
Other Agent Reviews
Checklist
Related Issues
Fixes #2
🤖 Generated with Claude Code