Summary

The pr-comment-responder agent does not properly handle Copilot's unique response pattern. When replying to @Copilot review comments, Copilot responds differently than human reviewers:

1. Creates a separate follow-up PR branched from the current PR
2. Posts an issue comment on the original PR (not a review comment reply)
3. Links to the follow-up PR in that issue comment

This causes orphaned conversations and unnecessary follow-up PRs cluttering the repository.

Problem Observed

From PR #57:

• Copilot posted review comments about "missing documentation files"
• We replied explaining the files exist (false positives)
• Copilot created PRs feat(installer): add command and prompt file installation support #58, feat: MCP config sync, session protocol enforcement, and platform prioritization #59, feat: AI-powered GitHub Actions workflows with security-hardened PowerShell implementation #60 as follow-ups
• These follow-up PRs had to be manually closed
• The original review threads were not automatically resolved

Current Behavior

The pr-comment-responder agent:

• Γ£à Retrieves review comments via gh api repos/{owner}/{repo}/pulls/{pr}/comments
• Γ£à Replies to comments via the replies endpoint
• Γ¥î Does NOT check issue comments for Copilot's follow-up responses
• Γ¥î Does NOT identify or close Copilot's generated follow-up PRs

Desired Behavior

1. Detect Copilot Response: After replying to @Copilot, poll issue comments for Copilot's response
2. Parse Response Intent: Determine if action is needed or if our explanation was accepted
3. Manage Follow-up PRs: If a follow-up PR is created and no action is required, close it with explanation
4. Report Results: Include "Copilot Interactions" table in the agent's output summary

User Stories

Functional Requirements

• FR-1: Retrieve issue comments via gh api repos/{owner}/{repo}/issues/{pr}/comments
• FR-2: Detect Copilot responses by author and timestamp
• FR-3: Extract follow-up PR numbers from Copilot's comment body
• FR-4: Close follow-up PRs with explanatory comment referencing original thread
• FR-5: Categorize replies (no action / acknowledged / needs discussion)
• FR-6: Implement polling strategy (60s timeout, 5s interval)
• FR-7: Ensure idempotency (do not close same PR twice)

Proposed Workflow Addition

Add Phase 4: Copilot Follow-up Handling after the current response phase:

[Is comment from @Copilot?] --No--> End
    |
   Yes
    v
Poll for Copilot's issue comment response
    |
    v
[Response found?] --No (timeout)--> Log warning, End
    |
   Yes
    v
Parse for follow-up PR reference
    |
    v
[Follow-up PR exists?] --No--> End (conversation complete)
    |
   Yes
    v
[Our reply = "no action"?] --No--> Log: manual review needed
    |
   Yes
    v
Close follow-up PR with explanation --> End

Open Questions

1. Should we auto-merge valid follow-up PRs? (Recommend: No - too risky)
2. What if Copilot creates multiple follow-up PRs?
3. Can we programmatically resolve the original review thread?
4. Should we skip closing PRs that already have reviews?
5. What is Copilot's exact author identifier? (copilot, github-copilot[bot], or github-actions[bot]?)

Related

• PRD: .agents/planning/PRD-copilot-response-handling.md
• Example: PR Claude skill installation support #57 ΓåÆ Follow-up PRs feat(installer): add command and prompt file installation support #58, feat: MCP config sync, session protocol enforcement, and platform prioritization #59, feat: AI-powered GitHub Actions workflows with security-hardened PowerShell implementation #60

🤖 Generated with Claude Code

Migrated from rjmurillo/vs-code-agents#65