Agent Capability Gaps: Comparison with awesome-copilot

Summary

Analyzed 127 agents from github/awesome-copilot and identified critical gaps in our agent catalog (18 agents). Three MUST HAVE gaps limit our effectiveness: strategic planning, systematic debugging, and TDD enforcement.

Analysis Document: .agents/analysis/003-awesome-copilot-gap-analysis.md

Context

Our Agent Catalog (18 agents)

• analyst, architect, critic, devops, explainer, high-level-advisor
• implementer, independent-thinker, memory, orchestrator, planner
• pr-comment-responder, qa, retrospective, roadmap, security
• skillbook, task-generator

Awesome-Copilot Agent Count: 127

Categories: Language experts (40), Platform specialists (25), Workflow agents (20), Domain specialists (15), Tool integrations (10), Thinking modes (8), Database experts (7)

Gap Analysis

Critical Gaps Identified (MUST HAVE)

Gap ID	Missing Agent	Impact	Effort
GAP-001	Strategic Planner	Jump to implementation without thorough planning	M (8-12 hrs)
GAP-002	Debugger	No systematic 4-phase debugging workflow	M (6-8 hrs)
GAP-003	TDD Mode (QA enhancement)	Test-after instead of test-first development	L (3-4 hrs)

Important Improvements (SHOULD HAVE)

Gap ID	Missing Agent	Impact	Effort
GAP-004	ADR Auto-Generator	Manual ADR creation is slower	S (2-3 hrs)
GAP-005	Issue Refiner	GitHub issues lack clarity	S (2-3 hrs)
GAP-006	Tech Debt Planner	Tech debt addressed ad-hoc	M (6-8 hrs)
GAP-007	Accessibility Agent	No WCAG compliance review	M (6-8 hrs)
GAP-008	Code Tour Generator	Manual onboarding docs	S (3-4 hrs)

Optional Enhancements (NICE TO HAVE)

Gap ID	Missing Agent	Impact	Effort
GAP-009	Thinking Transparency Modes	Enhanced reasoning visibility	XS (1-2 hrs)
GAP-010	Agent Foundry	Meta-agent for creating agents	S (2-3 hrs)
GAP-011	Blueprint Mode	Architecture visualization	M (6-8 hrs)

Detailed Gap Analysis

GAP-001: Strategic Planner Agent (P0 - MUST HAVE)

Problem: We lack pre-implementation strategic planning. Current workflow jumps from analyst research to implementer code without thorough codebase analysis and impact assessment.

Awesome-Copilot Equivalent: plan.agent.md

Capabilities Needed:

• Analyze codebase before proposing solutions
• Clarify requirements through structured questions (3-5 upfront)
• Assess impact on existing components
• Develop step-by-step implementation roadmap
• Present options with trade-offs

Proposed Solution:

• Create src/claude/strategic-planner.md agent
• Orchestrator routes complex features to strategic-planner before task-generator
• Outputs to .agents/planning/

Evidence from awesome-copilot:

• plan.agent.md: "Think through complex coding challenges before implementation"
• implementation-plan.agent.md: "Deterministic, executable implementation plans"
• Pattern: Research → Plan → Implement (we skip Plan step)

Estimated Effort: 8-12 hours (1-2 sessions)

---

GAP-002: Debugger Agent (P0 - MUST HAVE)

Problem: No systematic debugging workflow. Implementer handles bugs ad-hoc without structured methodology.

Awesome-Copilot Equivalent: debug.agent.md

Capabilities Needed:

• Phase 1 - Problem Assessment: Reproduce bug, gather context, generate report
• Phase 2 - Investigation: Trace execution paths, identify root cause
• Phase 3 - Fix Implementation: Targeted, minimal changes
• Phase 4 - Verification: Test execution, regression checks, edge cases

Proposed Solution:

• Create src/claude/debugger.md agent
• Orchestrator routes bug reports to debugger (not implementer)
• Outputs debug report to .agents/analysis/ + fix implementation

Evidence from awesome-copilot:

• debug.agent.md: "Systematic four-phase debugging methodology"
• Structured approach: Assess → Investigate → Fix → Verify
• Emphasis on root cause identification before fixing

Estimated Effort: 6-8 hours (1 session)

---

GAP-003: TDD Mode for QA Agent (P0 - MUST HAVE)

Problem: QA agent handles testing but doesn't systematically enforce test-first development. We advocate TDD but don't enforce it.

Awesome-Copilot Equivalent: tdd-red.agent.md, tdd-green.agent.md, tdd-refactor.agent.md

Capabilities Needed:

• Red Phase: Write failing test first (acceptance criteria → test)
• Green Phase: Implement minimal code to pass test
• Refactor Phase: Improve code while keeping tests green
• Enforce test-first workflow for features (not just bug fixes)

Proposed Solution:

• Enhance existing src/claude/qa.md with TDD mode
• QA agent detects feature requests → triggers TDD workflow
• Document cycle in .agents/qa/

Evidence from awesome-copilot:

• Separate agents for each TDD phase (red/green/refactor)
• Enforces test-first instead of test-after
• Systematic quality gate we currently lack

Estimated Effort: 3-4 hours (1 session)

---

GAP-004: ADR Auto-Generator (P1 - SHOULD HAVE)

Problem: ADR creation is manual. Architect agent supports ADRs but doesn't automate sequential numbering, validation, or structured formatting.

Awesome-Copilot Equivalent: adr-generator.agent.md

Capabilities Needed:

• Sequential numbering (examine /docs/adr/, auto-increment)
• Structured documentation (context, decision, consequences, alternatives)
• 14-point validation checklist
• Systematic codes (POS-001, NEG-001, ALT-001)

Proposed Solution:

• Enhance src/claude/architect.md with ADR auto-generation mode
• Automate file naming, numbering, validation

Estimated Effort: 2-3 hours

---

GAP-005: Issue Refiner (P1 - SHOULD HAVE)

Problem: GitHub issues may be incomplete or unclear. No systematic refinement before triage.

Awesome-Copilot Equivalent: refine-issue.agent.md

Capabilities Needed:

• Detect incomplete issues (missing repro steps, acceptance criteria)
• Ask clarifying questions
• Update issue with refined information

Proposed Solution:

• Create src/claude/issue-refiner.md agent
• Runs before analyst triage

Estimated Effort: 2-3 hours

---

GAP-006: Tech Debt Planner (P1 - SHOULD HAVE)

Problem: Tech debt remediation is ad-hoc. No dedicated planning workflow.

Awesome-Copilot Equivalent: tech-debt-remediation-plan.agent.md

Capabilities Needed:

• Identify tech debt sources
• Assess impact and urgency
• Create remediation roadmap
• Prioritize work packages

Proposed Solution:

• Create src/claude/tech-debt-planner.md agent
• Complements roadmap agent

Estimated Effort: 6-8 hours

---

GAP-007: Accessibility Agent (P1 - SHOULD HAVE)

Problem: No systematic WCAG compliance review. Accessibility is not part of standard workflows.

Awesome-Copilot Equivalent: accessibility.agent.md

Capabilities Needed:

• WCAG 2.1 Level AA compliance checks
• Semantic HTML validation
• ARIA attribute review
• Keyboard navigation testing
• Screen reader compatibility

Proposed Solution:

• Create src/claude/accessibility.md agent
• Integrate with qa workflow

Estimated Effort: 6-8 hours

---

GAP-008: Code Tour Generator (P1 - SHOULD HAVE)

Problem: Onboarding documentation is manual. No automated codebase walkthroughs.

Awesome-Copilot Equivalent: code-tour.agent.md

Capabilities Needed:

• Generate interactive code tours
• Explain architecture patterns
• Highlight key files and their purposes
• Create learning paths

Proposed Solution:

• Create src/claude/code-tour-generator.md agent
• Outputs to .agents/onboarding/

Estimated Effort: 3-4 hours

Recommendations

P0 - Implement Within 2 Weeks

1. Create strategic-planner agent (8-12 hrs)

   • Fills critical gap between analyst and implementer
   • Prevents premature implementation
   • Orchestrator routes complex features here first

2. Create debugger agent (6-8 hrs)

   • Systematic 4-phase debugging workflow
   • Improves bug resolution speed and quality
   • Orchestrator routes bugs here (not implementer)

3. Enhance qa agent with TDD mode (3-4 hrs)

   • Enforce test-first development
   • Add red/green/refactor cycle
   • QA detects feature → triggers TDD

Total P0 Effort: 17-24 hours (2-3 sessions)

P1 - Implement Within 1 Month

4. Enhance architect with ADR auto-generation (2-3 hrs)
5. Create issue-refiner agent (2-3 hrs)
6. Create accessibility agent (6-8 hrs)
7. Create tech-debt-planner agent (6-8 hrs)

Total P1 Effort: 16-22 hours (2-3 sessions)

P2 - Backlog

8. Create code-tour-generator agent (3-4 hrs)
9. Add thinking transparency modes (1-2 hrs)
10. Create agent-foundry tool (2-3 hrs)

Our Strengths (Not Found in awesome-copilot)

We have capabilities awesome-copilot lacks:

1. Memory/Skillbook System: Cross-session learning persistence
2. Retrospective Learning: Systematic extraction of lessons learned
3. Multi-Agent Orchestration: Sophisticated routing and coordination
4. Strategic Advisors: high-level-advisor, independent-thinker
5. Multi-Platform Support: Claude Code, VS Code, Copilot CLI

Success Metrics

After implementing P0 recommendations:

Metric	Before	Target
Features with strategic plan	0%	80%
Bugs with debug report	0%	90%
Features with test-first TDD	20%	70%
Implementation rework due to poor planning	High	Low

Next Steps

1. User approval: Review recommendations, approve P0 implementation
2. Implement GAP-001: Create strategic-planner agent
3. Implement GAP-002: Create debugger agent
4. Implement GAP-003: Enhance qa agent with TDD mode
5. Test workflow: Route complex feature through new agents
6. Retrospective: Extract learnings, adjust as needed

References

• Analysis Document: .agents/analysis/003-awesome-copilot-gap-analysis.md
• Awesome-Copilot Repository: https://github.com/github/awesome-copilot
• Sample Agents Analyzed:
  • plan.agent.md
  • debug.agent.md
  • tdd-red.agent.md
  • implementation-plan.agent.md
  • adr-generator.agent.md