docs(governance): document trust-based compliance antipattern#711
Conversation
Create PROTOCOL-ANTIPATTERNS.md documenting: - Trust-based compliance antipattern with evidence from PR #669 - Verification-based enforcement replacement pattern - Three case studies (branch verification, session init, test execution) - Design guidelines and implementation checklist Also adds links from SESSION-PROTOCOL.md and AGENT-INSTRUCTIONS.md to the new antipatterns document. Closes #686 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
PR Validation ReportTip ✅ Status: PASS Description Validation
QA Validation
Powered by PR Validation workflow |
AI Quality Gate ReviewTip ✅ Final Verdict: PASS WalkthroughThis PR was reviewed by six AI agents in parallel, analyzing different aspects of the changes:
Review Summary
💡 Quick Access: Click on individual agent jobs (e.g., "🔒 security Review", "🧪 qa Review") in the workflow run to see detailed findings and step summaries. Roadmap Review DetailsStrategic Alignment Assessment
Feature Completeness
Impact Analysis
Concerns
Recommendations
VerdictQA Review DetailsBased on the PR description and diff provided: VERDICT: PASS PR TYPE: DOCS FILES:
EVIDENCE:
Link Verification:
Content Quality:
DevOps Review DetailsBased on the PR description and diff provided, I can complete the DevOps review. PR Scope Detection
Classification: DOCS-only PR. No CI/CD, workflow, script, or infrastructure changes. Pipeline Impact Assessment
CI/CD Quality Checks
Findings
Template Assessment
Automation Opportunities
Documentation-only PR. The content documents protocol antipatterns which may inform future automation but requires no immediate DevOps changes. Recommendations
Analyst Review DetailsCode Quality Score
Overall: 5/5 Impact Assessment
Findings
Recommendations
VerdictSecurity Review DetailsSecurity Review: PR #711PR Type Detection
Classification: DOCS-only PR. All changed files are markdown documentation. Findings
Analysis
RecommendationsNone. This is a documentation-only PR that captures lessons learned about protocol design. VerdictArchitect Review DetailsDesign Quality Assessment
Overall Design Score: 5/5 Architectural Concerns
Breaking Change Assessment
Technical Debt Analysis
ADR Assessment
Recommendations
VerdictRun Details
Powered by AI Quality Gate workflow |
Session completed 4 PRs from priority issues: - PR #708: Issue #700 - ConvertFrom-Json error handling - PR #709: Issue #699 - GITHUB_OUTPUT error handling - PR #710: Issue #675 - Canonical source principle - PR #711: Issue #686 - Trust-based compliance antipattern 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Session Protocol Compliance ReportTip ✅ Overall Verdict: PASS All session protocol requirements satisfied. What is Session Protocol?Session logs document agent work sessions and must comply with RFC 2119 requirements:
See .agents/SESSION-PROTOCOL.md for full specification. Compliance Summary
Detailed Results2025-12-31-session-112-autonomous-developmentBased on the session log content provided in the context, I can now validate protocol compliance: Notes:
Run Details
Powered by AI Session Protocol Validator workflow |
✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
📝 WalkthroughWalkthroughThree documentation files updated to introduce a new governance document on protocol antipatterns, covering trust-based compliance failures versus verification-based enforcement, with cross-references added to existing related documents. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
.agents/governance/PROTOCOL-ANTIPATTERNS.md (1)
1-20: Minor: File naming convention consistency.The file is named
PROTOCOL-ANTIPATTERNS.md. Coding guidelines suggest governance documents follow patternPOLICY-NAME-UPPERCASE.mdor[ACRONYM]-[POLICY-AREA].md(e.g.,POLICY-ANTIPATTERNS.mdorPAP-DESIGN.md). However, related governance documents (SESSION-PROTOCOL.md, PROJECT-CONSTRAINTS.md) also deviate from this pattern, so this is a minor consistency note rather than a blocker.Consider standardizing governance file naming if this becomes a pattern.
📜 Review details
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
.agents/sessions/2025-12-31-session-112-autonomous-development.mdis excluded by!.agents/sessions/**
📒 Files selected for processing (3)
.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md.agents/governance/PROTOCOL-ANTIPATTERNS.md
🧰 Additional context used
📓 Path-based instructions (11)
.agents/**/*.{md,yml,yaml,json}
📄 CodeRabbit inference engine (.agents/critique/001-agent-templating-critique.md)
For agent platform files, evaluate whether near-identical variants (99%+ overlap) can be consolidated with conditional configuration rather than maintaining separate files
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md
.agents/**/*.md
📄 CodeRabbit inference engine (.agents/retrospective/pr43-coderabbit-root-cause-analysis.md)
.agents/**/*.md: Use PREFIX-NNN naming convention (e.g., EPIC-001, CRITIQUE-001) for sequenced artifacts and type-prefixed naming (e.g., prd-, tasks-) for non-sequenced artifacts
Normalize all file paths in markdown documents to be repository-relative before committing, removing absolute machine-specific paths
.agents/**/*.md: Session logs and documentation must include Phase checklist verification (Phase 1-3 protocol compliance including agent activation, instruction reading, handoff file updates, and session logging)
Session logs must document Session ID, date, agent name, and branch information in a standardized header formatAll artifact files in .agents/ must be in Markdown format
Document analysis recommendations with specific rationale when adding new governance documents like PROJECT-CONSTRAINTS.md
Maintain debugging skills documentation in
.agents/directoryDocument implementation notes explaining deviations from user prompts or decisions made during development (e.g., using plural form for directory names)
Run markdown lint on all generated artifacts before completing a session
Run markdown lint validation (0 errors expected) before committing documentation files in the .agents directory
.agents/**/*.md: Use consistent absolute file paths throughout task and PRD documentation instead of mixing relative and absolute path formats
Run markdown linting withnpx markdownlint-cli2 --fixon all agent-generated documentation before commitAll modifications to agent documentation and specifications must be marked with status updates (e.g., DRAFT → CONSOLIDATED) and include consolidation notes in headers
Configure GitHub MCP server in project MCP settings and create github-agent.md with agent-specific tool binding following the agent isolation pattern from superpowers-chrome
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md
**/.agents/**/*.md
📄 CodeRabbit inference engine (.agents/roadmap/epic-agent-consolidation.md)
Single-source agent files should use frontmatter markers to delineate platform-specific sections for VS Code and Copilot CLI variants
Maintain artifact synchronization markers in tracking files (.md) with status indicators ([COMPLETE], [RESOLVED], [VERIFIED]) and timestamps to document completion and verification of work
Separate domain knowledge from methodology - store domain expertise in knowledge documents, not in methodology/protocol files
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md
.agents/governance/**/*.md
📄 CodeRabbit inference engine (.agents/retrospective/phase3-p2-learnings.md)
Create canonical DRY reference documents in .agents/governance/ for multi-agent patterns instead of duplicating across individual agent files
Follow RFC 2119 compliance with MUST/SHOULD/MAY tiering for constraint documentation
All RFC 2119 keywords (MUST, SHOULD, MAY, etc.) in governance and protocol documents must follow RFC 2119 semantics as documented in SESSION-PROTOCOL.md
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md
.agents/governance/*.md
📄 CodeRabbit inference engine (.agents/planning/PHASE-PROMPTS.md)
.agents/governance/*.md: Document artifact cross-reference format in governance naming conventions, establishing traceability between requirements, designs, and tasks
Document parallel execution patterns in governance, assessing when tasks can run in parallel based on independence criteria (no shared file modifications, no data dependencies, no sequence requirements, no coordination needs)
Use aggregation strategies for parallel results: merge (non-conflicting outputs), vote (redundant execution), escalate (conflicts detected), first-wins (race conditions)
Create evaluation rubric with weighted criteria (Completeness, Correctness, Clarity, Actionability each 25%) scoring 1-4 per criterion, with 70% threshold for acceptance and maximum 3 iterations before escalationGovernance policy documents should follow naming convention:
POLICY-NAME-UPPERCASE.mdor[ACRONYM]-[POLICY-AREA].md(e.g., COST-GOVERNANCE.md, AI-REVIEW-MODEL-POLICY.md)
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md
**/.agents/**/**.md
📄 CodeRabbit inference engine (.agents/critique/001-pr365-remediation-critique.md)
Verify existence of referenced documentation files before updating them in automation procedures
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md
{**/.agents/**,**/*prompt*.{js,ts,md},**/*agent*.ps1}
📄 CodeRabbit inference engine (.agents/critique/465-spec-validation-false-positive.md)
Require explicit verdict patterns in all AI agent outputs rather than relying on substring keyword matching for verdict detection
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md
.agents/**
⚙️ CodeRabbit configuration file
Agent configuration files. Only flag security issues or broken cross-references. Ignore style, formatting, and structure.
Files:
.agents/governance/PROTOCOL-ANTIPATTERNS.md.agents/AGENT-INSTRUCTIONS.md.agents/SESSION-PROTOCOL.md
.agents/AGENT-INSTRUCTIONS.md
📄 CodeRabbit inference engine (.agents/SESSION-START-PROMPT.md)
Read
.agents/AGENT-INSTRUCTIONS.mdsecond - contains task execution protocolReference SESSION-PROTOCOL.md in AGENT-INSTRUCTIONS.md instead of duplicating Session Start/End checklist format requirements
Files:
.agents/AGENT-INSTRUCTIONS.md
.agents/SESSION-PROTOCOL.md
📄 CodeRabbit inference engine (.agents/retrospective/2025-12-18-parallel-implementation-retrospective.md)
Use verification-based BLOCKING gates in SESSION-PROTOCOL.md that require tool output rather than trust-based guidance
Establish SESSION-PROTOCOL integration for automated compliance and session automation features, building on existing SESSION-PROTOCOL.md rather than replacing it
SESSION-PROTOCOL.md Session End phase must be marked as BLOCKING with enforcement criteria and rejection conditions, not merely REQUIRED, to achieve compliance rates equivalent to Session Start (79% vs 4%)
SESSION-PROTOCOL.md MUST remain the canonical protocol definition, with Session State MCP tools reading from it for gate validation requirements
Update SESSION-PROTOCOL.md version to 1.4 documenting that agents MUST NOT update HANDOFF.md and must use session logs and Serena memory instead
Implement Phase 1.5 BLOCKING gate in session protocol requiring skill validation (Check-SkillExists.ps1) before starting work
SESSION-PROTOCOL.md must include an 'Unattended Execution Protocol' section with at least 8 MUST requirements for autonomous agent operation
SESSION-PROTOCOL.md must integrate lifecycle hooks with automated pre-session (context loading, Serena activation, HANDOFF reading) and post-session (HANDOFF update, markdownlint, skill extraction, commit) phases
Update SESSION-PROTOCOL.md to document that agents MAY skip QA validation when sessions are investigation-only by using the 'SKIPPED: investigation-only' evidence marker
Files:
.agents/SESSION-PROTOCOL.md
**/.agents/SESSION-PROTOCOL.md
📄 CodeRabbit inference engine (.agents/planning/PRD-agent-orchestration-mcp.md)
Session end protocol in SESSION-PROTOCOL.md MUST require agents to update HANDOFF.md with session summary, but Agent Orchestration MCP MUST aggregate parallel outputs and update HANDOFF.md once to prevent staging conflicts
Files:
.agents/SESSION-PROTOCOL.md
🔍 Remote MCP DeepWiki, GitHub Copilot
Perfect! I have gathered comprehensive context. Let me compile the findings.
Summary of Additional Context for PR #711 Review
Repository Context,
The PR belongs to rjmurillo/ai-agents, a multi-agent system for software development with a sophisticated governance structure. The repository maintains extensive governance documentation in the .agents/ directory including session protocols, agent instructions, and now protocol antipatterns.
PR Scope & Alignment with Objectives
PR #711 creates three files totaling 278 additions:
- New document:
.agents/governance/PROTOCOL-ANTIPATTERNS.md(210 lines) - Updates:
.agents/AGENT-INSTRUCTIONS.md(8 additions) - Updates:
.agents/SESSION-PROTOCOL.md(1 addition) - Session log:
.agents/sessions/2025-12-31-session-112-autonomous-development.md(59 additions)
All changes directly address the acceptance criteria from issue #686:
- ✅ PROTOCOL-ANTIPATTERNS.md created with evidence from PR #669
- ✅ Trust-based compliance antipattern documented
- ✅ Verification-based enforcement pattern documented
- ✅ 3 case studies included (branch verification, session init, test execution)
- ✅ Design guidelines and implementation checklist provided
- ✅ Links added to SESSION-PROTOCOL.md and AGENT-INSTRUCTIONS.md
Evidence & Root Cause Documentation,
PR #669 (merged 2025-12-31) provides the foundational evidence:
- Root Cause: Trust-based compliance for git operations
- Failure Evidence: 5 PRs affected by cross-branch commit contamination; agents made commits without branch verification despite explicit instructions in CLAUDE.md
- Failure Rate: 100% when relying on agent memory alone
Issue #684 (closed 2025-12-31) documents the replacement pattern:
- Implements mandatory branch verification gates at session start and pre-commit
- References the antipattern as verification-based enforcement
- Demonstrates the design checklist approach used in the PR's documentation
Governance Document Architecture
The repository uses verification-based enforcement (not trust-based compliance) for critical operations through:
- SESSION-PROTOCOL.md: Canonical source defining session start/end requirements with RFC 2119 keywords (MUST/SHOULD/MAY)
- AGENT-INSTRUCTIONS.md: Task execution protocol that references SESSION-PROTOCOL.md rather than duplicating it
- PROTOCOL-ANTIPATTERNS.md: New governance document establishing design patterns for future protocol creation
Documentation Quality Indicators
The new document includes:
- Concrete examples: Failure modes vs. success indicators with quantified success rates (0% vs. 90%+)
- Case studies: Three detailed examples showing trust-based failures and verification-based successes
- Design hierarchy: Verification hierarchy table mapping mechanisms to blocking levels
- Red flags section: Patterns that indicate vulnerable trust-based compliance
- Implementation checklist: Four concrete questions for designing new requirements
- RFC 2119 compliance: Uses standardized requirement language
Related Context
The PR includes Session 112 log documenting the autonomous development workflow:
🔇 Additional comments (3)
.agents/SESSION-PROTOCOL.md (1)
671-671: Use repository-relative paths instead of relative paths.The new link uses
./governance/PROTOCOL-ANTIPATTERNS.mdbut should be normalized to repository-relative format:.agents/governance/PROTOCOL-ANTIPATTERNS.md. This aligns with the coding guideline: "Normalize all file paths in markdown documents to be repository-relative before committing."Note: The existing links in this section also use inconsistent relative formats (
../AGENTS.md,./HANDOFF.md, etc.). Consider normalizing all paths in the Related Documents section during this commit..agents/AGENT-INSTRUCTIONS.md (1)
793-798: Normalize paths to repository-relative format.All three links in the new Related Documents section use relative paths instead of repository-relative paths. Update to:
-## Related Documents - -- [SESSION-PROTOCOL.md](./SESSION-PROTOCOL.md) - Session start/end requirements -- [PROTOCOL-ANTIPATTERNS.md](./governance/PROTOCOL-ANTIPATTERNS.md) - Protocol design antipatterns and replacement patterns -- [PROJECT-CONSTRAINTS.md](./governance/PROJECT-CONSTRAINTS.md) - Hard constraints +## Related Documents + +- [SESSION-PROTOCOL.md](.agents/SESSION-PROTOCOL.md) - Session start/end requirements +- [PROTOCOL-ANTIPATTERNS.md](.agents/governance/PROTOCOL-ANTIPATTERNS.md) - Protocol design antipatterns and replacement patterns +- [PROJECT-CONSTRAINTS.md](.agents/governance/PROJECT-CONSTRAINTS.md) - Hard constraintsPer coding guideline: "Normalize all file paths in markdown documents to be repository-relative before committing, removing absolute machine-specific paths."
.agents/governance/PROTOCOL-ANTIPATTERNS.md (1)
1-210: Strong documentation of antipatterns and replacement patterns.The document comprehensively covers trust-based compliance failure modes, verification-based enforcement principles, and provides detailed evidence from PR #669. All acceptance criteria from issue #686 are met: antipattern described with evidence, replacement pattern documented, three case studies included (branch verification, session initialization, test execution), design guidelines with implementation checklist provided, and red flags identified.
RFC 2119 semantics used correctly throughout. Evidence is concrete and quantified (0% failure with trust-based, 90%+ success with verification-based).
Verify markdown linting has been run per coding guideline requirement:
npx markdownlint-cli2 --fix ".agents/governance/PROTOCOL-ANTIPATTERNS.md"produces zero errors.
Summary
Document the trust-based compliance antipattern in architecture governance to prevent future protocol design errors. This captures lessons learned from PR #669 (branch verification failures).
Specification References
Changes
.agents/governance/PROTOCOL-ANTIPATTERNS.mdwith:Type of Change
Testing
Agent Review
Security Review
Other Agent Reviews
Checklist
Related Issues