## Background As identified in the [Relationally Intelligent strategic analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/520502), users struggle to make Duo's outputs consistently align with their personal preferences and their team's established practices. This forces them to repeatedly correct or modify Duo's responses to match their needs. This problem directly impacts one of our core Trust Pillars from our [Making Duo a Trusted Collaborator](https://gitlab.com/gitlab-org/gitlab/-/issues/523421) strategy: developers want AI to recognize and follow their conventions so they can focus on solving unique problems rather than enforcing standards. While our long-term vision is for Duo to become relationally intelligent automatically, this epic focuses on enabling an explicit approach where users can tell Duo how to align its output with their preferences and practices. ## Problem Statement Users report two key challenges: 1. **Team standards**: Teams struggle to ensure consistent adherence to coding standards and best practices across members when using AI assistance 2. **Individual preferences**: Developers need to repeatedly correct or rephrase Duo outputs to match their personal coding style and current project needs Both challenges result in wasted time, diminished productivity, and reduced trust in AI-assisted development. ## Desired Outcomes This initiative will be successful when: 1. **Reduced rework**: Developers spend less time modifying Duo-generated code to match their preferences and team standards 2. **Developer control with sensible defaults**: Developers have control over which rules are applied, with project/group rules applied by default but with the flexibility to disable or override them for specific tasks 3. **Increased efficiency**: Teams can define rules once and have them automatically applied across multiple interactions 4. **Enhanced trust**: Developers gain confidence that Duo understands and respects their preferences and team conventions 5. **Cross-application and cross-client consistency**: Custom rules at all levels (user, project, group) are consistently applied across all Duo features (Code Suggestions, Chat, Code Review) and all supported clients (Web UI, Web IDE, VS Code, JetBrains, Visual Studio, Neovim), with the specific understanding that Code Review should only use project/group level rules (not user-level preferences) 6. **Full deployment coverage**: The solution must work across all deployment options: GitLab.com, Dedicated, and Self-Managed (including self-hosted models) ## Established Direction Based on customer feedback and internal discussions, we've established the following direction: **Communication-over-enforcement approach**: - Custom rules primarily serve to make developers more productive by reducing rework - Developers should have control over which rules are applied because they know their current goals best - Project/group rules should be applied by default, with developers having flexibility to turn them off or switch to other rules - AI-based enforcement is impractical; the CI/CD pipeline remains the proper place for strict enforcement **Full scope**: This initiative encompasses both: 1. **User-level rules**: Enabling individual developers to customize Duo's behavior according to personal preferences 2. **Project/group-level rules**: Allowing teams to define standards at project and group levels ## Approach: Implementation Phases While we have a clear vision of the full scope, we recommend a phased implementation approach to validate assumptions and deliver value quickly. ### Phase 1: Project/Group Rules The team should implement project/group level rules across all Duo features: - Focus initially on Code Review where teams urgently need support for consistent code standards - Extend implementation to Chat and Code Suggestions to ensure consistent application - Build robust inheritance and conflict resolution systems between group and project levels - Provide clear visibility into which rules are being applied - Implement appropriate rule validation and size limits ### Phase 2: Enhance with User-Level Customization After establishing project/group rules infrastructure: - Add user-level preference capabilities for Code Suggestions and Chat - Enable users to override or supplement project/group rules where appropriate - Focus on personal productivity improvements - Maintain primacy of project/group rules for Code Review feature - Build on existing inheritance system to handle user-level exceptions ### Key Questions to Validate 1. **Cross-feature consistency**: Can rules be effectively applied across different Duo features while maintaining appropriate context? 2. **Technical feasibility**: What are the performance and quality implications of including custom rules in model prompts? 3. **Scope and boundaries**: What are appropriate limitations for rule length and complexity? 4. **Rule conflicts**: How should conflicts between project and group-level rules be handled? How will user-level rules interact with team standards? 5. **Adoption drivers**: Do teams actively use and maintain project/group rules? What improvements would increase adoption? 6. **How to prevent** [**specific security** concern](https://www.pillar.security/blog/new-vulnerability-in-github-copilot-and-cursor-how-hackers-can-weaponize-code-agents) ### Learning Goals Through this implementation, we aim to learn: 1. What types of rules teams find most valuable for maintaining standards (e.g., architectural patterns, security requirements, library preferences) 2. The most effective interface for teams to define and manage rules 3. The optimal scope, granularity, and format for rules to be effective across different features 4. How to balance team standards with individual productivity needs 5. The impact of rules on response quality, latency, and user satisfaction ## Success Metrics We'll measure success through: **Phase 1 Metrics** - % of eligible projects/groups with at least one custom rule - Rule application rate across different features (Code Review, Chat, Code Suggestions) - User satisfaction with project/group custom rules - Reduction in style-related code review comments **Phase 2 Metrics (Additional)** - % of eligible users who create at least one custom rule - User satisfaction with user-level customization - Time saved through reduced modifications (survey) ## Tiering Decision **Supported Duo add-ons** - Duo Pro - Duo Enterprise **Supported deployment options** - GitLab.com - Dedicated - Self Managed - Self-hosted models ## Future Considerations After implementing both project/group-level and user-level rules, we may consider: - Supporting rules based on external sources (like existing style guides) - More sophisticated rule management and inheritance - Moving toward an intelligent system that learns preferences implicitly - Advanced analytics on rule effectiveness and impact These directions should be guided by the learnings from our initial implementations. ## Additional Context for Implementation The following information may be helpful for implementation but is not prescriptive: ### Rule Specificity and Targeting There are several approaches to consider for rule targeting: - **Path-based application**: Rules could be configured to apply only to specific file paths or directories - **File type specificity**: Rules could target specific file extensions (e.g., *.go, *.js, *.py) - **Pattern matching**: Consider supporting glob patterns for more flexible targeting - **Feature-specific rules**: Enable different rules for different Duo features while maintaining consistency where appropriate This approach allows for more granular rule application based on the specific context, which internal discussions have identified as a common pattern in developer tooling. This would enable teams to have different rules for different parts of their codebase (e.g., frontend vs. backend rules). ### Token Size Considerations **Example style guide sizes for context**: - [GitLab JavaScript style guide](https://docs.gitlab.com/development/fe_guide/style/javascript/): ~1,700 tokens - [GitLab SCSS style guide](https://docs.gitlab.com/development/fe_guide/style/scss/): ~3,300 tokens - [GitLab Ruby style guide](https://docs.gitlab.com/development/backend/ruby_style_guide/): ~1,800 tokens - [GitLab Go guide](https://docs.gitlab.com/development/go_guide/): ~4,100 tokens - [Google Go best practices](https://google.github.io/styleguide/go/best-practices): ~20,000 tokens - [Google TypeScript style guide](https://google.github.io/styleguide/tsguide.html): ~20,000 tokens - [Airbnb JavaScript style guide](https://github.com/airbnb/javascript): ~18,000 tokens - [Airbnb Ruby style guide](https://github.com/airbnb/ruby): ~8,000 tokens - [Airbnb CSS style guide](https://github.com/airbnb/css): ~1,500 tokens **Current relevant feature limits (as of March 2025)**: - Code generation context window: ~142,856 tokens (roughly 500,000 characters) - Chat context window: ~200,000 tokens The team should set specific limits, considering: - Providing sufficient flexibility for teams and users - Managing context window limitations of underlying models - Maintaining reasonable latency - Avoiding quality degradation with verbose instructions - Validation errors for rules exceeding system capabilities ### Programming Language Support - No specific limitations expected; all languages supported by Duo should work with custom rules - Primary focus on [core languages](https://docs.gitlab.com/user/project/repository/code_suggestions/supported_extensions/#supported-languages) - Custom rules may help improve quality for languages outside the core set ### Telemetry Considerations - For each request, logging whether project/group or user rules were applied would be valuable - For privacy reasons, not logging the actual rule content by default is recommended - Consider logging rule content only when users have opted in to AI logging - Track usage patterns across different features to understand effectiveness