Skip to content

[CHORE][CICD]: Setup SonarQube quality gate #391

@crivetimihai

Description

@crivetimihai

📊 Chore: Setup SonarQube Quality Gate

Goal

Integrate SonarQube/SonarCloud analysis into the CI/CD pipeline with quality gates that block PRs failing code quality thresholds.

Why Now?

  1. Local Setup Exists: sonar-code.properties and podman-compose-sonarqube.yaml are already configured
  2. Quality Enforcement: PRs should be blocked if they introduce bugs, vulnerabilities, or code smells
  3. Technical Debt Tracking: SonarQube provides visibility into accumulated technical debt
  4. Coverage Tracking: Centralized coverage reporting with historical trends
  5. Security Analysis: SAST (Static Application Security Testing) for vulnerability detection

Current State

Component Status
sonar-code.properties ✅ Configured
podman-compose-sonarqube.yaml ✅ Available
Makefile targets sonar-up-*, sonar-submit-*
GitHub Actions workflow ❌ Not implemented
SonarCloud integration ❌ Not configured
Quality gate ❌ Not defined

📖 User Stories

US-1: Maintainer - PR Quality Gate

As a Maintainer
I want PRs to be blocked if they fail the SonarQube quality gate
So that code quality is enforced before merge

Acceptance Criteria:

Given a PR is opened with code changes
When the SonarQube analysis workflow runs
Then the analysis should check:
  - No new bugs introduced
  - No new vulnerabilities
  - No new security hotspots
  - Code coverage >= 80% on new code
  - Duplication <= 3% on new code
And the PR status check should fail if any threshold is exceeded
And the PR should be blocked from merge

Technical Requirements:

  • Create .github/workflows/sonarqube.yml
  • Configure SonarCloud or self-hosted SonarQube
  • Add SONAR_TOKEN to GitHub secrets
  • Configure quality gate thresholds
US-2: Developer - Analysis Feedback

As a Developer
I want to see SonarQube analysis results in my PR
So that I can fix issues before review

Acceptance Criteria:

Given a PR with code quality issues
When the SonarQube analysis completes
Then the PR should show:
  - Summary comment with issue counts
  - Link to full SonarQube dashboard
  - Inline annotations on problem lines (if using SonarCloud)
And new issues should be clearly distinguished from existing ones

Technical Requirements:

  • Enable PR decoration in SonarCloud/SonarQube
  • Configure analysis scope to focus on changed files
  • Add inline annotations where supported
US-3: Tech Lead - Quality Trends

As a Tech Lead
I want to track code quality trends over time
So that I can monitor technical debt accumulation

Acceptance Criteria:

Given the main branch is analyzed on each merge
When I view the SonarQube dashboard
Then I should see:
  - Overall quality rating (A-E)
  - Bug, vulnerability, and code smell counts
  - Test coverage percentage with trends
  - Duplication percentage with trends
  - Technical debt estimation
And historical data should be preserved for comparison

Technical Requirements:

  • Analyze main branch on push (not just PRs)
  • Configure long-lived branch analysis
  • Set up project dashboard in SonarQube/SonarCloud
US-4: Security Engineer - Vulnerability Detection

As a Security Engineer
I want SonarQube to detect security vulnerabilities
So that security issues are caught early in development

Acceptance Criteria:

Given code with potential security issues (SQL injection, XSS, etc.)
When the SonarQube analysis runs
Then security vulnerabilities should be flagged
And security hotspots should be identified for review
And the quality gate should fail on new vulnerabilities

Technical Requirements:

  • Enable security rules in SonarQube profile
  • Configure Python security rules
  • Set quality gate to fail on any new vulnerability

📋 Implementation Tasks

Phase 1: SonarCloud Setup (Recommended)

  • Create SonarCloud organization for IBM/mcp-context-forge
  • Import project and configure analysis
  • Generate SONAR_TOKEN and add to GitHub secrets
  • Enable automatic analysis on PRs
  • Configure PR decoration

Phase 2: GitHub Actions Workflow

  • Create .github/workflows/sonarqube.yml
  • Configure triggers: push to main, pull_request
  • Generate coverage report in Cobertura/XML format
  • Run sonar-scanner with coverage data
  • Add quality gate check step

Phase 3: Quality Gate Configuration

  • Define quality gate thresholds:
    • Coverage on new code >= 80%
    • Duplicated lines on new code <= 3%
    • Maintainability rating >= A
    • Reliability rating >= A
    • Security rating >= A
    • No new bugs
    • No new vulnerabilities
  • Apply quality gate to project

Phase 4: Documentation & Monitoring

  • Add SonarCloud badge to README
  • Document quality gate requirements in CONTRIBUTING.md
  • Set up notifications for quality gate failures
  • Create dashboard for team visibility

⚙️ Proposed Workflow

.github/workflows/sonarqube.yml

name: SonarQube Analysis

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  contents: read
  pull-requests: read

jobs:
  sonarqube:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for blame data

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          pip install -e ".[dev]"

      - name: Run tests with coverage
        run: |
          pytest --cov=mcpgateway --cov-report=xml:coverage.xml \
                 --junitxml=junit.xml -n auto

      - name: SonarCloud Scan
        uses: SonarSource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
        with:
          args: >
            -Dsonar.projectKey=IBM_mcp-context-forge
            -Dsonar.organization=ibm
            -Dsonar.python.coverage.reportPaths=coverage.xml
            -Dsonar.python.xunit.reportPath=junit.xml

      - name: Check Quality Gate
        uses: SonarSource/sonarcloud-github-action@master
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
        with:
          args: -Dsonar.qualitygate.wait=true

Quality Gate Thresholds

Metric Threshold Applies To
Coverage >= 80% New Code
Duplicated Lines <= 3% New Code
Maintainability Rating A New Code
Reliability Rating A New Code
Security Rating A New Code
Bugs 0 New Code
Vulnerabilities 0 New Code

✅ Success Criteria

  • SonarCloud/SonarQube project created
  • GitHub Actions workflow running on PRs
  • Quality gate blocking failing PRs
  • Coverage data visible in dashboard
  • PR decoration showing issues
  • README badge showing quality status

🔗 Related

  • sonar-code.properties - Existing scanner configuration
  • podman-compose-sonarqube.yaml - Local SonarQube setup
  • Makefile - sonar-* targets for local scanning

📚 References

Metadata

Metadata

Assignees

Labels

SHOULDP2: Important but not vital; high-value items that are not crucial for the immediate releasechoreLinting, formatting, dependency hygiene, or project maintenance chorescicdIssue with CI/CD process (GitHub Actions, scaffolding)devopsDevOps activities (containers, automation, deployment, makefiles, etc)readyValidated, ready-to-work-on items

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions