[CHORE] Evaluate and select additional CI/CD quality tools #2383
Description
Summary
Evaluate and select additional linters, security scanners, and quality tools to enhance the CI/CD pipeline. A comprehensive analysis has identified 100+ tools across 27 categories that could improve code quality, security, reliability, and developer experience.
Reference document: todo/missing-tools.md
Current Tooling Coverage
Well Covered
- ✅ Python linting (flake8, ruff, pylint, radon, vulture, etc.)
- ✅ Container security (Trivy, Grype, Hadolint, Dockle, Syft)
- ✅ Secret detection (gitleaks)
- ✅ Dependency vulnerabilities (pip-audit, dependency-review)
- ✅ Web linting (ESLint, HTMLHint, Stylelint)
Major Gaps
- ❌ GitHub Actions - No workflow linting (14 workflows unlinted)
- ❌ Helm Charts -
charts/directory has no CI validation - ❌ Type Checking - mypy/pyright in deps but disabled in CI
- ❌ API Quality - No OpenAPI linting or fuzzing
- ❌ Supply Chain - No SLSA provenance or signing
- ❌ Go/Rust - Limited linting for
mcp-servers/go/andplugins/rust/ - ❌ Commit Messages - Conventional commits not enforced
- ❌ PR Experience - Lint results don't appear as PR comments
Recommended Priority Implementation
Phase 1: Quick Wins (High Value, Low Effort)
- actionlint - Lint GitHub Actions workflows
- zizmor - Security-focused GitHub Actions analysis
- OSSF Scorecard - Measure repo security posture
- commitlint - Enforce conventional commits
- helm lint + chart-testing + helm-unittest - Validate Helm charts
- markdown-link-check - Find broken links in docs
- codespell - Catch common typos across codebase
- reviewdog - Surface lint results in PR comments
- safety - Second opinion on dependency vulnerabilities
- diff-cover - Coverage for changed lines only
Phase 2: Security Hardening
- Snyk (free tier) - Comprehensive security scanning
- trufflehog - Deep secret scanning (git history)
- checkov/kics - IaC security for Dockerfiles and Helm
- cosign - Sign container images
- slsa-github-generator - SLSA provenance for supply chain
- semgrep - Custom security rules (resolve opentelemetry conflict)
- kube-linter - K8s security best practices
- Socket.dev - Supply chain security
- gosec + govulncheck - Go security
- cargo-deny + cargo-audit - Rust security
Phase 3: Code Quality Improvements
- mypy in CI - Enable type checking (currently commented)
- refurb - Modern Python suggestions
- sourcery (free tier) - AI code suggestions
- schemathesis - Property-based API fuzzing
- pytest-testmon - Smart test selection
- wily - Track complexity over time
- spectral - OpenAPI linting
- darglint in CI - Docstring validation
- golangci-lint - Comprehensive Go linting
- clippy - Rust linting
- depcheck - Find unused JS dependencies
Phase 4: Developer Experience
- semantic-release or release-please - Automated releases
- git-cliff - Automated changelogs
- danger - PR automation
- act - Test GitHub Actions locally
- dive - Analyze Docker image layers
- pytest-benchmark - Performance regression testing
Phase 5: Advanced
- atheris - Coverage-guided Python fuzzing
- memray - Memory profiling
- toxiproxy - Chaos engineering
- Lighthouse CI - Web performance audits
- MegaLinter - Unified meta-linter (50+ linters)
- in-toto - Supply chain attestation
- cargo-udeps / cargo-geiger - Rust dependency analysis
Tool Categories (27 Total)
| # | Category | Key Missing Tools |
|---|---|---|
| 1 | Python Static Analysis | fixit, xenon, wemake-styleguide |
| 2 | Python Security | safety, semgrep, pyt |
| 3 | Python Type Checking | mypy (enable), pyright (enable) |
| 4 | Python Code Quality | refurb, sourcery, flake8-bugbear |
| 5 | Testing & Coverage | pytest-randomly, pytest-testmon, diff-cover |
| 6 | Fuzzing | schemathesis, atheris, OSS-Fuzz |
| 7 | Performance & Profiling | memray, py-spy, scalene |
| 8 | API Quality | spectral, optic, openapi-spec-validator |
| 9 | Documentation Quality | vale, codespell, markdown-link-check |
| 10 | Dependency Management | safety, deptry, Renovate |
| 11 | Container & Image Security | cosign, slsa-github-generator, docker-slim, dive |
| 12 | Infrastructure as Code | checkov, kics, terrascan |
| 13 | Kubernetes & Helm | helm lint, chart-testing, kube-linter, pluto |
| 14 | Secret Detection | trufflehog, detect-secrets (enable) |
| 15 | General Security | Snyk, OSSF Scorecard, Socket.dev |
| 16 | Git & Workflow Quality | actionlint, zizmor, commitlint, reviewdog |
| 17 | Frontend & Web | Biome, Lighthouse CI, depcheck, axe-core |
| 18 | License Compliance | FOSSA, scancode-toolkit, reuse |
| 19 | Code Metrics & Reporting | wily, MegaLinter |
| 20 | Accessibility | axe-core, pa11y, Lighthouse |
| 21 | Database | sqlfluff, squawk |
| 22 | Reliability & Chaos | Chaos Toolkit, toxiproxy |
| 23 | Code Review Automation | reviewdog, danger, CODEOWNERS |
| 24 | Release Management | semantic-release, release-please, git-cliff |
| 25 | Monorepo & Build | act, dagger, earthly |
| 26 | Go Language Tools | golangci-lint, gosec, govulncheck |
| 27 | Rust Language Tools | cargo-deny, cargo-udeps, cargo-geiger, clippy |
Acceptance Criteria
- Review
todo/missing-tools.mdfor full tool descriptions - Select tools for Phase 1 implementation
- Create individual issues/PRs for each selected tool
- Update
CLAUDE.mdwith new tool commands as added - Consider MegaLinter as single-action alternative
Alternative: MegaLinter
Instead of adding tools individually, consider MegaLinter which runs 50+ linters in a single GitHub Action with automatic PR comments and reporting. This could replace many individual tool integrations.
- uses: oxsecurity/megalinter@v7
env:
VALIDATE_ALL_CODEBASE: true
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}Related Files
todo/missing-tools.md- Full tool catalog with descriptionstodo/missing-linters.md- Original ChatGPT analysis (merged).github/workflows/- Current CI workflowsMakefile- Current tool targets.pre-commit-config.yaml- Pre-commit hooks