Skip to content

fix: add coverage gate to block PRs on coverage reduction#245

Merged
tjgreen42 merged 1 commit intomainfrom
fix/coverage-gate
Mar 2, 2026
Merged

fix: add coverage gate to block PRs on coverage reduction#245
tjgreen42 merged 1 commit intomainfrom
fix/coverage-gate

Conversation

@tjgreen42
Copy link
Copy Markdown
Collaborator

@tjgreen42 tjgreen42 commented Mar 1, 2026

Summary

  • The coverage job collected data and uploaded to Codecov but never enforced
    thresholds — it always succeeded regardless of coverage values
  • Codecov status checks (codecov/project, codecov/patch) weren't appearing
    on PRs, likely due to a missing/invalid CODECOV_TOKEN, and
    fail_ci_if_error was false so the upload failure was silent
  • Branch protection had an empty required checks list (all-tests-passed was
    not listed), so CI failures couldn't block merges — now fixed

Changes

  • Add in-workflow coverage gate to both ci.yml and coverage.yml that:
    • Enforces a minimum 85% line coverage threshold
    • Caches the main branch baseline via actions/cache
    • Compares PR coverage against baseline, fails if it drops by more than 1%
  • Change fail_ci_if_error from false to true on Codecov upload
  • Added all-tests-passed to branch protection required status checks

How the baseline works

  • On pushes to main, the coverage percentage is saved to GitHub Actions
    cache with key coverage-baseline-<sha>
  • On PRs, the most recent main branch cache is restored and compared against
    the PR's coverage
  • First run after this merges will skip the reduction check (no baseline yet)
    and establish the initial baseline

Testing

  • CI on this PR validates the gate runs (no baseline exists yet, so the
    reduction check was skipped, but the minimum threshold check ran and passed)
  • After merging, the next PR that reduces coverage by >1% will fail

The coverage job collected data and uploaded to Codecov but never
enforced thresholds. Codecov status checks (codecov/project,
codecov/patch) weren't appearing on PRs, likely due to a missing or
invalid CODECOV_TOKEN, and fail_ci_if_error was set to false so the
failure was silent.

Add an in-workflow coverage gate that:
- Enforces a minimum 85% line coverage threshold
- Caches the main branch baseline and compares on PRs
- Fails the job if coverage drops by more than 1%

Also change fail_ci_if_error from false to true on the Codecov upload
step so token issues surface immediately.

Note: branch protection currently has no required status checks. To
fully block merges, add "all-tests-passed" as a required check in
the repo's branch protection settings.
@tjgreen42 tjgreen42 marked this pull request as ready for review March 2, 2026 04:12
@tjgreen42 tjgreen42 merged commit e8601e2 into main Mar 2, 2026
15 checks passed
@tjgreen42 tjgreen42 deleted the fix/coverage-gate branch March 2, 2026 04:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant