Add check_changelog_updated.py: pre-push + CI CHANGELOG enforcement (Closes #478)#479
Conversation
…478) Closes #478. Adds scripts/check_changelog_updated.py, which diffs the current branch against origin/main and fails if any substantive file (vera/, spec/, SKILL.md) was changed without a matching new entry in CHANGELOG.md. ## Files - scripts/check_changelog_updated.py: the check itself - tests/test_check_changelog_updated.py: 54 unit + end-to-end tests (classification, diff parsing, trailer detection, temp-repo integration covering substantive/exempt/label/trailer paths) ## Integration - .pre-commit-config.yaml: new 'check-changelog-updated' hook at 'pre-push' stage (not per-commit, to avoid feature-branch noise) - .github/workflows/ci.yml: new step in the 'lint' job with 'fetch-depth: 0' so origin/main is available for the diff; the step is skipped when the PR has a 'skip-changelog' label ## Escape hatches - Commit trailer 'Skip-changelog: <reason>' (Git-native, local + CI) - PR label 'skip-changelog' (CI-only) - CHANGELOG_CHECK_BASE=<ref> env var to override the base ref ## Documentation - CONTRIBUTING.md: new 'Pre-push hook: CHANGELOG enforcement' subsection documenting the rule, install command, and escape hatches; install instructions updated to 'pre-commit install --hook-type pre-push' - TESTING.md: hook count 23 -> 24, test count 3,253 -> 3,307, new row for test_check_changelog_updated.py - ROADMAP.md: test count 3,307 + v0.0.113 totals line - CHANGELOG.md: new '[Unreleased] Added' entry documenting the rule Motivation: #474 merged without a CHANGELOG entry, HISTORY.md update, or KNOWN_ISSUES.md cleanup because none of those files were in the commit — the existing hooks only run on files they touch. This check reverses the logic: run always, check that the PR diff is self-consistent. Co-Authored-By: Claude <noreply@anthropic.invalid>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThe PR adds enforcement that substantive changes (e.g., Changes
Sequence Diagram(s)sequenceDiagram
participant Dev as Developer
participant Local as Pre-push hook (local)
participant CI as CI lint job
participant Git as Git remote / origin/main
participant Script as check_changelog_updated.py
participant CH as CHANGELOG.md
Dev->>Local: git push
Local->>Git: fetch base (origin/main)
Local->>Script: run check (diff HEAD..base)
Script->>Git: git diff & git log
Script->>CH: inspect CHANGELOG diff
alt changelog present or skip
Script-->>Local: exit 0 (allow push)
else
Script-->>Local: exit 1 (block push)
end
CI->>Git: workflow run (push/PR)
CI->>Script: run check (same flow)
Script-->>CI: pass/fail outcome
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #479 +/- ##
=======================================
Coverage 90.37% 90.37%
=======================================
Files 58 58
Lines 19588 19588
Branches 225 225
=======================================
Hits 17703 17703
Misses 1881 1881
Partials 4 4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Matches the existing pattern: HISTORY rows track work alongside the CHANGELOG's [Unreleased] section, with '—' as a placeholder version that gets replaced during release-prep for the version that ships this work. Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CONTRIBUTING.md`:
- Line 88: The sentence incorrectly implies all 24 hooks run on every commit;
update the wording after "pre-commit install" to clearly separate which hooks
run on commit vs which run on push by stating something like: X commit-time
hooks run on each commit and Y pre-push hooks run only when pushing (explicitly
call out the changelog/pre-push hook), and adjust the subsequent guideline about
"**/*.md" to remain as-is for Markdown file review scope; locate and edit the
line containing "pre-commit install" / "pre-commit install --hook-type pre-push"
and the mention of "changelog hook" to make this distinction explicit.
In `@scripts/check_changelog_updated.py`:
- Around line 139-151: The current is_substantive uses any(path == p or
path.startswith(p) for p in EXEMPT_PREFIXES) which overmatches file-like
exemptions (e.g. "README.md" also matches "README.md.bak"); change the check so
that EXEMPT_PREFIXES entries that represent directories (e.g. end with '/' or
os.sep, or explicitly contain a trailing slash) are matched with
path.startswith(prefix) but entries that represent files are matched with exact
equality (path == file_name). Update is_substantive to normalize path and then
iterate EXEMPT_PREFIXES, using startswith only for directory-style entries and
exact match for file-style entries to avoid accidental overmatching.
- Around line 153-172: _changelog_has_new_entry currently treats any added "- "
line as a valid new entry; tighten it so added bullets only count if they appear
inside the Unreleased section (or if the diff itself adds a new version
heading). Modify _changelog_has_new_entry to iterate diff lines while tracking
the current section header: whenever a line (context " " or added "+") has a
heading starting with "## [", parse the bracketed name (e.g., "Unreleased" or a
version) and set current_section accordingly; then only treat an added "+ - "
bullet as a new entry if current_section == "Unreleased". Additionally, keep the
existing behavior that an added "## [" line itself (i.e., an added version
heading) immediately returns True. Update the function _changelog_has_new_entry
to implement this section-tracking logic.
In `@TESTING.md`:
- Line 377: Rephrase the sentence that currently states "After running
`pre-commit install`, every commit is checked by 24 hooks:" to avoid implying
all 24 hooks run on every commit—clarify that 24 hooks are configured overall
but some (e.g., `check-changelog-updated`) run at pre-push rather than
per-commit; also update the pre-commit section to include the coding guideline
line exactly as given: "`**/*.md`: Review Markdown files for factual accuracy
against the codebase, broken links, and outdated information." Ensure you update
the line in TESTING.md where the original sentence and hook count appear and
keep the hook count (24) but separate per-commit vs pre-push behavior in the
wording.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3991da71-3b7e-4d89-8df0-a657d98bd929
📒 Files selected for processing (8)
.github/workflows/ci.yml.pre-commit-config.yamlCHANGELOG.mdCONTRIBUTING.mdROADMAP.mdTESTING.mdscripts/check_changelog_updated.pytests/test_check_changelog_updated.py
Three legitimate correctness findings from CodeRabbit on #479: 1. is_substantive overmatched file-style exemptions. With EXEMPT_PREFIXES containing entries like 'README.md' (no trailing slash), 'path.startswith("README.md")' returned True for 'README.md.bak' — silently exempting files that aren't actually README.md. Split matching by shape: directory-style entries (trailing '/') use startswith; file-style entries use exact equality. Added _is_exempt helper. 2. _changelog_has_new_entry counted any added '+- ' line as a new entry, including bullets inserted inside released-version entries (e.g. a prose fix on the v0.0.111 description would pass the check). Rewrote to walk the diff while tracking the current section header (from both context lines and added lines); added bullets now only count when current_section == 'Unreleased'. Added version headings (+## [X.Y.Z]) still immediately count. Fixed an off-by-one in section-name parsing along the way (content[3:end] included the opening '['). 3. CONTRIBUTING.md and TESTING.md wording implied all 24 hooks run on every commit. Rewrote to clarify: 23 at commit time, 1 (check-changelog-updated) at push time. Kept the 'checked by 24 hooks' phrase so check_doc_counts.py still matches. Tests: +9 regression cases (file-style overmatch for .bak/.orig/etc, bullet outside [Unreleased], bullet inside [Unreleased], section tracking across context lines, directory-style nested paths). 63 tests in test_check_changelog_updated.py (was 54), 3,316 project total (was 3,307). Co-Authored-By: Claude <noreply@anthropic.invalid>
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CONTRIBUTING.md`:
- Around line 110-111: Update the sentence describing the pre-push hook so it
reflects the actual classifier behaviour: instead of saying only changes under
`vera/`, `spec/`, and `SKILL.md` trigger the check, state that the hook flags
any change to a non-exempt top-level path as substantive and requires adding an
entry to `CHANGELOG.md`; keep the note that the same check runs in CI and
mention the current exemption mechanism (e.g., maintained patterns like
`**/*.md` or other exempt paths) so contributors aren’t surprised by failures on
new or unknown top-level files.
In `@scripts/check_changelog_updated.py`:
- Around line 177-180: The current heading-check logic in
scripts/check_changelog_updated.py treats any added "## [...]" as valid
including "## [Unreleased]"; change the check so that an added heading is only
accepted if it is not exactly "Unreleased" OR if it is "Unreleased" then there
is at least one added bullet line inside that Unreleased section. Locate the
code that inspects added_lines and the regex that detects new headings (the
variable/logic that sets has_new_version_heading or matches "## \\[(.*?)\\]")
and update it to reject a sole "+## [Unreleased]" by verifying either
match.group(1) != "Unreleased" or, when it is "Unreleased", scanning the
subsequent added_lines for a "+- " (or other bullet marker) within the
Unreleased block before returning success.
In `@tests/test_check_changelog_updated.py`:
- Around line 177-192: Add a regression test alongside
test_detects_new_version_heading that constructs a diff where the only change is
an added "## [Unreleased]" heading with no bullets (e.g. a line starting with
"+## [Unreleased]" and no subsequent "+" bullet lines), use
monkeypatch.setattr(_mod, "_run", lambda cmd: diff) like the existing tests, and
assert that _mod._changelog_has_new_entry("origin/main") returns False to ensure
a heading-only addition does not count as a new entry.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: bcdcd595-c074-4608-9ef5-1d498180bac9
📒 Files selected for processing (5)
CONTRIBUTING.mdROADMAP.mdTESTING.mdscripts/check_changelog_updated.pytests/test_check_changelog_updated.py
Three findings, all valid:
1. scripts/check_changelog_updated.py was short-circuiting on *any*
added '## [...]' heading, including a bare '+## [Unreleased]'
that represents structural scaffolding rather than actual
content. Changed the heading branch so it only short-circuits
on non-'Unreleased' version headings; a bare '+## [Unreleased]'
now falls through to the bullet-under-section check, which
will return True only if there's a subsequent '+- ' bullet.
2. Added two regression tests matching the new semantics:
- test_bare_unreleased_heading_alone_does_not_count
(asserts '+## [Unreleased]' alone returns False)
- test_added_unreleased_heading_with_bullet_counts
(asserts '+## [Unreleased]' + '+- bullet' returns True)
3. CONTRIBUTING.md prose said only 'vera/, spec/, SKILL.md'
trigger the check, but the classifier is actually exempt-list-
based (any non-exempt top-level path is substantive). Rewrote
the 'Pre-push hook: CHANGELOG enforcement' subsection to list
the exempt categories explicitly and note the conservative
default for new top-level folders, so contributors aren't
surprised by failures on files like 'stdlib/' or 'runtime/'.
Test count: 3,316 -> 3,318 (2 new regression tests).
Co-Authored-By: Claude <noreply@anthropic.invalid>
Closes #478.
Summary
Adds
scripts/check_changelog_updated.py, which diffs the current branch againstorigin/mainand fails if any substantive file (vera/,spec/,SKILL.md) was changed without a matching new entry inCHANGELOG.md. Wired into the pre-commit chain at thepre-pushstage and into the CIlintjob.Catches the kind of missed release-prep that happened on #474 (calls.py decomposition merged without a CHANGELOG entry).
Files
scripts/check_changelog_updated.pytests/test_check_changelog_updated.py.pre-commit-config.yamlpre-pushstage.github/workflows/ci.ymllintjob +fetch-depth: 0CONTRIBUTING.mdTESTING.mdROADMAP.mdCHANGELOG.md[Unreleased] AddedentryClassification
Substantive (requires a CHANGELOG entry):
vera/,spec/,SKILL.md.Exempt:
tests/,scripts/,.github/,docs/,examples/,editors/,assets/; all root-level docs (README.md,HISTORY.md,ROADMAP.md,KNOWN_ISSUES.md,FAQ.md,CONTRIBUTING.md,TESTING.md,AGENTS.md,CLAUDE.md,DE_BRUIJN.md,EXAMPLES.md,CHANGELOG.md,LICENSE);pyproject.toml,uv.lock,.pre-commit-config.yaml,.coderabbit.yaml,.gitignore.Unknown paths default to substantive (conservative — a hypothetical new
stdlib/orruntime/folder won't accidentally bypass the check).Escape hatches
Skip-changelog: <reason>trailer in any commit message on the branch (works locally and in CI).skip-changeloglabel to the PR (CI-only; theif:condition on the step skips it when the label is present).Why pre-push, not pre-commit
Feature branches typically have 5–20 commits. Running the check per-commit would block every intermediate commit with "no CHANGELOG entry" — pure friction. Pre-push gives the whole branch's worth of commits before checking. Enable locally with
pre-commit install --hook-type pre-push(CONTRIBUTING.md updated).Self-test
Running the script on this PR exits 0 — all files touched are exempt. The CHANGELOG entry added here is a bonus (following project convention for CI tooling, mirroring the v0.0.107 pattern), not a requirement. The substantive-without-CHANGELOG path is covered by the end-to-end pytest suite against temp git repos.
Verification
check_changelog_updated.pyself-test (exit 0)🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Chores
Tests
Documentation