ci: Major refactor of release-workflows by ko3n1g · Pull Request #1911 · NVIDIA-NeMo/Curator

ko3n1g · 2026-05-05T08:54:56Z

Why

See the design discussion in NVIDIA-NeMo/FW-CI-templates#466.

What

Delete .github/workflows/build-test-publish-wheel.yml.
Rewrite .github/workflows/release.yml as the single caller for both push and workflow_dispatch.

Test plan

PR validate-only (push (mirror), sha fdbac7f, 2026-05-07T12:01:00Z, success): https://github.com/NVIDIA-NeMo/Curator/actions/runs/25494488916
workflow_dispatch dry-run=true (sha fdbac7f, 2026-05-07T12:00:56Z, success): https://github.com/NVIDIA-NeMo/Curator/actions/runs/25494485717
Real release via workflow_dispatch dry-run=false on the next planned RC.

Rollout

Land FW-CI-templates#466.
Cut FW-CI-templates v1.0.0.
Bump the SHA pin in this PR → tag.

…nly mode See NVIDIA-NeMo/FW-CI-templates#466 for design discussion. - Delete build-test-publish-wheel.yml. - Rewrite release.{yml,yaml} as the single caller for both push and workflow_dispatch. validate-only derives from the trigger. - One pin to FW-CI-templates governs PR rehearsal and real release. Signed-off-by: oliver könig <okoenig@nvidia.com>

greptile-apps · 2026-05-05T08:58:07Z

Greptile Summary

This PR consolidates the two separate release workflows (build-test-publish-wheel.yml and release.yml) into a single release.yml that handles both push-triggered validation and workflow_dispatch-triggered releases, delegating to a new v1.0.0 of the shared FW-CI-templates reusable workflow. It also adds MANIFEST.in data-file globs and a [tool.check-manifest] configuration to improve sdist completeness.

release.yml rewrite: adds push triggers with validate-only: true, a new publish-docs input, concurrency cancellation for push events, secrets: inherit, and a release-summary job that queries gh run view — though the actions: read permission needed for those calls is absent from the top-level permissions block, and the || true fallback on inputs.generate-changelog and inputs.publish-docs makes those flags non-overridable to false via workflow_dispatch.
MANIFEST.in + pyproject.toml: the recursive-include directive correctly adds data files to the sdist, but include-package-data = true (or an equivalent [tool.setuptools.package-data] entry) is still absent, so the same files will be missing from the installed wheel.

Confidence Score: 3/5

Not ready to merge: the release-summary job will always fail on both gh run view calls due to missing permissions, boolean expressions prevent callers from disabling changelog generation or doc publishing, and data files added via MANIFEST.in are still absent from the built wheel.

Three distinct defects exist in the changed code. The release-summary job uses gh run view but the top-level permissions block does not grant actions: read, so every run of this job fails with a 403. The expressions inputs.generate-changelog || true and inputs.publish-docs || true collapse to true regardless of what a caller passes, making both flags effectively non-functional for workflow_dispatch users who want to suppress them. Finally, MANIFEST.in correctly adds data files to the sdist but the wheel build is untouched — include-package-data is absent from [tool.setuptools], so pip install nemo-curator users still won't get the data files the PR intends to ship.

release.yml needs the permission, boolean-expression, and release-summary fixes; pyproject.toml needs include-package-data = true under [tool.setuptools].

Important Files Changed

Filename	Overview
.github/workflows/build-test-publish-wheel.yml	Deleted — its push-trigger validation role is subsumed by the refactored release.yml.
.github/workflows/config/changelog-config.json	Indentation normalised, cherry-pick transformer added, tag_resolver reformatted; missing trailing newline at EOF.
.github/workflows/release.yml	Major rewrite: adds push-trigger validate-only path, docs publishing, concurrency, and a release-summary job; several issues remain around boolean coercion expressions, missing permissions, and the release-summary job logic.
MANIFEST.in	Adds recursive-include for data files (csv, json, yaml, txt, md, py.typed) under nemo_curator; fixes sdist inclusion but wheel packaging still requires a matching setuptools configuration.
pyproject.toml	Adds [tool.check-manifest] ignore list; however include-package-data is absent from [tool.setuptools], so non-Python data files added via MANIFEST.in are still excluded from the built wheel.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A([push or workflow_dispatch]) --> B{event_name?}
    B -- push --> C[release job validate-only true]
    B -- workflow_dispatch --> D[release job validate-only false]
    C --> E[FW-CI-templates _release_library v1.0.0 validation only]
    D --> F[FW-CI-templates _release_library v1.0.0 full release]
    E --> G[release-summary job]
    F --> G
    G --> H[gh run view GITHUB_RUN_ID]
    H --> I{failed jobs?}
    I -- no --> J[exit 0 success]
    I -- yes --> K[exit 1 failure]
    H -. missing actions read permission .-> L[403 always fails]

_{Reviews (38): Last reviewed commit: "Merge branch 'main' into ko3n1g/refactor..." | Re-trigger Greptile}

greptile-apps · 2026-05-05T08:58:11Z

+      validate-only: ${{ github.event_name != 'workflow_dispatch' }}
+      dry-run: ${{ inputs.dry-run || false }}
+      version-bump-branch: ${{ inputs.version-bump-branch || github.ref_name }}
+      gh-release-use-changelog-builder: ${{ inputs.generate-changelog || true }}


The || true fallback silently overrides an explicit false from workflow_dispatch. In GitHub Actions expressions, false || true evaluates to true, so dispatching with generate-changelog: false will still produce a changelog run. For push events (where the input is absent and evaluates to an empty string), '' || true correctly produces true — but the same expression breaks the user-controlled path.

Suggested change

gh-release-use-changelog-builder: ${{ inputs.generate-changelog || true }}

gh-release-use-changelog-builder: ${{ github.event_name == 'workflow_dispatch' && inputs.generate-changelog || true }}

greptile-apps · 2026-05-05T08:58:12Z

  release:
-    uses: NVIDIA-NeMo/FW-CI-templates/.github/workflows/_release_library.yml@v0.73.0
+    if: '!cancelled()'
+    uses: NVIDIA-NeMo/FW-CI-templates/.github/workflows/_release_library.yml@b7091aadca729136b12c48d45b832633c681b131


The reusable workflow is pinned to a raw commit SHA (b7091aadca729136b12c48d45b832633c681b131) that has no corresponding public tag. The PR's own rollout plan (step 3) says to replace this SHA with the v1.0.0 tag before merging. If merged in its current state, the pin references an internal, untagged commit that is harder to audit and could silently break if upstream history is cleaned up.

greptile-apps · 2026-05-05T08:58:13Z

+      release-ref: ${{ inputs.release-ref || github.sha }}
      python-package: nemo_curator
-      python-version: "3.11"
+      python-version: "3.10"


The Python version was silently downgraded from 3.11 to 3.10. Both the deleted build-test-publish-wheel.yml and the previous release.yml used 3.11. Releasing a wheel built against an older Python minor version can change ABI compatibility and may produce a different wheel tag than what consumers expect. Was this intentional (e.g., to match the project's declared minimum version)?

…quirements-file) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

greptile-apps · 2026-05-06T19:10:45Z

+[tool.check-manifest]
+ignore = [


MANIFEST.in directives only affect the sdist. Setuptools consults a separate mechanism for wheels: either include-package-data = true in [tool.setuptools] (respects VCS + MANIFEST.in) or explicit [tool.setuptools.package-data] globs. Without one of these, nemo_curator/utils/code_meta.csv and every other non-.py file added via MANIFEST.in will still be absent from the installed wheel — the runtime breakage this PR intends to fix will remain for pip install nemo-curator users.

Suggested change

[tool.check-manifest]

ignore = [

[tool.setuptools]

include-package-data = true

[tool.check-manifest]

ignore = [

…ansformer Why: HYBRID mode renders raw commits when no PR matches by merge_commit_sha (helps release branches built via cherry-pick). The transformer cleans up cp titles to show the inner PR title only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T08:26:40Z

/ok to test 6beb683

…ry-run) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T08:28:17Z

/ok to test 79c89e5

…sthrough Why: SLACK_WEBHOOK now resolves at the env scope (public/main) so the env-scoped secret value is used. No longer pass it as a workflow_call secret. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

Lets env-scoped SLACK_WEBHOOK reach the notify job in the called workflow. Signed-off-by: oliver könig <okoenig@nvidia.com>