ci: Major refactor of release-workflows by ko3n1g · Pull Request #4602 · NVIDIA/Megatron-LM

ko3n1g · 2026-05-04T15:06:18Z

Why

See the design discussion in NVIDIA-NeMo/FW-CI-templates#466.

What

Delete .github/workflows/build-test-publish-wheel.yml.
Rewrite .github/workflows/release.yaml as the single caller for both push and workflow_dispatch.

Test plan

PR validate-only (push (mirror), sha 1717260, 2026-05-07T11:35:45Z, success): https://github.com/NVIDIA/Megatron-LM/actions/runs/25493355697
workflow_dispatch dry-run=true (sha 1717260, 2026-05-07T11:35:50Z, success): https://github.com/NVIDIA/Megatron-LM/actions/runs/25493359313
Real release via workflow_dispatch dry-run=false on the next planned RC.

Rollout

Land FW-CI-templates#466.
Cut FW-CI-templates v1.0.0.
Bump the SHA pin in this PR → tag.

Switch the release.yaml orchestration to use NVIDIA-NeMo/FW-CI-templates' new composable workflows: - _release_bump.yml — multi-target bump for both megatron-core and megatron_fsdp via a JSON `bump-targets` input. - _release_finalize.yml — GH release + Slack notify, taking release-version as an input from the bump output. Megatron-LM keeps: - Its own _build_test_publish_wheel.yml (multi-arch matrix manylinux build for megatron-core arm64+amd64 and megatron-fsdp amd64), wired in as a sibling job between bump and finalize. - Its own release-docs.yml (custom docs flow), invoked after finalize. This removes the local _release_library.yml (now superseded) and aligns release orchestration with the rest of the NeMo framework so future upstream improvements (validate-only PR rehearsal, etc.) become available without copying YAML. Signed-off-by: oliver könig <okoenig@nvidia.com>

copy-pr-bot · 2026-05-04T15:06:24Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

ko3n1g · 2026-05-04T15:06:57Z

/ok to test

Merge build-test-publish-wheel.yml (push-triggered, wheel-only) into release.yaml (workflow_dispatch-triggered, full release) so a single file with one FW-CI-templates pin governs both per-PR rehearsal and real release. Mirrors the Megatron-Bridge consolidation. On push (PR/main/deploy-release/merge_group): validate-only=true; the full pipeline rehearses (bump computes only, wheels build without publish, GH release payload echoed, docs publish skipped, Slack suppressed). On workflow_dispatch: validate-only=false; existing dry-run knob still controls whether wheel publish + GH release POST + docs publish fire or stay inert. The push trigger now exercises the bump and finalize paths every PR, not just the wheel build. Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-04T15:13:06Z

/ok to test 729bb50

Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-04T15:44:57Z

/ok to test ed1d2fe

When dispatching against a copy-pr-bot mirror branch (pull-request/<id>), pre-flight's 'Get PR info' step matches startsWith(github.ref, 'refs/heads/pull-request/') and tries to look up a PR — but the event_name is workflow_dispatch, not pull_request, so the lookup fails. Skip pre-flight entirely on dispatch events; downstream jobs already short-circuit their pre-flight-output checks when github.event_name == 'workflow_dispatch'. Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-04T16:07:59Z

/ok to test b8919ca

ko3n1g · 2026-05-04T16:08:39Z

/ok to test ed1d2fe

Default needs-success behavior would skip bump/wheels when pre-flight is skipped. Switch their if: to !cancelled() && (pre-flight.result success or skipped), keeping the auto-skip-on-failure semantics intact. Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-04T16:09:51Z

/ok to test f273553

Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-04T16:16:16Z

/ok to test 4292889

ko3n1g · 2026-05-06T19:06:57Z

/ok to test 3fbbf8c

ko3n1g · 2026-05-06T22:23:34Z

/ok to test 3fbbf8c

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-06T22:23:56Z

/ok to test ee9b3f9

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T08:20:40Z

/ok to test d5c90a9

ko3n1g · 2026-05-07T08:26:52Z

/ok to test d5c90a9

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T08:28:29Z

/ok to test be26a40

…sthrough Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

Lets env-scoped SLACK_WEBHOOK reach the notify job in the called workflow. Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T09:30:16Z

/ok to test a03fd03

Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T11:20:46Z

/ok to test f54a6a1

Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g · 2026-05-07T11:35:38Z

/ok to test 1717260

Signed-off-by: oliver könig <okoenig@nvidia.com>

…!failure) Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> # Conflicts: # .github/workflows/build-test-publish-wheel.yml

svcnvidia-nemo-ci · 2026-05-11T12:24:50Z

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/25669939636

svcnvidia-nemo-ci · 2026-05-11T13:01:13Z

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/25671734703

svcnvidia-nemo-ci · 2026-05-11T15:04:26Z

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/25678362086

svcnvidia-nemo-ci · 2026-05-11T17:17:39Z

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/25685644892

copy-pr-bot Bot had a problem deploying to test May 4, 2026 15:08 Error

copy-pr-bot Bot temporarily deployed to main May 4, 2026 15:13 Inactive

copy-pr-bot Bot temporarily deployed to test May 4, 2026 15:14 Inactive

copy-pr-bot Bot temporarily deployed to public May 4, 2026 15:16 Inactive

ci: bump pre-flight pin to handle workflow_dispatch

ed1d2fe

Signed-off-by: oliver könig <okoenig@nvidia.com>

copy-pr-bot Bot temporarily deployed to main May 4, 2026 15:45 Inactive

copy-pr-bot Bot had a problem deploying to test May 4, 2026 15:47 Error

copy-pr-bot Bot temporarily deployed to public May 4, 2026 15:49 Inactive

copy-pr-bot Bot temporarily deployed to main May 4, 2026 16:08 Inactive

copy-pr-bot Bot had a problem deploying to test May 4, 2026 16:09 Error

copy-pr-bot Bot had a problem deploying to test May 4, 2026 16:10 Error

copy-pr-bot Bot temporarily deployed to main May 4, 2026 16:11 Inactive

copy-pr-bot Bot temporarily deployed to public May 4, 2026 16:14 Inactive

ci: bump FW-CI-templates pin to slack-message fix

4292889

Signed-off-by: oliver könig <okoenig@nvidia.com>

copy-pr-bot Bot had a problem deploying to test May 4, 2026 16:17 Error

copy-pr-bot Bot temporarily deployed to main May 4, 2026 16:17 Inactive

ko3n1g had a problem deploying to main May 4, 2026 16:19 — with GitHub Actions Failure

copy-pr-bot Bot temporarily deployed to public May 4, 2026 16:20 Inactive

refactor(release): bump FW-CI pin to 8b68280 (changelog HYBRID)

ee9b3f9

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

refactor(release): bump FW-CI pin to b57ebf9 ([skip ci] on bump commit)

d5c90a9

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

refactor(release): bump FW-CI pin to cb5e93b

be26a40

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g and others added 3 commits May 7, 2026 08:44

refactor(release): bump FW-CI pin to 43d259e + drop SLACK_WEBHOOK pas…

f7b4e95

…sthrough Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: bump FW-CI pin to d2f3dd3 + use secrets inherit

1a124e4

Lets env-scoped SLACK_WEBHOOK reach the notify job in the called workflow. Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: strip orphan secret keys after secrets inherit

a03fd03

Signed-off-by: oliver könig <okoenig@nvidia.com>

fix: TWINE_PASSWORD required-false (resolved at env scope)

f54a6a1

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: bump FW-CI pin to 64293f6 (slack render fix)

1717260

Signed-off-by: oliver könig <okoenig@nvidia.com>

ko3n1g added 6 commits May 7, 2026 20:14

ci: pin FW-CI templates to v1.0.0

76fcf36

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: pragma allowlist secret on 'secrets: inherit' lines

a8e0026

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: simplify release-summary if-block (always() was dead code)

f024e5d

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: simplify release if-block (replace explicit success/skipped with …

8c9f942

…!failure) Signed-off-by: oliver könig <okoenig@nvidia.com>

Merge branch 'main' into ko3n1g/refactor/validate-only-release

5b33c35

Merge branch 'main' into ko3n1g/refactor/validate-only-release

6c706dc

Signed-off-by: oliver könig <okoenig@nvidia.com> # Conflicts: # .github/workflows/build-test-publish-wheel.yml

chtruong814 approved these changes May 10, 2026

View reviewed changes

Merge branch 'main' into ko3n1g/refactor/validate-only-release

a9466e9

svcnvidia-nemo-ci mentioned this pull request May 12, 2026

chore: nightly sync main into dev (11_05_2026) #4739

Closed

Conversation

ko3n1g commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Test plan

Rollout

Uh oh!

copy-pr-bot Bot commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

ko3n1g commented May 6, 2026

Uh oh!

ko3n1g commented May 6, 2026

Uh oh!

ko3n1g commented May 6, 2026

Uh oh!

ko3n1g commented May 7, 2026

Uh oh!

ko3n1g commented May 7, 2026

Uh oh!

ko3n1g commented May 7, 2026

Uh oh!

ko3n1g commented May 7, 2026

Uh oh!

ko3n1g commented May 7, 2026

Uh oh!

ko3n1g commented May 7, 2026

Uh oh!

svcnvidia-nemo-ci commented May 11, 2026

Uh oh!

svcnvidia-nemo-ci commented May 11, 2026

Uh oh!

svcnvidia-nemo-ci commented May 11, 2026

Uh oh!

svcnvidia-nemo-ci commented May 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ko3n1g commented May 4, 2026 •

edited

Loading