ci(e2e): withhold live messaging secrets from target refs by cv · Pull Request #4862 · NVIDIA/NemoClaw

cv · 2026-06-05T20:06:07Z

Summary

Prevent selective nightly E2E dispatches with an explicit target ref from receiving live messaging provider secrets. The messaging providers job now opts into live secrets only on trusted refs and blanks each live Slack, Discord, and Telegram secret for target-ref dispatches.

Changes

Guard messaging_live_secrets in .github/workflows/nightly-e2e.yaml with the same trusted-ref predicate used for Docker Hub credentials.
Blank live messaging provider secret mappings when workflow_dispatch supplies inputs.target_ref.
Update workflow contract tests to assert target-ref dispatches cannot expose live messaging secrets.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Verification

npx prek run --all-files passes
npm test passes
Tests added or updated for new or changed behavior
No secrets, API keys, or credentials committed
Docs updated for user-facing behavior changes
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Targeted checks run:

npx vitest run test/e2e-script-workflow.test.ts test/validate-e2e-coverage.test.ts --testTimeout 60000
npm run checks
commit and pre-push hooks

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

Security Improvements
- Enhanced protection of messaging provider credentials by conditionally gating access to live secrets, ensuring they are only available in trusted workflow scenarios.
Tests
- Strengthened E2E workflow contract tests to enforce stricter security gating for live messaging secrets in reusable workflows.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

coderabbitai · 2026-06-05T20:06:23Z

Complex PR? Review this PR in Change Stack to move by importance, not file order.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4f923673-fc76-41cc-b32c-509d738fafcc

📥 Commits

Reviewing files that changed from the base of the PR and between 9b16a9e and 53c848a.

📒 Files selected for processing (3)

.github/workflows/nightly-e2e.yaml
test/e2e-script-workflow.test.ts
test/validate-e2e-coverage.test.ts

📝 Walkthrough

Walkthrough

The PR gates live messaging provider credentials (Telegram, Discord, Slack tokens and channel IDs) behind a "trusted ref" expression in the nightly E2E workflow. The workflow now conditionally passes real secrets or empty strings based on whether the event is a scheduled run or a trusted dispatch. Tests are updated to verify this gating behavior and validate coverage expectations.

Changes

Trusted-ref gating for live messaging secrets

Layer / File(s)	Summary
Workflow conditional gating implementation `.github/workflows/nightly-e2e.yaml`	The `messaging-providers-e2e` job derives `messaging_live_secrets` from a trusted-ref expression and conditionally wires Telegram/Discord/Slack credentials as either real secrets or empty strings based on workflow dispatch source.
E2E contract testing for trusted-ref gating `test/e2e-script-workflow.test.ts`	Tests introduce a shared `trustedRefGuard` expression and validate that the messaging nightly job's `messaging_live_secrets` input equals this guard. New test replaces prior opt-in logic with one requiring trusted target refs and verifying each secret value uses the guard.
Coverage validation for conditional secret expressions `test/validate-e2e-coverage.test.ts`	Coverage tests now expect `messaging_live_secrets` to be a conditional GitHub Actions expression (not a boolean) and validate that each secret is forwarded only when the trusted-dispatch condition is satisfied.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested labels

CI/CD, E2E, area: ci, area: e2e, integration: slack

Poem

🐰 The secrets now hide behind trusted gates,
No more wide-open doors for dispatch awaits!
When the schedule runs true and the refs align,
The credentials flow safe—chef's kiss, divine! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main change: restricting live messaging secrets from target refs in CI/E2E workflows.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/e2e-live-messaging-secret-target-ref

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-06-05T20:07:15Z

E2E Advisor Recommendation

Required E2E: messaging-providers-e2e
Optional E2E: None

Dispatch hint: messaging-providers-e2e

Auto-dispatched E2E: messaging-providers-e2e via nightly-e2e.yaml at 6d077ae2aead7fcddaa9b5172ba11c5a02e2000e — nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

messaging-providers-e2e (high): Directly exercises the affected nightly job and validates the messaging credential provider/placeholder/L7-proxy chain with fake tokens, plus optional live-message behavior when repository secrets are available on trusted refs. This is the only existing E2E job whose workflow inputs and secrets were changed.

Optional E2E

None.

New E2E recommendations

ci-e2e-workflow-security (high): Static tests assert that live messaging secrets are gated for workflow_dispatch target_ref runs, but there is no existing E2E canary that dispatches the reusable runner against an explicit target_ref and proves the live secret environment variables are empty at runtime.
- Suggested test: Add a workflow-dispatched security canary for e2e-script.yaml that runs a harmless test/e2e shell script against an explicit target_ref and asserts TELEGRAM_BOT_TOKEN_REAL, DISCORD_BOT_TOKEN_REAL, SLACK_BOT_TOKEN_REAL, SLACK_APP_TOKEN_REAL, and Docker Hub credentials are not exposed.

Dispatch hint

Workflow: .github/workflows/nightly-e2e.yaml
jobs input: messaging-providers-e2e

github-actions · 2026-06-05T20:07:16Z

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

None. Changed files affect legacy/non-scenario E2E workflow validation only and do not touch test/e2e-scenario/, .github/workflows/e2e-scenarios*.yaml, scenario metadata, expected states, validation suites, or scenario runtime code.

Optional scenario E2E

None.

Relevant changed files

None.

github-actions · 2026-06-05T20:08:58Z

PR Review Advisor

Findings: 0 needs attention, 1 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 0 still apply, 1 new item found

Review findings

🛠️ Needs attention

None.

🔎 Worth checking

Runtime validation would improve confidence in the GitHub Actions secret gate (.github/workflows/nightly-e2e.yaml:363): The changed secret boundary depends on GitHub Actions expression evaluation and reusable-workflow boolean/secret propagation. The PR adds useful static contract tests for the YAML shape, but it does not prove at runtime that a workflow_dispatch run with a non-empty target_ref receives empty live messaging environment variables while trusted runs still receive configured live secrets.
- Recommendation: Consider adding or identifying a targeted runtime validation for the three relevant cases: workflow_dispatch with non-empty target_ref, workflow_dispatch with empty target_ref, and scheduled/non-dispatch execution.
- Evidence: messaging_live_secrets and all live messaging secret mappings now use the predicate `github.event_name != 'workflow_dispatch' || inputs.target_ref == ''`; tests assert those literal expressions in `test/e2e-script-workflow.test.ts` and `test/validate-e2e-coverage.test.ts`, but the diff does not add a runtime truth-table check.

🌱 Nice ideas

None.

Consider writing more tests for

**Runtime validation** — Verify workflow_dispatch messaging-providers-e2e with a non-empty target_ref receives empty TELEGRAM_BOT_TOKEN_REAL, TELEGRAM_CHAT_ID_E2E, DISCORD_BOT_TOKEN_REAL, DISCORD_CHANNEL_ID_E2E, SLACK_BOT_TOKEN_REAL, SLACK_APP_TOKEN_REAL, and SLACK_CHANNEL_ID_E2E environment values.. The static tests are appropriate for YAML contract drift, but this is a workflow/infrastructure secret boundary where runtime expression evaluation and reusable-workflow propagation are the remaining confidence gap.
**Runtime validation** — Verify workflow_dispatch messaging-providers-e2e with an empty target_ref preserves configured live messaging environment values when messaging_live_secrets evaluates true.. The static tests are appropriate for YAML contract drift, but this is a workflow/infrastructure secret boundary where runtime expression evaluation and reusable-workflow propagation are the remaining confidence gap.
**Runtime validation** — Verify scheduled messaging-providers-e2e preserves configured live messaging environment values when messaging_live_secrets evaluates true.. The static tests are appropriate for YAML contract drift, but this is a workflow/infrastructure secret boundary where runtime expression evaluation and reusable-workflow propagation are the remaining confidence gap.
**Runtime validation** — Add a trusted-ref predicate truth-table check covering workflow_dispatch plus target_ref, workflow_dispatch plus empty target_ref, and a non-workflow_dispatch event.. The static tests are appropriate for YAML contract drift, but this is a workflow/infrastructure secret boundary where runtime expression evaluation and reusable-workflow propagation are the remaining confidence gap.
**Runtime validation would improve confidence in the GitHub Actions secret gate** — Consider adding or identifying a targeted runtime validation for the three relevant cases: workflow_dispatch with non-empty target_ref, workflow_dispatch with empty target_ref, and scheduled/non-dispatch execution.

Since last review details

Current findings:

Runtime validation would improve confidence in the GitHub Actions secret gate (.github/workflows/nightly-e2e.yaml:363): The changed secret boundary depends on GitHub Actions expression evaluation and reusable-workflow boolean/secret propagation. The PR adds useful static contract tests for the YAML shape, but it does not prove at runtime that a workflow_dispatch run with a non-empty target_ref receives empty live messaging environment variables while trusted runs still receive configured live secrets.
- Recommendation: Consider adding or identifying a targeted runtime validation for the three relevant cases: workflow_dispatch with non-empty target_ref, workflow_dispatch with empty target_ref, and scheduled/non-dispatch execution.
- Evidence: messaging_live_secrets and all live messaging secret mappings now use the predicate `github.event_name != 'workflow_dispatch' || inputs.target_ref == ''`; tests assert those literal expressions in `test/e2e-script-workflow.test.ts` and `test/validate-e2e-coverage.test.ts`, but the diff does not add a runtime truth-table check.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

github-actions · 2026-06-05T20:23:24Z

Selective E2E Results — ✅ All requested jobs passed

Run: 27037445257
Target ref: 53c848a43553d75aa7f28409af4e572d22022bfd
Workflow ref: main
Requested jobs: messaging-providers-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job	Result
messaging-providers-e2e	✅ success

github-actions · 2026-06-05T20:57:13Z

Selective E2E Results — ✅ All requested jobs passed

Run: 27038983770
Target ref: 6d077ae2aead7fcddaa9b5172ba11c5a02e2000e
Workflow ref: main
Requested jobs: messaging-providers-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job	Result
messaging-providers-e2e	✅ success

ci(e2e): withhold live messaging secrets from target refs

53c848a

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv added security Potential vulnerability, unsafe behavior, or access risk area: security Security controls, permissions, secrets, or hardening labels Jun 5, 2026

cv self-assigned this Jun 5, 2026

cv added the v0.0.61 Release target label Jun 5, 2026

cv requested review from cjagwani, ericksoa and prekshivyas June 5, 2026 20:22

github-actions Bot mentioned this pull request Jun 5, 2026

feat(ci): polish nightly scorecard Slack rendering #4783

Merged

12 tasks

Merge branch 'main' into fix/e2e-live-messaging-secret-target-ref

6d077ae

cv added v0.0.60 Release target and removed v0.0.61 Release target labels Jun 5, 2026

cv merged commit 2220e78 into main Jun 5, 2026
28 checks passed

cv deleted the fix/e2e-live-messaging-secret-target-ref branch June 5, 2026 20:56

coderabbitai Bot mentioned this pull request Jun 8, 2026

fix(e2e): gate scenario fan-out by onboarding+secret support contract #4978

Closed

wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026

Conversation

cv commented Jun 5, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Type of Change

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Suggested labels

Poem

Uh oh!

github-actions Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

github-actions Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Scenario Advisor Recommendation

E2E Scenario Advisor

Required scenario E2E

Optional scenario E2E

Relevant changed files

Uh oh!

github-actions Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor

🛠️ Needs attention

🔎 Worth checking

🌱 Nice ideas

Uh oh!

github-actions Bot commented Jun 5, 2026

Selective E2E Results — ✅ All requested jobs passed

Uh oh!

Uh oh!

github-actions Bot commented Jun 5, 2026

Selective E2E Results — ✅ All requested jobs passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cv commented Jun 5, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 5, 2026 •

edited

Loading

github-actions Bot commented Jun 5, 2026 •

edited

Loading

github-actions Bot commented Jun 5, 2026 •

edited

Loading

github-actions Bot commented Jun 5, 2026 •

edited

Loading