fix(policy): drop phantom /usr/bin/gh from github preset binaries (#2179) by prekshivyas · Pull Request #3377 · NVIDIA/NemoClaw

prekshivyas · 2026-05-12T00:41:54Z

Summary

Fixes #2179. The github policy preset whitelisted /usr/bin/gh in its binaries: list (and advertised it in the (git, gh) description), but the sandbox base image only apt-installs git, not gh. Users selecting ● github and running gh api … hit bash: gh: command not found.

This is the 4th instance of the "phantom binary in preset whitelist" pattern (brew × 2, docker × 1, github × 1 — see issue body for the bug cross-references).

Approach

Going with the reporter's recommended option (b): drop /usr/bin/gh from the preset binary list and from the description text. The preset surface now honestly reflects what's actually usable in the shipped sandbox image.

The fuller fix (apt-install gh in the sandbox base image) trades size + supply-chain surface for the added capability and is intentionally out of scope. Users who want gh can still install it themselves inside the sandbox.

Changes

nemoclaw-blueprint/policies/presets/github.yaml — drop /usr/bin/gh binary entry; update description from "(git, gh)" → "(git)"; update header comment to explain why gh was dropped (pointer to [All Platforms][Policy] 'github' policy preset whitelists binary /usr/bin/gh but gh CLI is not installed in the sandbox image #2179).
agents/hermes/policy-additions.yaml — same drop in Hermes' github policy (Hermes inherits the same phantom-binary issue), with an inline comment pointing at [All Platforms][Policy] 'github' policy preset whitelists binary /usr/bin/gh but gh CLI is not installed in the sandbox image #2179.
docs/security/best-practices.md — two lines that mentioned gh as an example or available binary updated to reflect the actual preset binary set.
.agents/skills/nemoclaw-user-configure-security/references/best-practices.md — regenerated via scripts/docs-to-skills.py to match the docs change.

Verification

Empirically confirmed against v0.0.38 (current latest) on a fresh Brev box. Onboarded a sandbox, applied ● github, verified inside the sandbox:

/usr/bin/gh does not exist (only /usr/bin/git is present)
which gh returns nothing
gh --version → bash: gh: command not found
git (the other preset binary) works as expected

After this PR, the preset surface matches reality.

Test plan

CI passes
Onboard a sandbox, apply ● github preset, confirm nemoclaw <name> policy-list shows the updated description (git) and git continues to work through the preset
Verify the regenerated skill file stays in sync (CI verify docs-to-skills check)

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

Summary by CodeRabbit

Documentation
- Clarified GitHub access guidance and examples to state only git is available for GitHub.com interactions and updated preset descriptions to reflect least-privilege access.
Bug Fixes
- Removed a phantom gh binary from sandbox network policies so allowed tools match the shipped base image.
Tests
- Added regression tests to assert presets and policies reference git-only binaries and descriptions.

) The `github` preset's `binaries:` list whitelisted both `/usr/bin/gh` and `/usr/bin/git`, and the preset description advertised "(git, gh)". But the sandbox base image (nemoclaw-blueprint/Dockerfile.base for OpenClaw, agents/hermes/Dockerfile.base for Hermes) only apt-installs `git`, not `gh`. Users selecting the `github` preset and running `gh api …` inside the sandbox hit `bash: gh: command not found`. This is the 4th instance of the same "phantom binary in preset whitelist" pattern (brew x2, docker x1, github x1 — see #2179 body for cross-references). Cheapest fix per the reporter's option (b): drop `/usr/bin/gh` from the preset binary list and from the description text, so the preset surface honestly reflects what's shipped. The fuller fix (apt-install gh in the sandbox base image) trades size + supply-chain surface for the added capability and is intentionally out of scope here — users who want `gh` can still install it themselves inside the sandbox. Changes: - nemoclaw-blueprint/policies/presets/github.yaml — drop /usr/bin/gh binary entry, update description from "(git, gh)" to "(git)", update the header comment to explain why gh was dropped. - agents/hermes/policy-additions.yaml — same drop in Hermes' github policy with an inline comment pointing at #2179. - docs/security/best-practices.md — two lines that mentioned `gh` as an example or available binary updated to reflect the actual preset binary set. - .agents/skills/nemoclaw-user-configure-security/references/best-practices.md regenerated via scripts/docs-to-skills.py to match. Closes #2179. Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

copy-pr-bot · 2026-05-12T00:41:58Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2026-05-12T00:42:05Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 36423726-c1d2-462e-bcbe-3bab1eae80e5

📥 Commits

Reviewing files that changed from the base of the PR and between 781c6c4 and cb55343.

📒 Files selected for processing (4)

.agents/skills/nemoclaw-user-manage-policy/references/integration-policy-examples.md
docs/network-policy/integration-policy-examples.md
test/policies.test.ts
test/validate-blueprint.test.ts

📝 Walkthrough

Walkthrough

This PR removes /usr/bin/gh from the github preset and Hermes policy, updates preset description and related documentation to advertise only /usr/bin/git, and adds tests asserting the preset/policy are git-only.

Changes

GitHub Policy Preset Cleanup

Layer / File(s)	Summary
Preset comments and description `nemoclaw-blueprint/policies/presets/github.yaml`	Preset header comments updated to note `gh` is not present in the base image; `preset.description` changed to remove `(gh)`.
Preset binaries whitelist `nemoclaw-blueprint/policies/presets/github.yaml`	Removed `/usr/bin/gh` from the preset `binaries` allowlist; `/usr/bin/git` retained.
Hermes network policy `agents/hermes/policy-additions.yaml`	Hermes `github` policy `binaries` list updated to remove `/usr/bin/gh` and add inline comment explaining `gh` is not installed in the base image.
User documentation `docs/security/best-practices.md`, `.agents/skills/nemoclaw-user-configure-security/references/best-practices.md`, `.agents/skills/nemoclaw-user-manage-policy/references/integration-policy-examples.md`, `docs/network-policy/integration-policy-examples.md`	Examples and the Policy Presets table/guidance updated to describe GitHub access via `/usr/bin/git` only; `gh` mentions removed.
Tests `test/policies.test.ts`, `test/validate-blueprint.test.ts`	Added tests asserting Hermes policy and blueprint preset binaries lists and descriptions do not include `/usr/bin/gh` and resolve to `/usr/bin/git`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I hopped through presets, sniffed the build,
A ghostly gh lay where it should not be tilled.
I pulled the phantom right from the list,
Now only git runs—no more mist. 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(policy): drop phantom /usr/bin/gh from github preset binaries (`#2179`)' accurately describes the main change: removing the non-existent /usr/bin/gh binary from the github policy preset across multiple files.
Linked Issues check	✅ Passed	The PR fully implements option (b) from issue `#2179`: removes /usr/bin/gh from the github preset binaries, updates descriptions from '(git, gh)' to '(git)' across all relevant files, and adds test coverage validating the expected behavior.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to removing /usr/bin/gh from the github policy preset and updating associated documentation and tests; no unrelated modifications detected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/2179-drop-phantom-gh-from-github-preset

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-05-12T00:42:54Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: network-policy-e2e, hermes-e2e, test-e2e-sandbox

Dispatch hint: network-policy-e2e,hermes-e2e

Workflow run

Full advisor summary

Pi Semantic E2E Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. The PR removes a phantom binary (/usr/bin/gh) that is not installed in either the OpenClaw or Hermes sandbox base images, so no real user flow could have depended on it. The behavioral surface is already covered by unit-level regression tests in test/policies.test.ts ([All Platforms][Policy] 'github' policy preset whitelists binary /usr/bin/gh but gh CLI is not installed in the sandbox image #2179) and test/validate-blueprint.test.ts ([All Platforms][Policy] 'github' policy preset whitelists binary /usr/bin/gh but gh CLI is not installed in the sandbox image #2179, [All Platform] GitHub policy + binary is injected into sandbox runtime policy without user opt-in #1583), which run on every PR via the standard vitest suite invoked by the pr.yaml basic-checks action. The remaining changes are documentation-only. No required merge-blocking E2E is justified; optional E2E runs are listed for extra confidence.

Optional E2E

network-policy-e2e (medium): Exercises preset loading and live policy-add flow (TC-NET-03/04 apply presets like slack/jira/pypi non-interactively and via expect). Provides end-to-end confidence that the updated github.yaml preset still parses, loads, and is applyable through the user-facing nemoclaw policy-add surface, even though the unit tests already assert the YAML shape.
hermes-e2e (medium): Hermes onboarding loads agents/hermes/policy-additions.yaml, which this PR edits. Running the hermes E2E confirms the policy still parses and the Hermes sandbox reaches github.com via git after removing the phantom /usr/bin/gh entry. Optional because gh is not installed in the Hermes base image, so no runtime flow could have depended on it.
test-e2e-sandbox (low): Sandbox smoke test that loads the production image and fixtures. Cheap confidence check that preset YAML changes did not break sandbox bring-up; not directly policy-focused.

New E2E recommendations

network-policy-presets (low): There is no dedicated E2E that applies the github preset and verifies that github.com / api.github.com are reachable from the git binary (and only from git) while unapproved binaries are blocked. test-network-policy.sh exercises slack/pypi/jira flows but not github. Given that github was the exact preset whose binary list changed here and historically ([All Platform] GitHub policy + binary is injected into sandbox runtime policy without user opt-in #1583) was silently included in the baseline, a focused E2E would catch future drift (e.g. someone re-adding /usr/bin/gh or loosening the binary scope).
- Suggested test: Add a TC-NET case (or new test/e2e/test-github-preset.sh) that: (1) onboards a restricted sandbox, (2) confirms github.com is blocked, (3) runs nemoclaw <name> policy-add github --yes, (4) confirms git ls-remote https://github.com/... from the sandbox succeeds, (5) confirms that a non-git binary (e.g. curl/wget) is still denied to github.com by the binary-scoped rule.

Dispatch hint

Workflow: nightly-e2e.yaml
jobs input: network-policy-e2e,hermes-e2e

coderabbitai

🧹 Nitpick comments (2)

nemoclaw-blueprint/policies/presets/github.yaml (1)
25-37: Recommend running policy E2E before merge.

Because this changes preset egress/binary constraints, run the network policy E2E job to validate behavior in sandbox flows.

As per coding guidelines "nemoclaw-blueprint/policies/** ... E2E test recommendation: network-policy-e2e."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@nemoclaw-blueprint/policies/presets/github.yaml` around lines 25 - 37, This
change modifies the network_policies.github preset (network_policies -> github)
by altering endpoints and binaries; before merging, run the network-policy-e2e
job to validate the egress and binary constraints for the github preset (verify
endpoints: github.com and api.github.com on port 443 and the binaries list
including /usr/bin/git) and attach the resulting report to the PR so CI confirms
sandbox flows behave as expected.
agents/hermes/policy-additions.yaml (1)
69-84: Please run the Hermes E2E set for this policy-surface change.

This touches Hermes onboarding/policy behavior, so a targeted E2E sweep is worth running pre-merge.

As per coding guidelines "agents/hermes/** ... E2E test recommendation: hermes-e2e, hermes-inference-switch-e2e, hermes-discord-e2e, hermes-slack-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@agents/hermes/policy-additions.yaml` around lines 69 - 84, This change
updates Hermes policy surface (agents/hermes/policy-additions.yaml) and requires
running the Hermes E2E suite before merging; please trigger the full targeted
E2E sweep: run hermes-e2e, hermes-inference-switch-e2e, hermes-discord-e2e,
hermes-slack-e2e, rebuild-hermes-e2e and rebuild-hermes-stale-base-e2e (or the
CI job group that maps to those names) and report any failures tied to the
policy changes so they can be addressed before merge.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@agents/hermes/policy-additions.yaml`:
- Around line 69-84: This change updates Hermes policy surface
(agents/hermes/policy-additions.yaml) and requires running the Hermes E2E suite
before merging; please trigger the full targeted E2E sweep: run hermes-e2e,
hermes-inference-switch-e2e, hermes-discord-e2e, hermes-slack-e2e,
rebuild-hermes-e2e and rebuild-hermes-stale-base-e2e (or the CI job group that
maps to those names) and report any failures tied to the policy changes so they
can be addressed before merge.

In `@nemoclaw-blueprint/policies/presets/github.yaml`:
- Around line 25-37: This change modifies the network_policies.github preset
(network_policies -> github) by altering endpoints and binaries; before merging,
run the network-policy-e2e job to validate the egress and binary constraints for
the github preset (verify endpoints: github.com and api.github.com on port 443
and the binaries list including /usr/bin/git) and attach the resulting report to
the PR so CI confirms sandbox flows behave as expected.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 42e9c30b-0420-43dc-833b-4f14d4698430

📥 Commits

Reviewing files that changed from the base of the PR and between 0007f4d and bc3da7c.

📒 Files selected for processing (4)

.agents/skills/nemoclaw-user-configure-security/references/best-practices.md
agents/hermes/policy-additions.yaml
docs/security/best-practices.md
nemoclaw-blueprint/policies/presets/github.yaml

github-actions · 2026-05-12T16:37:47Z

Selective E2E Results — ✅ All requested jobs passed

Run: 25747668898
Branch: fix/2179-drop-phantom-gh-from-github-preset
Requested jobs: network-policy-e2e,hermes-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job	Result
hermes-e2e	✅ success
network-policy-e2e	✅ success

jyaunches · 2026-05-12T17:03:34Z

Targeted E2E validation requested by the E2E Advisor has passed.

Run: https://github.com/NVIDIA/NemoClaw/actions/runs/25747668898

Passed targeted jobs: network-policy-e2e, hermes-e2e

PR review can proceed with that E2E signal in mind.

ericksoa

Thanks, this is the right behavioral direction for #2179: the GitHub preset and Hermes policy should not advertise or whitelist a gh binary that the shipped images do not install.

I think this needs two small follow-ups before merge:

docs/network-policy/integration-policy-examples.md and the generated skill mirror at .agents/skills/nemoclaw-user-manage-policy/references/integration-policy-examples.md still say the github preset is for GitHub API, Git, or gh access. Please update those to GitHub API/Git only so the user-facing docs do not keep the phantom gh claim alive.
Please add focused regression coverage for #2179. The existing GitHub preset test only proves the preset exists and parses; it does not assert that the preset description is now (git) or that /usr/bin/gh stays out of both the built-in github preset and agents/hermes/policy-additions.yaml.

I validated a local patch for those updates with npm run build:cli, npm run typecheck:cli, npx vitest run test/validate-blueprint.test.ts test/policies.test.ts, python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix nemoclaw-user --dry-run, npm run source-shape:check, and git diff --check.

Retracting review posted by mistake.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

ericksoa

Reviewed policy and docs/test follow-up on current head cb55343. The PR removes the phantom gh binary from the GitHub preset/Hermes policy, keeps git access intact, and now has focused regression coverage plus aligned docs. All visible checks and CodeRabbit are green.

Merge branch 'main' into fix/2179-drop-phantom-gh-from-github-preset

bc3da7c

prekshivyas requested a review from ericksoa May 12, 2026 00:42

prekshivyas self-assigned this May 12, 2026

prekshivyas added the v0.0.39 label May 12, 2026

coderabbitai Bot reviewed May 12, 2026

View reviewed changes

Merge branch 'main' into fix/2179-drop-phantom-gh-from-github-preset

04faa01

cv added v0.0.40 and removed v0.0.39 labels May 12, 2026

Merge branch 'main' into fix/2179-drop-phantom-gh-from-github-preset

781c6c4

prekshivyas requested a review from jyaunches May 12, 2026 16:20

wscurran added fix labels May 12, 2026

Merge branch 'main' into fix/2179-drop-phantom-gh-from-github-preset

58f9b28

ericksoa previously requested changes May 12, 2026

View reviewed changes

prekshivyas and others added 2 commits May 12, 2026 11:33

Merge branch 'main' into fix/2179-drop-phantom-gh-from-github-preset

4acf41b

test(policy): cover github preset gh removal

cb55343

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

ericksoa approved these changes May 12, 2026

View reviewed changes

ericksoa merged commit d577e6f into main May 12, 2026
23 checks passed

ericksoa deleted the fix/2179-drop-phantom-gh-from-github-preset branch May 12, 2026 19:24

latenighthackathon mentioned this pull request May 20, 2026

[Sandbox] brew policy preset cannot bootstrap Homebrew: sandbox lacks filesystem write to /home/linuxbrew #3913

Closed

wscurran added the area: integrations Third-party service integration behavior label Jun 3, 2026

Conversation

prekshivyas commented May 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Approach

Changes

Verification

Test plan

Summary by CodeRabbit

Uh oh!

copy-pr-bot Bot commented May 12, 2026

Uh oh!

coderabbitai Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

github-actions Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

Pi Semantic E2E Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 12, 2026

Selective E2E Results — ✅ All requested jobs passed

Uh oh!

jyaunches commented May 12, 2026

Uh oh!

ericksoa left a comment

Choose a reason for hiding this comment

Uh oh!

ericksoa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

prekshivyas commented May 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 12, 2026 •

edited

Loading

github-actions Bot commented May 12, 2026 •

edited

Loading