fix(hermes): avoid rc rewrites after capability drop#3914
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughRefactors Hermes startup to emit a guarded /tmp/nemoclaw-proxy-env.sh, adds a Bash security-posture assertions module and optional E2E phases, adds unit tests for the runtime env bootstrap, updates Brev E2E robustness and stdio capture, and wires two nightly E2E jobs into CI reporting. ChangesSecurity posture testing and runtime environment refactor
Brev E2E resilience and stdio capture
Sequence DiagramsequenceDiagram
participant Nightly as Nightly Workflow
participant E2E as E2E Test Runner
participant Hermes as agents/hermes/start.sh
participant Sandbox as Sandbox (PID 1)
participant Assertions as security_posture_assertions.sh
Nightly->>E2E: dispatch job with NEMOCLAW_E2E_SECURITY_POSTURE=1
E2E->>Hermes: invoke start.sh (write_runtime_shell_env)
Hermes->>Sandbox: emit /tmp/nemoclaw-proxy-env.sh (guarded)
Hermes->>Sandbox: lock /sandbox/.bashrc and /sandbox/.profile (mode 444)
E2E->>Assertions: run security_posture_assertions_run(sandbox, agent)
Assertions->>Sandbox: inspect /proc/1 (Uid, CapBnd, CapEff, NoNewPrivs)
Assertions->>Sandbox: verify /sandbox/.bashrc & /sandbox/.profile (root:root 444, no inline guard)
Assertions->>Sandbox: verify /tmp/nemoclaw-proxy-env.sh (guard markers, guard function)
Assertions->>Sandbox: source proxy env and probe guard (expect configure-block message)
Assertions-->>E2E: return posture check results
E2E-->>Nightly: report job pass/fail
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Comment |
PR Review AdvisorRecommendation: info only This is an automated advisory review. A human maintainer must make the final merge decision. Limitations: Advisor execution failed: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt Full advisor summaryPR Review AdvisorBase: PR review advisor failed: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt Gate status
🔴 Blockers
🟡 Warnings
🔵 Suggestions
Acceptance coverage
Security review
Test / E2E status
✅ What looks good
Review completeness
|
E2E Advisor RecommendationRequired E2E: Dispatch hint: Auto-dispatched E2E: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/nightly-e2e.yaml (1)
806-809: 💤 Low valueOptional: Add
persist-credentials: falseto checkout steps.Static analysis flagged these checkout steps for not setting
persist-credentials: false. While many existing jobs in this workflow also omit it, adding it prevents Git credentials from persisting into artifacts on failure. Low priority since the workflow only runs on schedule/manual dispatch.🔒 Proposed fix
- name: Checkout uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: ref: ${{ inputs.target_ref || github.ref }} + persist-credentials: falseApply to both
openclaw-onboard-security-posture-e2e(line 806) andhermes-onboard-security-posture-e2e(line 844) checkout steps.Also applies to: 844-847
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/nightly-e2e.yaml around lines 806 - 809, The checkout steps using actions/checkout (the steps named "Checkout" in the openclaw-onboard-security-posture-e2e and hermes-onboard-security-posture-e2e jobs) do not set persist-credentials and should be hardened; update each checkout step's with block to include persist-credentials: false so Git credentials are not persisted into artifacts on failure, ensuring the key remains alongside the existing ref: ${{ inputs.target_ref || github.ref }} entry.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/nightly-e2e.yaml:
- Around line 806-809: The checkout steps using actions/checkout (the steps
named "Checkout" in the openclaw-onboard-security-posture-e2e and
hermes-onboard-security-posture-e2e jobs) do not set persist-credentials and
should be hardened; update each checkout step's with block to include
persist-credentials: false so Git credentials are not persisted into artifacts
on failure, ensuring the key remains alongside the existing ref: ${{
inputs.target_ref || github.ref }} entry.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: bb7af2ab-1e2e-4ed7-9bb0-9addf07f8e67
📒 Files selected for processing (6)
.github/workflows/nightly-e2e.yamlagents/hermes/start.shtest/e2e/lib/security-posture-assertions.shtest/e2e/test-full-e2e.shtest/e2e/test-hermes-e2e.shtest/hermes-start.test.ts
Selective E2E Results — ❌ Some jobs failedRun: 26180287332
|
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/nightly-e2e.yaml (1)
797-863:⚠️ Potential issue | 🟠 MajorAdd missing
.coderabbit.yamlpath_instructions entries for the new security-posture E2E jobs.The PR adds two new E2E jobs (
openclaw-onboard-security-posture-e2eandhermes-onboard-security-posture-e2e) but does not include corresponding path_instructions entries in.coderabbit.yaml. Per the coding guidelines, new E2E jobs require matching path_instructions for their source paths.Missing or incomplete entries:
- No entry for
test/e2e/test-full-e2e.sh(run by openclaw-onboard-security-posture-e2e)- No entry for
test/e2e/test-hermes-e2e.sh(run by hermes-onboard-security-posture-e2e)agents/hermes/**entry exists but does not reference the newhermes-onboard-security-posture-e2ejob in its recommendation listAdd these entries to
.coderabbit.yamlwith instructions pointing reviewers to run the new security-posture lanes when those paths are modified.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/nightly-e2e.yaml around lines 797 - 863, The .coderabbit.yaml is missing path_instructions for the two new E2E jobs: add entries for the scripts "test/e2e/test-full-e2e.sh" (used by openclaw-onboard-security-posture-e2e) and "test/e2e/test-hermes-e2e.sh" (used by hermes-onboard-security-posture-e2e) that instruct reviewers to run the corresponding security-posture lanes, and update the existing agents/hermes/** path_instructions recommendation list to include the new hermes-onboard-security-posture-e2e job name so changes under agents/hermes/** surface the new lane to reviewers.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/lib/security-posture-assertions.sh`:
- Around line 157-160: The check currently treats the case that echoes
"NON_ROOT_PROXY_ENV_OWNER" as acceptable; modify the in-sandbox validation (the
shell snippet executed by security_posture_sandbox_exec that inspects
/tmp/nemoclaw-proxy-env.sh) so that when owner matches "$current_owner" it sets
bad=1 and exits non-zero (i.e., treat NON_ROOT_PROXY_ENV_OWNER as a failure just
like BAD_PROXY_ENV_OWNER). Locate the case block that emits
NON_ROOT_PROXY_ENV_OWNER in the long command string (it references variables f,
meta, mode, owner, current_owner and echoes NON_ROOT_PROXY_ENV_OWNER) and add
bad=1 (and/or an exit non-zero) in that branch so the overall check fails when
the file is not root-owned.
---
Outside diff comments:
In @.github/workflows/nightly-e2e.yaml:
- Around line 797-863: The .coderabbit.yaml is missing path_instructions for the
two new E2E jobs: add entries for the scripts "test/e2e/test-full-e2e.sh" (used
by openclaw-onboard-security-posture-e2e) and "test/e2e/test-hermes-e2e.sh"
(used by hermes-onboard-security-posture-e2e) that instruct reviewers to run the
corresponding security-posture lanes, and update the existing agents/hermes/**
path_instructions recommendation list to include the new
hermes-onboard-security-posture-e2e job name so changes under agents/hermes/**
surface the new lane to reviewers.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e487dee4-6ccf-4d22-aa99-6ae72678a353
📒 Files selected for processing (4)
.github/workflows/nightly-e2e.yamlagents/hermes/Dockerfile.baseagents/hermes/start.shtest/e2e/lib/security-posture-assertions.sh
✅ Files skipped from review due to trivial changes (1)
- agents/hermes/Dockerfile.base
Selective E2E Results — ✅ All requested jobs passedRun: 26180794094
|
Selective E2E Results — ❌ Some jobs failedRun: 26180718921
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/nightly-e2e.yaml (1)
797-802:⚠️ Potential issue | 🔴 CriticalAdd missing
path_instructionsentries in.coderabbit.yamlfor new E2E jobs.Per coding guidelines, new E2E jobs must have corresponding
path_instructionsentries in.coderabbit.yaml. The two new security posture jobs are missing entries:
openclaw-onboard-security-posture-e2erunstest/e2e/test-full-e2e.sh(not in.coderabbit.yaml)hermes-onboard-security-posture-e2erunstest/e2e/test-hermes-e2e.sh(not in.coderabbit.yaml)Also add
test/e2e/lib/security-posture-assertions.shto.coderabbit.yamlsince both scripts source it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/nightly-e2e.yaml around lines 797 - 802, Add missing path_instructions in .coderabbit.yaml for the two new E2E jobs and their shared helper: add entries mapping openclaw-onboard-security-posture-e2e to test/e2e/test-full-e2e.sh and hermes-onboard-security-posture-e2e to test/e2e/test-hermes-e2e.sh, and also add test/e2e/lib/security-posture-assertions.sh (since both scripts source it); ensure the keys match the job names openclaw-onboard-security-posture-e2e and hermes-onboard-security-posture-e2e and that the path_instructions format matches existing entries in .coderabbit.yaml.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/nightly-e2e.yaml:
- Around line 797-802: Add missing path_instructions in .coderabbit.yaml for the
two new E2E jobs and their shared helper: add entries mapping
openclaw-onboard-security-posture-e2e to test/e2e/test-full-e2e.sh and
hermes-onboard-security-posture-e2e to test/e2e/test-hermes-e2e.sh, and also add
test/e2e/lib/security-posture-assertions.sh (since both scripts source it);
ensure the keys match the job names openclaw-onboard-security-posture-e2e and
hermes-onboard-security-posture-e2e and that the path_instructions format
matches existing entries in .coderabbit.yaml.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: af84991d-df50-45ee-8111-d1cb82fe9c37
📒 Files selected for processing (1)
.github/workflows/nightly-e2e.yaml
Selective E2E Results — ❌ Some jobs failedRun: 26180979975
|
Selective E2E Results — ✅ All requested jobs passedRun: 26181521660
|
Selective E2E Results — ✅ All requested jobs passedRun: 26182053633
|
Selective E2E Results — ✅ All requested jobs passedRun: 26182621536
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/hermes-start.test.ts`:
- Around line 155-156: The test in test/hermes-start.test.ts is only checking a
prefix so expect(run.envFileContent).toContain("export SSL_CERT_FILE=") will
pass when the variable is non-empty; change the assertion to verify an empty
assignment exactly by using a line-anchored check (e.g., a multiline regex
match) against the literal "export SSL_CERT_FILE=" on its own line so
run.envFileContent must contain an exact empty assignment; update the assertion
that currently references run.envFileContent and "export SSL_CERT_FILE="
accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 20270730-f9b1-4793-ba66-786567169223
📒 Files selected for processing (1)
test/hermes-start.test.ts
Selective E2E Results — ✅ All requested jobs passedRun: 26182579369
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/hermes-start.test.ts`:
- Around line 155-156: The test currently checks only the filename suffix;
instead assert that the exact generated CA path is propagated by using the test
helper's run.caFile value. Replace the loose assertion
(expect(run.envFileContent).toContain("/proxy\\ ca.pem")) with an assertion that
the env content contains the full path from run.caFile (e.g.
expect(run.envFileContent).toContain(run.caFile) or
expect(run.envFileContent).toContain(`export SSL_CERT_FILE=${run.caFile}`)),
keeping references to run, run.envFileContent and run.caFile to locate and
validate the real path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: cf886755-cc83-4e3e-9f33-f665e67d7b29
📒 Files selected for processing (1)
test/hermes-start.test.ts
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.coderabbit.yaml (1)
453-500: ⚡ Quick winAdd the OpenClaw security-posture lane to the shared entrypoint mapping too.
These new entries cover the helper and top-level E2E drivers, but reviewers touching
scripts/nemoclaw-start.shorscripts/lib/sandbox-init.shstill will not get a recommendation foropenclaw-onboard-security-posture-e2e, even though that lane exercises the non-root boot posture those files own.As per coding guidelines, "If a new E2E job is added, verify corresponding
path_instructionsentries exist in.coderabbit.yamlfor the source files it covers."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.coderabbit.yaml around lines 453 - 500, The shared entrypoint mapping in .coderabbit.yaml is missing the openclaw-onboard-security-posture-e2e recommendation for source files that exercise the non-root boot posture (e.g., scripts/nemoclaw-start.sh and scripts/lib/sandbox-init.sh); add an entry that maps those script paths to include the openclaw-onboard-security-posture-e2e lane just like the existing entries for test/e2e/lib/security-posture-assertions.sh, test/e2e/test-full-e2e.sh, and test/e2e/test-hermes-e2e.sh so reviewers touching nemoclaw-start.sh or sandbox-init.sh will get the proper E2E recommendation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.coderabbit.yaml:
- Around line 453-500: The shared entrypoint mapping in .coderabbit.yaml is
missing the openclaw-onboard-security-posture-e2e recommendation for source
files that exercise the non-root boot posture (e.g., scripts/nemoclaw-start.sh
and scripts/lib/sandbox-init.sh); add an entry that maps those script paths to
include the openclaw-onboard-security-posture-e2e lane just like the existing
entries for test/e2e/lib/security-posture-assertions.sh,
test/e2e/test-full-e2e.sh, and test/e2e/test-hermes-e2e.sh so reviewers touching
nemoclaw-start.sh or sandbox-init.sh will get the proper E2E recommendation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0b70bc15-3e46-47ea-8c44-0e27b19b5ac4
📒 Files selected for processing (4)
.coderabbit.yaml.github/workflows/nightly-e2e.yamltest/e2e/lib/security-posture-assertions.shtest/hermes-start.test.ts
Selective E2E Results — ✅ All requested jobs passedRun: 26183406642
|
Selective E2E Results — ✅ All requested jobs passedRun: 26183311331
|
Selective E2E Results — ✅ All requested jobs passedRun: 26183684739
|
Selective E2E Results — ✅ All requested jobs passedRun: 26183971147
|
|
❌ Brev E2E (full): FAILED on branch |
|
❌ Brev E2E (full): FAILED on branch |
Selective E2E Results — ✅ All requested jobs passedRun: 26184964011
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/brev-e2e.test.ts`:
- Around line 170-172: The callback parameter for rawInstances.flatMap currently
has an implicit any; explicitly annotate it (e.g., change the callback to
rawInstances.flatMap((instance: unknown) => { ... }) ) so the compiler knows the
element type and the normalizeBrevInstance call receives a properly typed
argument; update any related callbacks in the same expression if present.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d22e12e9-1a89-4227-a86d-081e8819e454
📒 Files selected for processing (2)
test/e2e/brev-e2e.test.tstest/runner.test.ts
Selective E2E Results — ✅ All requested jobs passedRun: 26185995744
|
|
✅ Brev E2E (full): PASSED on branch |
Selective E2E Results — ✅ All requested jobs passedRun: 26187020907
|
|
✅ Brev E2E (full): PASSED on branch |
|
@cv could you please take a look for approval? The latest head is |
Selective E2E Results — ✅ All requested jobs passedRun: 26191367897
|
Summary
/tmp/nemoclaw-proxy-env.shinstead of rewriting/sandbox/.bashrcand/sandbox/.profileafterdrop_capabilities.Issues
Validation
bash -n agents/hermes/start.sh test/e2e/lib/security-posture-assertions.sh test/e2e/test-full-e2e.sh test/e2e/test-hermes-e2e.shnpm test -- test/validate-e2e-coverage.test.ts test/e2e-advisor-dispatch.test.tsnpm test -- test/hermes-start.test.ts test/sandbox-init.test.tsnpm run typecheck:cligit diff --check HEAD~1 HEADNightly proof plan
Dispatch only the new proof lanes from this PR branch:
openclaw-onboard-security-posture-e2ehermes-onboard-security-posture-e2eSummary by CodeRabbit
Tests
Chores
Documentation