Skip to content

docs(cli): sync commands.md + install.sh --help and add per-PR parity check (#3224)#3234

Merged
ericksoa merged 12 commits into
NVIDIA:mainfrom
Dongni-Yang:fix/docs-cli-parity-3224
May 9, 2026
Merged

docs(cli): sync commands.md + install.sh --help and add per-PR parity check (#3224)#3234
ericksoa merged 12 commits into
NVIDIA:mainfrom
Dongni-Yang:fix/docs-cli-parity-3224

Conversation

@Dongni-Yang

@Dongni-Yang Dongni-Yang commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Close the recurring CLI/docs drift surfaced by NV QA's vitest pipeline (TC5882289 fired on v0.0.31, .34, .35, .36) and prevent it from re-occurring.

  1. Sync the current drifts in docs/reference/commands.md and install.sh --help.
  2. Wire the check-docs.sh parity script into PR CI so future drift is caught before merge instead of overnight.
  3. Tighten the parity script itself so the kind of drift NemoClaw#3224 describes is actually flagged. The original --only-cli mode only caught command-level drift (a missing ### nemoclaw foo heading); flag-level drift, install.sh provider drift, and stdin races silently passed.

Locally negative-tested: removing --probe-only, --json, --yes, --gateway, routed, or anthropicCompatible is now reported by the script. Restoring each makes the check pass cleanly.

Related Issue

Fixes #3224

Changes

Docs sync

  • c5ce4614 docs(cli)nemoclaw onboard USAGE adds [--yes | -y]; nemoclaw list USAGE adds [--json]; nemoclaw <name> connect adds --probe-only; install.sh bootstrap_usage adds routed to NEMOCLAW_PROVIDER.
  • 853cb05e docs(cli) — split --probe-only description into prose per the one-sentence-per-line guideline (CodeRabbit nit).
  • 9477d6ba docs(cli) — close three more drift instances surfaced once flag-level parity actually iterated every command: nemoclaw setup and nemoclaw setup-spark (deprecated aliases that pass through all onboard flags) now list those flags inline; nemoclaw uninstall flag table adds --gateway <name>.

CI workflow

  • 51c5ac4e ci(docs) — new .github/workflows/docs-cli-parity-pr.yaml. Runs check-docs.sh --only-cli and --only-install on every PR touching src/**, bin/**, install.sh, scripts/install.sh, docs/reference/commands.md, package.json, package-lock.json, tsconfig.src.json, the parity script itself, or this workflow file. Same Node 22 + npm install --ignore-scripts + npm run build:cli recipe as the rest of PR CI.

Parity script tightening

  • c288aa2d test(docs) — added flag-level parity (phase 3) and --only-install mode that compares install.sh + scripts/install.sh NEMOCLAW_PROVIDER blocks against src/lib/onboard/providers.ts.
  • ab391834 fix(test) — flag check is now section-scoped (per ### \nemoclaw `heading) with word-boundary regex, so--yesdocumented under one command no longer hides drift in another, and--yesdoes not match inside--yes-i-accept-third-party-software`. Loop invocation switched to argv array (clears SC2086). Sandbox-name limitation documented inline. Addresses CodeRabbit "scope flag checks to the matching command section" feedback.
  • 61d03afe fix(test) — install provider check switched from grep -qF (substring) to grep -qxF against tokenized identifiers, so anthropic no longer falsely passes when only anthropicCompatible is present. Awk block-extractor boundary tightened so NEMOCLAW_POLICY_MODE no longer bleeds into the provider block (would have hidden custom drift). Addresses CodeRabbit "compare provider names as exact values, not substrings" feedback.
  • 3a3fbd94 fix(test) — redirect inner node ... --help stdin to /dev/null. The outer while read consumed $_tmp/help.txt via done < redirection, and node startup paths were reading a byte from that stdin, silently jumping the outer loop past debug, deploy, gc, list, onboard, etc. This was the most material bug — without it phase 3 only covered the alphabetic prefix of commands. Negative-tested.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • bash test/e2e/e2e-cloud-experimental/check-docs.sh --only-cli passes on the branch.
  • bash test/e2e/e2e-cloud-experimental/check-docs.sh --only-install passes on the branch.
  • Negative tests: removing --probe-only, --json, --yes, --gateway, routed, anthropicCompatible each cause the relevant phase to fail with a clear "not in section" or "absent from" message.
  • No secrets, API keys, or credentials committed.
  • Docs updated for user-facing behavior changes.

Test plan

  • bash install.sh --help | grep routed shows the new entry in the bootstrap NEMOCLAW_PROVIDER list.
  • bash scripts/install.sh --help (the local-payload path) was already current — confirmed.
  • YAML validates: python3 -c "import yaml; yaml.safe_load(open('.github/workflows/docs-cli-parity-pr.yaml'))".
  • PR CI run validates the new workflow triggers correctly on this PR (this PR touches install.sh and docs/reference/commands.md, so the new workflow itself should run).
  • NV QA's TC5882289 / TC5956310 vitest assertions reflagged on the branch — expected to drop from 2 failures to 0.

Known limitation

For sandbox-prefixed commands (nemoclaw <name> channels add, destroy, rebuild, doctor, skill install), --help is gated behind sandbox-name validation that requires a registered sandbox to bypass. On a CI runner with no sandbox they emit only an error and we skip the empty help text — flag drift on those specific commands is not caught by the per-PR check. The existing E2E suite (which runs against a real sandbox) covers them. Documented inline in check-docs.sh and tracked as a follow-up to provision a placeholder sandbox in CI or add a CLI mode that bypasses name validation for --help.


Signed-off-by: Dongni Yang dongniy@nvidia.com

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added --probe-only to verify SSH reachability without opening a shell.
    • Added "routed" as a supported provider option.
  • Documentation

    • Updated CLI command reference to reflect current flag syntax, onboarding options, and list command usage.
    • Deprecated setup aliases documented to accept onboarding flags.
  • Tests

    • Improved automated parity checks to verify command and flag alignment and install-provider coverage.
  • Chores

    • Added a PR pipeline to run docs/CLI/install parity checks on main-bound PRs.

Dongni-Yang and others added 2 commits May 8, 2026 13:10
…ace (NVIDIA#3224)

Three small drifts caught by NV QA's vitest pipeline against v0.0.36:

- docs/reference/commands.md: onboard USAGE was missing `[--yes | -y]`
  (added in PR NVIDIA#3066 via the oclif onboard adapter)
- docs/reference/commands.md: list USAGE was missing `[--json]` (the
  prose mentions --json, but the leading code-block USAGE line did not)
- docs/reference/commands.md: connect section never documented
  `--probe-only` despite top-level help advertising it
- install.sh bootstrap_usage was missing the `routed` provider key
  added by PR NVIDIA#2202 (model router); scripts/install.sh already lists it

Refs NVIDIA#3224. The recurring `commands.md` drift the issue calls out
(TC5882289 fired on v0.0.31, .34, .35, .36) is addressed in a follow-up
commit that wires the existing CLI/docs parity check into PR CI.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…VIDIA#3224)

The `check-docs.sh --only-cli` script already validates that every
`nemoclaw` subcommand listed by `--dump-commands` has a matching
`### \`nemoclaw …\`` heading in docs/reference/commands.md. Today it
only runs in nightly E2E (docs-validation-e2e), so per-version drift
(TC5882289 has flagged it on v0.0.31, .34, .35, .36) only surfaces
the morning after a release.

Add a lightweight PR workflow that runs the same check whenever a PR
touches the CLI surface (src/**, bin/**, install.sh, commands.md,
package.json/lock, tsconfig.src.json, or check-docs.sh itself).

The job runs npm install + build:cli + the parity check (~60-90 s
overhead per affected PR; most PRs aren't on the path filter and skip).

Note: this catches command-level drift, not flag-level. Extending the
check to compare per-command flags is the natural follow-up — see the
issue thread on NVIDIA#3224 for that proposal.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a CI workflow that runs CLI/docs parity checks, enhances the parity script with command- and flag-level checks plus install-provider verification, updates three command docs, and extends the installer help text to list routed.

Changes

CLI Documentation and Parity Verification

Layer / File(s) Summary
Workflow Trigger & Setup
.github/workflows/docs-cli-parity-pr.yaml
New Docs CLI Parity PR workflow triggers on PRs to main with path filters for CLI source, docs, install scripts, parity script, and workflow file.
Workflow Permissions & Concurrency
.github/workflows/docs-cli-parity-pr.yaml
Sets permissions: contents: read and a concurrency group that cancels superseded runs.
Workflow Job Execution
.github/workflows/docs-cli-parity-pr.yaml
Job on ubuntu-latest with 10-minute timeout: checkout, Node.js 22, npm install --ignore-scripts, npm run build:cli, then run parity checks (--only-cli, --only-install).
Parity Script Modes & Usage
test/e2e/e2e-cloud-experimental/check-docs.sh
Adds --only-install mode and related control variables; updates usage and argument parsing; requires at least one check mode.
CLI Parity: Command-Level Diff
test/e2e/e2e-cloud-experimental/check-docs.sh
Replaces prior mismatch exit with comm-based diffs comparing nemoclaw --dump-commands to commands.md headings.
CLI Parity: Flag-Level Checks
test/e2e/e2e-cloud-experimental/check-docs.sh
Adds phase that runs nemoclaw <command> --help, extracts long --... flags, scopes verification to the command’s docs section, and fails on missing flags.
Install Provider Parity Check
test/e2e/e2e-cloud-experimental/check-docs.sh
Adds run_install_check() to extract canonical providers from TS source and verify exact presence in both install.sh bootstrap help and scripts/install.sh payload help blocks.
Command Reference Updates
docs/reference/commands.md
nemoclaw onboard usage now includes --yes-i-accept-third-party-software; nemoclaw list shows nemoclaw list [--json]; nemoclaw <name> connect documents [--probe-only]; deprecated aliases note onboarding flags.
Installer Help Text
install.sh
NEMOCLAW_PROVIDER help output extended to include routed as a documented provider value.
Planned Execution
test/e2e/e2e-cloud-experimental/check-docs.sh
Builds a _planned list of phases and executes [cli], [install], and [links] independently.

Sequence Diagram(s)

sequenceDiagram
  participant GH as GitHub Actions
  participant Runner as ubuntu-latest
  participant CLI as nemoclaw
  participant Script as check-docs.sh
  GH->>Runner: start cli-parity job
  Runner->>Runner: checkout, setup Node.js, npm install, build CLI
  Runner->>Script: bash check-docs.sh --only-cli
  Script->>CLI: run commands/flag extraction
  Runner->>Script: bash check-docs.sh --only-install
  Script->>Script: run_install_check (TS -> install.sh / scripts/install.sh)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I hop through docs and scripts at night,
I sniff for flags that vanished from sight,
I wake the workflow, run checks one by one,
Match providers and commands till it's done,
Tiny paws keep CI bright and light.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately captures the main purpose: syncing CLI documentation/help with actual CLI behavior and adding automated parity checks to prevent drift recurrence.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
docs/reference/commands.md (1)

313-313: ⚡ Quick win

Split the two-sentence flag description into separate source lines.

Line 313 has two sentences on one line; keep one sentence per line in docs source for diff readability.

As per coding guidelines, “One sentence per line in source (makes diffs readable). Flag paragraphs where multiple sentences appear on the same line.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reference/commands.md` at line 313, The table entry for the
`--probe-only` flag contains two sentences on one source line; split the
description into two source lines so each sentence is on its own line (e.g.,
first line: "Verify the sandbox is reachable over SSH." and second line: "Then
exit without dropping into a shell. Useful for health checks and scripted
readiness probes.") so diffs remain readable and follow the
one-sentence-per-line docs guideline.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/reference/commands.md`:
- Line 313: The table entry for the `--probe-only` flag contains two sentences
on one source line; split the description into two source lines so each sentence
is on its own line (e.g., first line: "Verify the sandbox is reachable over
SSH." and second line: "Then exit without dropping into a shell. Useful for
health checks and scripted readiness probes.") so diffs remain readable and
follow the one-sentence-per-line docs guideline.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5788cfef-1172-4cff-bb27-37dea2087db3

📥 Commits

Reviewing files that changed from the base of the PR and between b1320d5 and 51c5ac4.

📒 Files selected for processing (3)
  • .github/workflows/docs-cli-parity-pr.yaml
  • docs/reference/commands.md
  • install.sh

Dongni-Yang and others added 2 commits May 8, 2026 13:34
…uideline (NVIDIA#3224)

Convert the table-cell description into two prose sentences on separate
source lines per CodeRabbit nit on PR NVIDIA#3234. Markdown table cells can't
span source lines, so the table is dropped in favour of a short prose
paragraph immediately after the USAGE block.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…IDIA#3224)

Closes the two gaps in the existing PR-CI parity check that left
NV QA's recurring TC5882289 / TC5956310 vitest assertions undetected
at PR time:

1. Flag-level CLI parity. The previous --only-cli mode only diffed
   command names from --dump-commands against ### nemoclaw <cmd>
   headings. It missed per-command flag drift entirely (e.g.
   --probe-only being absent from the connect section was undetected).
   Add a phase 3 that, for each command, parses `nemoclaw <cmd>
   --help` for long-form flags and confirms each non-global flag
   appears somewhere in commands.md. Skips --help / --version.

2. install.sh provider parity. The script previously had no install.sh
   coverage. Add run_install_check() and a new --only-install mode
   that extracts the canonical "Valid values: ..." list from
   src/lib/onboard/providers.ts (with fallback to the legacy
   onboard-providers.ts location) and confirms each non-helper value
   appears in both the bootstrap NEMOCLAW_PROVIDER block of
   install.sh and in scripts/install.sh's usage(). Greps the
   provider-section block specifically rather than the whole file
   to avoid false positives from unrelated mentions of "gemini" or
   "ollama" in helper text.

Wire both new checks into docs-cli-parity-pr.yaml:
- existing --only-cli step now also runs flag-level parity
- new explicit --only-install step
- path filter widened to also trigger on scripts/install.sh

Validated locally with three negative tests — removing routed from
install.sh, removing gemini from scripts/install.sh, and removing
gemini from both — each reports the drift correctly and exits
non-zero.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/e2e-cloud-experimental/check-docs.sh`:
- Around line 319-330: The grep checks treat canonical provider names as
substrings (so "openai" matches "compatible-openai"); update the exact-match
logic in the loop that reads _values/_raw into v by using fixed-string
whole-line matching (replace the grep invocations against _bootstrap_block and
_payload_block with exact/full-match variants, e.g. switch grep -qF to grep -qFx
or another exact-match approach) for both the check against _bootstrap_block and
the check against _payload_block so providers are compared as exact values
rather than substrings.
- Around line 220-224: The script currently greps the whole COMMANDS_MD for each
flag, which allows flags documented under other commands to pass; update the
flag check in check-docs.sh to first extract only the documentation section for
the current command (use the command identifier derived from cmd_line) and then
grep that section for the flag instead of the whole COMMANDS_MD; specifically,
inside the while loop that reads flags (and around the variables cmd_line and
COMMANDS_MD), retrieve the matching command section (for example by locating the
heading for the command and the next heading boundary) and run grep -qF --
"$flag" against that extracted subsection so per-command flags like --json or
--yes documented elsewhere don’t cause false positives.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 03f7a411-05a2-4cee-b0b1-af4059eff1a9

📥 Commits

Reviewing files that changed from the base of the PR and between 51c5ac4 and c288aa2.

📒 Files selected for processing (3)
  • .github/workflows/docs-cli-parity-pr.yaml
  • docs/reference/commands.md
  • test/e2e/e2e-cloud-experimental/check-docs.sh
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/docs-cli-parity-pr.yaml

Comment thread test/e2e/e2e-cloud-experimental/check-docs.sh Outdated
Comment thread test/e2e/e2e-cloud-experimental/check-docs.sh
Dongni-Yang and others added 2 commits May 8, 2026 14:03
…DIA#3224)

The flag-level parity logic shipped in c288aa2 had three bugs that masked
real drift and one unsafe word-splitting pattern. This change addresses all
four and documents one remaining coverage gap.

* Section-scoped grep — global grep over commands.md would match `--yes`
  anywhere in the file, hiding the case where it is missing from the
  specific command's section. New `extract_md_section` awk function
  isolates one heading's body.
* Word-boundary flag match — substring grep matched `--yes` inside
  `--yes-i-accept-third-party-software`. Replaced with
  `(^|[^a-zA-Z0-9_-])${flag}([^a-zA-Z0-9_-]|$)`.
* `--dump-commands` lines start with `nemoclaw `; we now strip the prefix
  before invoking via `node bin/nemoclaw.js`, and replace `<name>` with
  `placeholder-sandbox` (the previous `_check_docs_sandbox_` was rejected
  by sandbox-name validation).
* Read the invocation into an argv array so token splitting is explicit
  rather than relying on unquoted expansion (clears SC2086).

Coverage caveat documented inline: most `nemoclaw <name> ...` commands
gate `--help` behind sandbox-name validation that requires a registered
sandbox, so flag drift on those commands is not caught here. The
existing E2E suite (which runs against a real sandbox) covers them.
Follow-up to provision a placeholder sandbox in CI or add a CLI mode
that bypasses name validation for `--help` is tracked as a follow-up to
NemoClaw#3224.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Substring-matching the canonical provider list against the help block
made `anthropic` falsely pass when only `anthropicCompatible` was
present (since the latter contains the former). The same class of
false positive was waiting to happen for any future provider added
as a prefix of an existing one.

Two changes:

* Tokenize the extracted printf block into discrete identifiers
  (stripping `\n` literals, splitting on shell-quote/punct/`|`,
  preserving camelCase) and exact-match each canonical value with
  `grep -qxF` against the token list.
* Tighten the awk block-extractor: check the next-NEMOCLAW_ boundary
  BEFORE printing, so the following `NEMOCLAW_POLICY_MODE` printf line
  no longer bleeds in. Previously the leak was hidden because POLICY
  values like `suggested` aren't canonical providers, but `custom` is
  both a canonical provider AND a POLICY_MODE token, so removing
  `custom` from the provider line would have falsely passed.

Negative test: removing `anthropicCompatible` from install.sh's
bootstrap_usage now correctly flags drift.

Addresses CodeRabbit feedback on PR NVIDIA#3234.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dongni-Yang and others added 3 commits May 8, 2026 14:19
The phase-3 outer loop reads `$_tmp/help.txt` line-by-line via
`done <"$_tmp/help.txt"`, which makes that file the loop's stdin.
The inner `node bin/nemoclaw.js <cmd> --help` invocation inside a
command substitution inherits that stdin. Some startup paths read a
byte from stdin, which silently consumes lines from the help file —
the outer `read` then jumps over them. In practice the loop was
exiting after `nemoclaw credentials reset` and never reaching `debug`,
`deploy`, `gc`, `list`, `onboard`, etc., so flag drift on every
alphabetically-later command was being missed.

Fix: redirect the inner node call's stdin to /dev/null so it cannot
race the outer read.

Negative-tested by mutating commands.md to remove `--json` from the
list section, `--probe-only` from connect, `--yes` from onboard,
`--gateway` from uninstall — each is now flagged. Pre-fix only
`--probe-only` was caught (it was alphabetically before the loop
exited).

Signed-off-by: Dongni Yang <dongniy@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…3224)

With the stdin fix from the preceding commit, phase-3 now successfully
iterates every command and surfaces three real doc gaps that the prior
silent truncation had been masking:

* `nemoclaw setup` and `nemoclaw setup-spark` are deprecated aliases
  that delegate to onboard's full flag surface (--non-interactive,
  --resume, --fresh, --recreate-sandbox, --gpu, --no-gpu, --from,
  --name, --agent, --control-ui-port, --yes, -y,
  --yes-i-accept-third-party-software). Their help text shows all
  these flags via the alias trampoline, but commands.md only said
  "compatibility alias" without listing them. Now spelled out.
* `nemoclaw uninstall --gateway <name>` was unadvertised in the docs
  flag table even though `--help` lists it (default `nemoclaw`). Added
  to the table and the USAGE line.

These are the real instances of the drift that NemoClaw#3224 was
filed against — the kind of silent post-oclif gap the per-PR parity
check is meant to prevent going forward.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Repo's prek shfmt config uses `-i 2 -ci`. Locally I lacked shfmt at
commit time and the formatter ran on CI, which then failed the
"Files were modified by following hooks" wrapper check. Apply the
formatting up front so CI is clean. No behavior change.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Dongni-Yang Dongni-Yang force-pushed the fix/docs-cli-parity-3224 branch from 9eaab89 to ce70255 Compare May 8, 2026 06:51

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/e2e-cloud-experimental/check-docs.sh`:
- Around line 404-418: The current loop checks that each canonical provider in
_canonical exists in _bootstrap_values and _payload_values but doesn't check the
inverse; add reciprocal checks that iterate tokenized _bootstrap_values and
_payload_values and assert each non-empty, non-exempt token (skip patterns
install-* and start-windows-ollama) is present in _canonical, emitting similar
error messages and setting _drift=1 when an extra is found; use the same token
normalization (tr -d '[:space:]') and grep -qxF -- comparisons and mirror the
existing error text format so extras in bootstrap or payload usage are reported.
- Around line 259-261: The current invocation that populates _help_text using
"$NODE" "$CLI_JS" "${_invoke_args[@]}" --help || true masks all failures; change
it to capture both the stdout into _help_text and the command's exit code, then
only tolerate a non-zero exit when the command is a known sandbox variant (check
_invoke_args for the sandbox-specific name(s)); if the help produced no stdout
and the exit code is non-zero for any other command, fail the test (emit an
error and exit non-zero) instead of continuing. Use the existing variables
_help_text, NODE, CLI_JS and _invoke_args to locate the call and add explicit
exit-code handling and a whitelist check for sandbox-name failures.
- Around line 275-285: The current loop only ensures flags from --help (variable
_flags) appear in the commands.md section ($_section) but not the reverse;
update the check to extract long flags from the section into a set (e.g. parse
$_section for tokens matching --[A-Za-z0-9_-]+ into a variable like
_section_flags), then perform a symmetric diff: (1) flags in _flags not in
_section_flags (current check) and (2) flags in _section_flags not in _flags,
and for either case echo a clear error (same format as the existing message) and
set _flag_drift=1; apply this logic alongside the existing pattern matching
(_pat, cmd_line, COMMANDS_MD) so it reports stale documented flags as well as
missing CLI flags.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 48dae1e2-9339-4b05-b974-5c8c060cf0f4

📥 Commits

Reviewing files that changed from the base of the PR and between 9eaab89 and ce70255.

📒 Files selected for processing (1)
  • test/e2e/e2e-cloud-experimental/check-docs.sh

Comment thread test/e2e/e2e-cloud-experimental/check-docs.sh Outdated
Comment thread test/e2e/e2e-cloud-experimental/check-docs.sh
Comment thread test/e2e/e2e-cloud-experimental/check-docs.sh
Address three CodeRabbit findings where the original one-way checks
would silently miss drift:

* Flag parity only flagged flags in --help missing from docs, not
  flags documented under a section that no longer exist in --help.
  Added a doc-flag extractor that handles fenced USAGE blocks (any
  --foo) and inline backtick refs (`--foo`), so prose cross-tool
  mentions like `openshell gateway start --recreate` aren't mistaken
  for nemoclaw flag documentation.
* Install provider parity only flagged canonical providers absent
  from install.sh. The reverse — install.sh advertising providers no
  longer canonical — was unchecked. tokenize_provider_block now drops
  `(aliases:...)` lines and `printf`/`NEMOCLAW_PROVIDER` keywords so
  the canonical-extras direction works without false positives.
* The phase-3 --help invocation swallowed all errors via `|| true`.
  Capture the exit code and only tolerate sandbox-name failures on
  `<name>`-prefixed commands; any other --help failure aborts the
  run with the captured stderr.

Latent issues fixed in passing:

* extract_md_section didn't match headings with placeholder suffix
  like `### `nemoclaw credentials reset <PROVIDER>``, so the section
  fell back to whole-doc and hid drift not also elsewhere in the
  file. Apply the same trailing-placeholder strip phase 2 uses.
* extract_md_section didn't stop at h1/h2 headings, so sections bled
  into the next top-level area. Stops at h1/h2/h3 (h4+ kept as
  sub-sections). Regex uses explicit alternation since traditional
  awk treats `{n,m}` literally.
* `nemoclaw onboard --from` is a --dump-commands variant entry, not
  a separate command — re-invoking it with --help triggers a
  flag-value parsing error. Skip cmd lines containing ` --`.

Negative-tested forward (removing --probe-only/--json/--yes/--gateway/
routed/anthropicCompatible each surfaces drift) and reverse (stale
--bogus in debug flag table, stale fakexyz provider in install.sh
both surface drift). Clean state passes both --only-cli and
--only-install.

Signed-off-by: Dongni Yang <dongniy@nvidia.com>

@ericksoa ericksoa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the updated head (91baaa1) against current main. The CLI parity check is now bidirectional at command and flag level, install provider parity is bidirectional against the canonical provider list, and ordinary --help failures are no longer swallowed. I also pushed a small portability fix so the provider tokenization passes on macOS/BSD sed as well as Linux.\n\nValidated locally: npm ci --ignore-scripts; npm run build:cli; bash test/e2e/e2e-cloud-experimental/check-docs.sh --only-cli; bash test/e2e/e2e-cloud-experimental/check-docs.sh --only-install; git diff --check. GitHub checks are green on 91baaa1.

@ericksoa ericksoa merged commit b9e68ba into NVIDIA:main May 9, 2026
14 checks passed
ericksoa pushed a commit that referenced this pull request May 13, 2026
#3382)

## Summary

Fixes #3358 (NV QA NVB#6165494). On Brev v0.0.38, running
`NEMOCLAW_AGENT=hermes nemoclaw onboard` printed `Run: nemohermes <name>
connect` in the post-onboard next-steps message — but users who launched
via the `nemoclaw` binary may not have the `nemohermes` alias on their
PATH, so copy-pasting the suggestion hit `nemohermes: command not
found`. The fix decouples the suggested CLI binary name from agent
product branding, so the message reflects whichever binary the user
actually invoked.

## Related Issue

Fixes #3358

## Changes

Three commits on this branch:

**1. `fix(hermes): use invoked CLI name in onboard next-steps output
(#3358)`** — the core fix.

- `bin/nemohermes.js` — launcher now also sets
`NEMOCLAW_INVOKED_AS=nemohermes` so the runtime can distinguish
"alias-invoked" from "base-binary-invoked with env-var".
- `src/lib/cli/branding.ts` — `getAgentBranding()` derives `cli` from
`NEMOCLAW_INVOKED_AS` (with a known-name allowlist, defaulting to
`nemoclaw`); `display` / `product` / `uninstallGoodbye` continue to be
agent-driven so Hermes branding stays correct everywhere else. All 142
existing `cliName()` / `CLI_NAME` callsites pick up the new behaviour
transparently.
- `src/lib/cli/branding.test.ts` *(new)* — 5 unit tests covering the
four invocation paths plus an unknown-value safety fallback.
- `test/nemohermes-alias.test.ts` — replaced the test that codified the
buggy `nemoclaw onboard --agent hermes` -> `nemohermes onboard`
assertion, and added two regression tests covering the exact NV QA repro
(`NEMOCLAW_AGENT=hermes nemoclaw onboard`) plus the `--agent hermes`
flag path.

**2. `chore(ci): allowlist NEMOCLAW_INVOKED_AS as internal launcher
marker (#3358)`** — CI follow-up. The `env-var-docs` gate caught the new
variable in `src/`. `NEMOCLAW_INVOKED_AS` is internal-only (set by
`bin/nemohermes.js`, value constrained to a known-name allowlist), so
it's listed in `ci/env-var-doc-allowlist.json` rather than documented
for users.

**3. `test(hermes): clear NEMOCLAW_INVOKED_AS from helper env for
hermeticity`** — addresses CodeRabbit's review feedback. The `runHermes`
/ `runNemoClaw` test helpers inherit `process.env`; an accidentally-set
`NEMOCLAW_INVOKED_AS` in the parent process could flip CLI branding and
make the regression assertions environment-dependent. Both helpers now
explicitly clear the marker alongside `NEMOCLAW_AGENT`.

Behaviour matrix after this PR:

| Invocation | Suggested command | Status |
|---|---|---|
| `nemohermes onboard` (alias bin) | `nemohermes onboard` | unchanged |
| `NEMOCLAW_AGENT=hermes nemoclaw onboard` (NV QA repro) | `nemoclaw
onboard` | **fixed** |
| `nemoclaw onboard --agent hermes` | `nemoclaw onboard` | **fixed** |

Out of scope (filed as a follow-up consideration):
`src/lib/actions/update.ts:60-68` has the same conceptual `cliName`
resolution but a separate code path, and the NV QA repro did not
exercise it. Keeping the diff narrow per the literal issue scope.

The recently-merged docs/CLI parity check from #3234 would not have
caught this — it validates `--help` <-> `docs/reference/commands.md`
parity, not onboard runtime output strings.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Verification

- [ ] `npx prek run --all-files` passes *(local env has Python 3.8; prek
requires >=3.9 — relying on CI)*
- [x] `npm test` passes for the impacted suites (208/208 across 9
nemohermes-related files; 5/5 in the new `branding.test.ts`; 8/8 in
`nemohermes-alias.test.ts`). 16 unrelated failures in `runtime-shell` /
`ssrf-parity` / `generate-openclaw-config` / `secret-redaction`
confirmed pre-existing on `upstream/main`.
- [x] `npm run typecheck:cli` passes
- [x] Plugin `tsc --noEmit` passes
- [x] Biome lint + format clean on all modified files
- [x] `npx tsx scripts/check-env-var-docs.ts` passes after the allowlist
entry
- [x] Tests added or updated for new or changed behavior
- [x] No secrets, API keys, or credentials committed
- [ ] Docs updated for user-facing behavior changes *(N/A —
docs/quickstart-hermes.md describes the alias-bin path which is
unchanged; the fix corrects an unintended divergence in the base-bin
path; the new env var is internal-only and allowlisted)*

Manual repro on the built CLI, all three invocation paths verified
directly:

```
$ HOME=/tmp/v1 node bin/nemohermes.js onboard --resume --non-interactive --yes-i-accept-third-party-software 2>&1 | grep -A1 "rebuild it"
  To change configuration on an existing sandbox, rebuild it:
    nemohermes onboard          <- unchanged (alias-bin path stays correct)

$ HOME=/tmp/v2 NEMOCLAW_AGENT=hermes node bin/nemoclaw.js onboard --resume --non-interactive --yes-i-accept-third-party-software 2>&1 | grep -A1 "rebuild it"
  To change configuration on an existing sandbox, rebuild it:
    nemoclaw onboard            <- NV QA repro, was 'nemohermes onboard' before the fix

$ HOME=/tmp/v3 node bin/nemoclaw.js onboard --agent hermes --resume --non-interactive --yes-i-accept-third-party-software 2>&1 | grep -A1 "rebuild it"
  To change configuration on an existing sandbox, rebuild it:
    nemoclaw onboard            <- --agent flag path, also fixed
```

Fail-condition counters (`grep -c 'nemohermes'` on full output):

| Path | Count | Expected |
|---|---|---|
| `nemohermes onboard` | 1 | >0 (alias is what user invoked) |
| `NEMOCLAW_AGENT=hermes nemoclaw onboard` | 0 | 0 (no leaked alias) |
| `nemoclaw onboard --agent hermes` | 0 | 0 (no leaked alias) |

---

Signed-off-by: Dongni Yang <dongniy@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* CLI now correctly identifies and applies proper branding based on
which binary (nemohermes or nemoclaw) you invoke, ensuring accurate
display names and help text.

* **Tests**
* Added comprehensive test coverage for environment-based branding
behavior and CLI launcher detection.

* **Documentation**
* Updated environment variable documentation to include internal
launcher marker.

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/NVIDIA/NemoClaw/pull/3382)

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Dongni Yang <dongniy@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wscurran wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Linux][Docs] commands.md and install.sh --help not updated after oclif migration and new provider additions

3 participants