Skip to content

nightly-e2e: check-docs CLI/docs parity mismatch (cloud-experimental-e2e step 4) #2333

@gaveezy

Description

@gaveezy

Description

Problem Statement

The cloud-experimental-e2e job in nightly-e2e run #24815818271 fails at step 4 (Documentation checks (check-docs.sh)) because the CLI --help output and docs/reference/commands.md headings are out of sync. The check-docs script found 36 unique command lines from --help but only 31 headings in the commands reference, with multiple false and genuine mismatches.

Three root causes:

  1. Single-space description separators in help() for channels remove, policy-remove, and credentials reset cause description text to leak into normalized signatures.
  2. The docs-side normalizer does not strip trailing <arg> and [optional] placeholders, unlike the help-side normalizer, producing false mismatches.
  3. Four commands (onboard --from, setup, start, stop) have no ### heading in commands.md.

Note: The same job also reports a Landlock /sandbox writability failure in step 3 (04-landlock-readonly). That is an intermittent infrastructure issue unrelated to these documentation changes.

Proposed Design

Fix three files:

  • src/nemoclaw.ts: widen single-space separators to 2+ spaces in three help() lines.
  • test/e2e/e2e-cloud-experimental/check-docs.sh: add trailing <arg> and [optional] stripping to the docs-side normalizer.
  • docs/reference/commands.md: add ### headings for onboard --from, setup, start, stop.

See fix PR: #2332

Alternatives Considered

  • Excluding deprecated aliases (setup, start, stop) from the check rather than documenting them. Rejected because other deprecated aliases like setup-spark already have headings, and consistent documentation is preferable.
  • Modifying only the docs headings (removing <arg> from headings). Rejected because keeping args in headings is reader-friendly; fixing the normalizer is the correct approach.

Category

config_error

Reproduction Steps

  1. Re-run cloud-experimental-e2e on commit 04757ed via:
    gh workflow run nightly-e2e.yaml --repo NVIDIA/NemoClaw --ref main
    
  2. Step 4 (check-docs.sh) will report a mismatch between --help and commands.md.

Environment

  • OS: Ubuntu 24.04.4 LTS
  • Node.js: v22.22.2 (installed via nvm)
  • Docker: GitHub Actions runner default
  • NemoClaw: commit 04757edf11ad857acabdec8e1ecc6d068f5abab4
  • Other: Run ID 24815818271

Debug Output

check-docs: [cli] phase 1: extracted 36 unique command line(s) from --help
check-docs: [cli] phase 2: extracted 31 heading(s) from docs/reference/commands.md
check-docs: [cli] mismatch between --help and docs/reference/commands.md

Only in --help (add ### to commands.md or fix help):
  nemoclaw <name> channels add
  nemoclaw <name> channels remove <channel> Clear credentials and rebuild
  nemoclaw <name> channels start
  nemoclaw <name> channels stop
  nemoclaw <name> policy-remove [preset] Remove an applied policy preset (--yes, --dry-run)
  nemoclaw <name> skill install
  nemoclaw <name> snapshot restore
  nemoclaw credentials reset
  nemoclaw credentials reset <KEY> Remove a stored credential so onboard re-prompts
  nemoclaw onboard --from
  nemoclaw setup
  nemoclaw start
  nemoclaw stop

Only in commands.md (add to help() in bin/nemoclaw.js or fix heading):
  nemoclaw <name> channels add <channel>
  nemoclaw <name> channels remove <channel>
  nemoclaw <name> channels start <channel>
  nemoclaw <name> channels stop <channel>
  nemoclaw <name> policy-remove
  nemoclaw <name> skill install <path>
  nemoclaw <name> snapshot restore [selector]
  nemoclaw credentials reset <KEY>

Logs

N/A — the debug output above contains the full failure evidence.

Checklist

  • I can reproduce the bug
  • I searched for duplicates

Labels (auto-applied on submit): bug, status: triage, VRDC

Metadata

Metadata

Assignees

No one assigned

    Labels

    VRDCIssues and PRs submitted by NVIDIA VRDC test team.area: ciCI workflows, checks, release automation, or GitHub Actionsci-failureAuto-created by nemoclaw-diagnosis skill

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions