fix: skip Brave policy preset for unsupported agents by ericksoa · Pull Request #3223 · NVIDIA/NemoClaw

ericksoa · 2026-05-08T04:03:35Z

Summary

thread web-search support into setup policy preset suggestions and selection
omit Brave from tier defaults when the active agent does not support the Brave web-search path
clean up stale Brave policy presets on re-onboard/resume while preserving other user-added presets

Tests

npm run build:cli
npx vitest run test/onboard.test.ts -t "computeSetupPresetSuggestions|agentSupportsWebSearch|configureWebSearch"
npx vitest run test/onboard-preset-diff.test.ts test/policy-tiers-onboard.test.ts
npm run typecheck:cli
npm run format:check
npm run source-shape:check
npm run lint
git diff --check

Summary by CodeRabbit

Release Notes

Bug Fixes
- Policy presets are now automatically filtered to exclude unsupported configurations based on system constraints (e.g., when web search is unavailable).
- The onboarding resume flow now detects and prevents reapplication of previously selected presets that are no longer supported.
Tests
- Added regression tests for policy preset handling when certain presets become unsupported during onboarding.

coderabbitai · 2026-05-08T04:03:47Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d6468752-a6d7-49db-b011-44ca04e381ae

📥 Commits

Reviewing files that changed from the base of the PR and between 32758e5 and 39f3657.

📒 Files selected for processing (1)

src/lib/onboard.ts

📝 Walkthrough

Walkthrough

The PR filters out unsupported policy presets (notably Brave when web search is unavailable) in three places: during non-interactive setup via environment-supplied presets, in selection replay logic, and during onboarding resume when detecting drift between recorded and supported presets. Two new test cases verify this filtering behavior.

Changes

Web Search Policy Preset Filtering

Layer / File(s)	Summary
Preset Filtering Helper and Non-Interactive Mode `src/lib/onboard.ts`	Introduces `filterSupportedPresetNames()` helper to filter preset names by custom/supported status under `webSearchSupported` constraint. Non-interactive env-supplied presets are filtered before use in custom and suggested modes.
Selection Replay and Resume Flow Filtering `src/lib/onboard.ts`	Selection replay now triggers on whether `selectedPresets` is non-null (not array length). Resume detects unsupported drift by length comparison, skips fast-path when drift is found, and gates `setupPoliciesWithSelection()` call on whether recorded presets are an array.
Preset-Diff Test Cases for Unsupported Presets `test/onboard-preset-diff.test.ts`	Adds two non-interactive test scenarios verifying that Brave preset is excluded when `webSearchSupported: false`. Re-onboard test verifies removal call and preservation of other tier presets; resume selection test verifies narrowing of `selectedPresets` array.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🐰 A rabbit hops through the presets with care,
Filtering Brave when web-search isn't there!
Resume flows check for unsupported drift,
Tests catch the drops—a quality shift. ✨
Web-search constraints now keep all in line,
Safe onboarding happens, by design! 🔍

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: skip Brave policy preset for unsupported agents' clearly and concisely describes the main change: filtering out the Brave preset when agents don't support web search.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/nemohermes-skip-brave-preset

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard.ts`:
- Around line 8605-8614: The resume/apply path currently treats an empty
filtered preset list as null and skips reapplying, and environment-driven
`chosen` presets are validated only against `allPresets` but not re-filtered for
feature support; update the logic that builds resumeSelection (using
options.selectedPresets or chosen) so an empty array remains an empty array (do
not coerce to null), re-run support filtering against the effective list (e.g.,
filter chosen/selectedPresets by supported presets using the same support check
used elsewhere such as webSearchSupported), and then proceed to call
onSelection, waitForSandboxReady, note, and syncPresetSelection with that
filtered array; ensure functions/variables referenced include
options.selectedPresets, chosen, allPresets, webSearchSupported,
resumeSelection, onSelection, waitForSandboxReady, and syncPresetSelection so
unsupported presets (like "brave" when webSearchSupported === false) are pruned
and an intentionally empty selection still triggers the resume branch.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ec7b96a2-7a2b-4f0e-a4da-7cba6b4d70b6

📥 Commits

Reviewing files that changed from the base of the PR and between b1320d5 and 32758e5.

📒 Files selected for processing (3)

src/lib/onboard.ts
test/onboard-preset-diff.test.ts
test/onboard.test.ts

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

github-actions · 2026-05-12T16:53:10Z

E2E Advisor Recommendation

Required E2E: onboard-resume-e2e, cloud-onboard-e2e, double-onboard-e2e
Optional E2E: onboard-repair-e2e, brave-search-e2e

Dispatch hint: onboard-resume-e2e,cloud-onboard-e2e,double-onboard-e2e

Workflow run

Full advisor summary

Pi Semantic E2E Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

onboard-resume-e2e: Directly exercises the changed code paths: NEMOCLAW_POLICY_MODE=custom with empty NEMOCLAW_POLICY_PRESETS must still fail at the policy step (message now emitted before filterSupportedPresetNames), and resume with NEMOCLAW_POLICY_MODE=skip must complete. Both touch the new selectedPresets!==null resume-trigger branch and the custom-mode env parsing change.
cloud-onboard-e2e: Runs non-interactive onboard with NEMOCLAW_POLICY_MODE=custom and NEMOCLAW_POLICY_PRESETS=npm,pypi end-to-end — the exact branch where filterSupportedPresetNames now runs over envPresets and where chosen is set post-filter. Regressions in env-preset validation (e.g. unknown-preset exit, authoritative vs suggested merging) would surface here.
double-onboard-e2e: Re-onboarding the same sandbox is the real-world trigger for the new recordedPolicyPresetsHaveUnsupported / clampSetupPolicyPresetNames resume-skip gate. The change in 'selectedPresets' being passed unconditionally (array, even empty) vs only when non-empty affects whether the second onboard re-applies, reconciles, or skips the policy step.

Optional E2E

onboard-repair-e2e: Repair path re-enters setupPoliciesWithSelection; worth running as a confidence check that preset reconciliation after partial failure still produces a consistent applied-set.
brave-search-e2e: The new filter specifically drops 'brave' when webSearchSupported=false. brave-search-e2e onboards with web search enabled and asserts the brave preset is applied and wired through to openclaw — a good sanity check that the supported-path still activates brave.

New E2E recommendations

onboarding/policy-presets (medium): No existing E2E asserts that re-onboarding a sandbox which previously had an unsupported preset (e.g. 'brave' after the provider loses webSearchSupported) actually removes that preset from the gateway-applied policy set. The unit test covers this in-process, but the full host→gateway syncPresetSelection→policy removal chain is not covered end-to-end.
- Suggested test: Add an E2E (e.g. test/e2e/test-onboard-unsupported-preset-removal.sh) that: (1) onboards with a web-search-capable provider so 'brave' is applied and visible via openshell policy get, (2) re-onboards with a provider whose webSearchSupported=false while NEMOCLAW_POLICY_MODE=suggested and NEMOCLAW_POLICY_PRESETS preserved, (3) asserts the recorded/applied policy list no longer contains 'brave' and that api.search.brave.com is no longer in the loaded gateway policy.

Dispatch hint

Workflow: .github/workflows/nightly-e2e.yaml
jobs input: onboard-resume-e2e,cloud-onboard-e2e,double-onboard-e2e

## Summary - Add v0.0.40 release notes and update docs version metadata. - Document release-prep behavior changes around onboarding, local inference, policy preset filtering, and config recovery. - Refresh generated `nemoclaw-user-*` skills from the source docs. ## Source summary - #3383 -> `docs/about/release-notes.md`, `docs/reference/commands.md`, `docs/manage-sandboxes/lifecycle.md`: Reflect macOS Docker-driver OpenShell gateway onboarding and upgrade behavior. - #3378 -> `docs/about/release-notes.md`: Capture the Docker-driver gateway TCP readiness fix and clearer startup failures. - #3338 -> `docs/about/release-notes.md`, `docs/inference/use-local-inference.md`: Reflect the Ollama auth proxy token requirement on native API routes. - #3420 -> `docs/about/release-notes.md`, `docs/get-started/prerequisites.md`, `docs/inference/use-local-inference.md`: Document the Linux Ollama `zstd` preflight and sudo messaging. - #3417 -> `docs/about/release-notes.md`, `docs/inference/inference-options.md`, `docs/inference/use-local-inference.md`: Reflect detected running vLLM provider selection. - #3223 -> `docs/about/release-notes.md`, `docs/reference/commands.md`, `docs/reference/network-policies.md`, `docs/get-started/quickstart.md`: Document agent-aware policy preset filtering. - #3385 -> `docs/about/release-notes.md`: Capture the dashboard forward TCP reachability check. - #3160 -> `docs/about/release-notes.md`, `docs/reference/troubleshooting.md`: Document empty `openclaw.json` baseline recovery. - #3367 -> `docs/about/release-notes.md`: Capture OpenClaw plugin compatibility metadata. ## Test plan - [x] `python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix nemoclaw-user` - [x] `make docs` - [x] `git diff --check` - [x] Skip-term scan for `docs/.docs-skip` terms  ## Summary by CodeRabbit # Release Notes v0.0.40 * **New Features** * Sandbox configuration recovery when inference changes cause data loss * Policy presets now intelligently filter based on agent capabilities * Enhanced gateway health checks and upgrade reliability * **Documentation** * Improved local inference setup instructions with clearer dependency requirements * Clarified vLLM experimental feature availability and prerequisites * Reorganized architecture documentation for enhanced clarity * Refined security and hardening guidance [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/NVIDIA/NemoClaw/pull/3427)

fix: skip Brave preset for unsupported agents

32758e5

coderabbitai Bot reviewed May 8, 2026

View reviewed changes

Comment thread src/lib/onboard.ts Outdated

wscurran added fix integration: brave Brave integration behavior labels May 8, 2026

jyaunches mentioned this pull request May 8, 2026

fix(runtime): repair stale inference DNS routes #3267

Merged

cv closed this May 12, 2026

cv reopened this May 12, 2026

Merge origin/main into fix/nemohermes-skip-brave-preset

39f3657

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

ericksoa self-assigned this May 12, 2026

Merge branch 'main' into fix/nemohermes-skip-brave-preset

6ba1391

ericksoa added the v0.0.40 label May 12, 2026

ericksoa requested a review from cv May 12, 2026 17:10

ericksoa added the status: rfr label May 12, 2026

cv approved these changes May 12, 2026

View reviewed changes

cv merged commit 7783d08 into main May 12, 2026
24 checks passed

miyoungc mentioned this pull request May 12, 2026

docs: prepare v0.0.40 release docs #3427

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: skip Brave policy preset for unsupported agents#3223

fix: skip Brave policy preset for unsupported agents#3223
cv merged 3 commits into
mainfrom
fix/nemohermes-skip-brave-preset

ericksoa commented May 8, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 8, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented May 12, 2026 •

edited

Loading

Pi Semantic E2E Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ericksoa commented May 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

Summary by CodeRabbit

Release Notes

Uh oh!

coderabbitai Bot commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

Pi Semantic E2E Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ericksoa commented May 8, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 8, 2026 •

edited

Loading

github-actions Bot commented May 12, 2026 •

edited

Loading