docs(policy): refine shared triage taxonomy by wscurran · Pull Request #4935 · NVIDIA/NemoClaw

wscurran · 2026-06-08T03:45:03Z

Summary

Updates the shared NemoClaw maintainer policy references used by triage agents and downstream consumers.

This PR keeps the policy package as the source of truth and tightens the taxonomy/routing guidance needed for the nvoss-velocity shared-policy migration:

clarifies that normal initial triage should not newly add needs: triage
narrows needs:* guidance to blocking action queues, especially needs: info and needs: design
adds PR type evidence from conventional commit prefixes
tightens specific integration routing so named Hermes/OpenClaw/etc. evidence gets the matching integration:* label
clarifies area: ci versus area: e2e so CI is not added just because an e2e failure was observed in CI
improves Windows ARM / WSL platform routing signals
adds concrete shared-policy examples for named integrations, OpenClaw e2e failures, and Windows ARM install failures

No application code changes are included here.

Evaluation

This policy version was evaluated in the nvoss-velocity migration experiment using a deterministic 100-item NemoClaw fixture: 50 issues and 50 PRs, all numbered #2000+.

The evaluation compares recommendations after translating legacy labels into the new shared-policy concepts, rather than raw old-label equality.

Baseline, current Llama with this v2 policy:

Label F1: 0.485
Label precision: 0.402
Label recall: 0.611
Issue Type match: 0.976
Project field match: 1.000
Parse failures: 0

Best Nemotron Ultra Preview candidate used the same policy files:

Label F1: 0.493
Label precision: 0.392
Label recall: 0.663
Issue Type match: 0.976
Project field match: 1.000
Parse failures: 0

The Ultra result did not require separate policy-file changes; it used the same v2 shared policy plus model-specific runtime controls in nvoss-velocity.

Validation

Parsed .agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.json with python3 -m json.tool
Ran git diff --check
Confirmed this branch starts from current origin/main
Confirmed only the shared policy reference files are modified

Notes

This is the policy-only first step. A follow-up nvoss-velocity PR will consume this shared policy package and switch normal triage runtime behavior to the selected model profile.

Summary by CodeRabbit

Documentation
- Refined issue triage and labeling guidance with clarified rules for label taxonomy, improved concrete examples, and tightened requirements for maintainer workflows to ensure better issue categorization and reduce placeholder labeling.

copy-pr-bot · 2026-06-08T03:45:06Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2026-06-08T03:45:10Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 09a1730d-5bb3-4627-9083-01f28087da4c

📥 Commits

Reviewing files that changed from the base of the PR and between c8be25d and 46863f4.

📒 Files selected for processing (4)

.agents/skills/nemoclaw-maintainer-policies/references/examples.md
.agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.json
.agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.md
.agents/skills/nemoclaw-maintainer-policies/references/triage-instructions.md

📝 Walkthrough

Walkthrough

This PR refines NemoClaw maintainer-policy documentation by tightening triage-label rules, expanding label-taxonomy guidance with richer positive/negative signal criteria, and adding concrete dry-run JSON examples that demonstrate the refined label-selection process for issue triage.

Changes

Maintainer Policy Refinements

Layer / File(s)	Summary
Triage workflow rules and label constraints `triage-instructions.md`, `examples.md`	Updated triage-instructions.md to forbid adding `needs: triage` or `status: triage` in normal triage output, refined Issue/PR Flow guidance to clarify when to use `needs: info` and `needs: design` vs placeholder labels, specified missing bug-report components that trigger info requests, and added minimal-labeling constraints. Replaced anti-example guidance in examples.md to reinforce the triage-label rules.
Label taxonomy and routing guidance `label-taxonomy.json`, `label-taxonomy.md`	Expanded positive/negative signal criteria for `pr_type`, `needs`, `area`, `platform`, and `integration` label families. Added explicit routing rules for `integration:` labels when a canonical integration name appears in issue/PR metadata, clarified when to use broad `area: integrations` vs specific `integration:` labels, enhanced platform-label evidence mapping (arm64, wsl), and expanded Hermes/OpenClaw artifact recognition.
Concrete triage examples `examples.md`	Added three new dry-run JSON triage examples: (1) Hermes TUI-missing integration issue with anti-examples showing overly broad labeling, (2) OpenClaw e2e failure with e2e/ci/bug classification, and (3) Windows ARM install failure in WSL with platform-specific labeling.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

NVIDIA/NemoClaw#4857: This PR refines the same maintainer-policy references introduced in #4857, tightening needs:* label rules and expanding label-taxonomy guidance with concrete examples.

Suggested labels

documentation, enhancement: policy

Suggested reviewers

cv
miyoungc

Poem

🐰 A rabbit tends the policy garden fair,
With clearer rules and guidance everywhere,
No triage tags shall clutter up the flow,
Just routing labels—now the path will show!
Examples bloom to light the triager's way,
A tidier taxonomy saves the day. 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title 'docs(policy): refine shared triage taxonomy' accurately summarizes the main change—refining documentation for the shared triage policy taxonomy used by maintainer agents.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/shared-triage-policy-v2-20260608

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-06-08T03:45:54Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. No E2E is recommended. The PR changes only maintainer-agent policy/reference documentation and taxonomy metadata used for issue/PR triage. It cannot affect runtime user flows such as install, onboarding, credentials, sandbox lifecycle, network policy, inference routing, deployment, or OpenClaw assistant behavior.

Optional E2E

None.

New E2E recommendations

None.

github-actions · 2026-06-08T03:45:55Z

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

None. Changed files are maintainer policy/agent reference documentation outside test/e2e-scenario/ and scenario workflows, with no direct scenario E2E runtime, catalog, metadata, expected-state, suite, onboarding helper, or workflow impact.

Optional scenario E2E

None.

Relevant changed files

None.

github-actions · 2026-06-08T03:47:21Z

PR Review Advisor

Findings: 0 needs attention, 3 worth checking, 1 nice ideas
Since last review: 0 prior items resolved, 3 still apply, 0 new items found

Review findings

🛠️ Needs attention

None.

🔎 Worth checking

Source-of-truth review needed: Policy reference consistency for PR type conventional-commit evidence: The advisor marked localized patch analysis as needs_followup.
- Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
- Evidence: label-taxonomy.json and triage-instructions.md include conventional-prefix mapping, while label-taxonomy.md lines 47-50 do not.
Human-facing PR type taxonomy still omits conventional-commit prefix signals (.agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.md:47): The PR adds conventional commit prefixes as PR type evidence in the machine-readable taxonomy and triage instructions, but the human-facing PR Type table still lists only the older positive signals. Since SKILL.md directs humans to label-taxonomy.md for label meaning and JSON for machine-readable validation, this leaves two policy sources out of sync.
- Recommendation: Update the PR Type table in label-taxonomy.md to document the same prefix mappings, for example fix -> bug-fix, feat -> feature, refactor -> refactor, and chore/docs-only/CI-only/generated policy maintenance -> chore.
- Evidence: label-taxonomy.json includes positive signals such as "fix conventional commit prefix", "feat conventional commit prefix", "refactor conventional commit prefix", and "chore conventional commit prefix". triage-instructions.md PR Flow maps those prefixes to PR type labels. label-taxonomy.md lines 47-50 still only mention regression/new command/cleanup/docs-only style signals.
Source-of-truth consistency still needs follow-up for PR type label evidence (.agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.md:47): The policy package is intended to remain the source of truth, but the same PR type behavior is currently described differently across references. This is not a localized workaround; it is source-of-truth drift between human-facing and machine-readable policy documents.
- Recommendation: Make label-taxonomy.md, label-taxonomy.json, and triage-instructions.md agree on PR type conventional-prefix evidence, and consider adding a consistency check so future policy edits cannot update one reference without the others.
- Evidence: SKILL.md says label-taxonomy.md is loaded for human-facing label meaning and label-taxonomy.json is loaded for machine-readable signal validation. The conventional-prefix mapping appears in JSON and triage instructions but not in the markdown PR Type table.

🌱 Nice ideas

Use unique item numbers in example dry-run payloads (.agents/skills/nemoclaw-maintainer-policies/references/examples.md:104): Two illustrative examples use item_number 104. This is not a runtime bug, but unique example IDs make the samples easier to copy into fixtures, compare in generated output, or discuss in reviews.
- Recommendation: Renumber either the Named Integration Issue example or the Daily Issue Tracking example so each sample item_number is unique.
- Evidence: examples.md contains item_number 104 in the new Named Integration Issue dry run and again in the Daily Issue Tracking dry run.

Consider writing more tests for

**Acceptance clause:** This PR keeps the policy package as the source of truth and tightens the taxonomy/routing guidance needed for the nvoss-velocity shared-policy migration: — add test evidence or identify existing coverage. Only policy reference files changed, but PR type conventional-commit evidence is present in triage-instructions.md and label-taxonomy.json while missing from the human-facing label-taxonomy.md PR Type table.
**Acceptance clause:** adds PR type evidence from conventional commit prefixes — add test evidence or identify existing coverage. triage-instructions.md maps fix/feat/refactor/chore-style prefixes to PR type labels, and label-taxonomy.json adds matching positive_signals. label-taxonomy.md does not yet expose those signals in the PR Type table.
**Acceptance clause:** This policy version was evaluated in the nvoss-velocity migration experiment using a deterministic 100-item NemoClaw fixture: 50 issues and 50 PRs, all numbered `fix: add local-inference policy preset for Ollama/vLLM host access (Fixes #693) #2000+`. — add test evidence or identify existing coverage. This is a PR-provided evaluation claim; the diff does not include the fixture or evaluation outputs for independent read-only verification.
**Acceptance clause:** Label F1: `0.485` — add test evidence or identify existing coverage. Metric is provided in the PR body only; no in-repository evaluation artifact changed in this PR verifies it.
**Acceptance clause:** Label precision: `0.402` — add test evidence or identify existing coverage. Metric is provided in the PR body only; no in-repository evaluation artifact changed in this PR verifies it.
**Policy reference consistency for PR type conventional-commit evidence** — Add a consistency check that asserts PR type prefix mappings in triage-instructions.md and label-taxonomy.json are also documented in the label-taxonomy.md PR Type table.. label-taxonomy.json and triage-instructions.md include conventional-prefix mapping, while label-taxonomy.md lines 47-50 do not.

Since last review details

Current findings:

Source-of-truth review needed: Policy reference consistency for PR type conventional-commit evidence: The advisor marked localized patch analysis as needs_followup.
- Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
- Evidence: label-taxonomy.json and triage-instructions.md include conventional-prefix mapping, while label-taxonomy.md lines 47-50 do not.
Human-facing PR type taxonomy still omits conventional-commit prefix signals (.agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.md:47): The PR adds conventional commit prefixes as PR type evidence in the machine-readable taxonomy and triage instructions, but the human-facing PR Type table still lists only the older positive signals. Since SKILL.md directs humans to label-taxonomy.md for label meaning and JSON for machine-readable validation, this leaves two policy sources out of sync.
- Recommendation: Update the PR Type table in label-taxonomy.md to document the same prefix mappings, for example fix -> bug-fix, feat -> feature, refactor -> refactor, and chore/docs-only/CI-only/generated policy maintenance -> chore.
- Evidence: label-taxonomy.json includes positive signals such as "fix conventional commit prefix", "feat conventional commit prefix", "refactor conventional commit prefix", and "chore conventional commit prefix". triage-instructions.md PR Flow maps those prefixes to PR type labels. label-taxonomy.md lines 47-50 still only mention regression/new command/cleanup/docs-only style signals.
Source-of-truth consistency still needs follow-up for PR type label evidence (.agents/skills/nemoclaw-maintainer-policies/references/label-taxonomy.md:47): The policy package is intended to remain the source of truth, but the same PR type behavior is currently described differently across references. This is not a localized workaround; it is source-of-truth drift between human-facing and machine-readable policy documents.
- Recommendation: Make label-taxonomy.md, label-taxonomy.json, and triage-instructions.md agree on PR type conventional-prefix evidence, and consider adding a consistency check so future policy edits cannot update one reference without the others.
- Evidence: SKILL.md says label-taxonomy.md is loaded for human-facing label meaning and label-taxonomy.json is loaded for machine-readable signal validation. The conventional-prefix mapping appears in JSON and triage instructions but not in the markdown PR Type table.
Use unique item numbers in example dry-run payloads (.agents/skills/nemoclaw-maintainer-policies/references/examples.md:104): Two illustrative examples use item_number 104. This is not a runtime bug, but unique example IDs make the samples easier to copy into fixtures, compare in generated output, or discuss in reviews.
- Recommendation: Renumber either the Named Integration Issue example or the Daily Issue Tracking example so each sample item_number is unique.
- Evidence: examples.md contains item_number 104 in the new Named Integration Issue dry run and again in the Daily Issue Tracking dry run.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

## Summary - Add the v0.0.61 release notes from the GitHub dev announcement. - Document managed vLLM recovery after host reboot and Slack denied-mention feedback. - Refresh generated `nemoclaw-user-*` skills from the source docs. ## Source summary - #4983 -> `docs/about/release-notes.mdx`: Added the v0.0.61 release summary from the dev announcement and linked behavior groups to deeper docs. - #4904 -> `docs/inference/use-local-inference.mdx`: Documented that managed vLLM restarts the `nemoclaw-vllm` container after host reboot during recovery. - #4933 -> `docs/manage-sandboxes/messaging-channels.mdx`: Documented Slack sender feedback for denied channel `@mention` events. - #4879, #4915, #4935, #4759, #4164, #4888, #4897, #4944, #4959 -> `.agents/skills/`: Refreshed generated user skills from the current source docs for release prep. ## Verification - `python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix nemoclaw-user --doc-platform fern-mdx` - `npm run docs` (passed outside the tool sandbox after `tsx` IPC pipe creation was blocked in the sandbox) - `npm run build:cli` (refreshed local `dist/` for the pre-push TypeScript hook) - Commit and pre-push hooks passed, including docs-to-skills verification, markdownlint, gitleaks, skills YAML tests, and CLI TypeScript.  ## Summary by CodeRabbit * **Documentation** * Updated sandbox security documentation with file descriptor limits. * Changed default inference model for DGX Station profile. * Enhanced agent policy and backup/restore documentation. * Improved command reference examples with clearer formatting. * Clarified Slack messaging denial notice behavior. * Added automatic vLLM container recovery during host reboot. * Updated release notes for v0.0.61.

docs(policy): refine shared triage taxonomy

46863f4

wscurran added v0.0.61 Release target area: skills Skills, agent behaviors, prompts, or skill packaging area: project-management Taxonomy, triage, workflow, roadmap, or project process area: docs Documentation, examples, guides, or docs build labels Jun 8, 2026

wscurran marked this pull request as ready for review June 8, 2026 03:49

wscurran requested a review from cv June 8, 2026 03:49

cv approved these changes Jun 8, 2026

View reviewed changes

cv merged commit ec408c8 into main Jun 8, 2026
34 checks passed

cv deleted the codex/shared-triage-policy-v2-20260608 branch June 8, 2026 05:44

miyoungc mentioned this pull request Jun 8, 2026

docs: refresh v0.0.61 release docs #4992

Merged

wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026

Conversation

wscurran commented Jun 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Evaluation

Validation

Notes

Summary by CodeRabbit

Uh oh!

copy-pr-bot Bot commented Jun 8, 2026

Uh oh!

coderabbitai Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Poem

Uh oh!

github-actions Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Scenario Advisor Recommendation

E2E Scenario Advisor

Required scenario E2E

Optional scenario E2E

Relevant changed files

Uh oh!

github-actions Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor

🛠️ Needs attention

🔎 Worth checking

🌱 Nice ideas

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wscurran commented Jun 8, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 8, 2026 •

edited

Loading

github-actions Bot commented Jun 8, 2026 •

edited

Loading

github-actions Bot commented Jun 8, 2026 •

edited

Loading

github-actions Bot commented Jun 8, 2026 •

edited

Loading