Skip to content

docs: add maturity scorecard mirror#91317

Merged
kevinslin merged 4 commits into
mainfrom
dev/kevinlin/maturity-scorecard-docs-copy
Jun 8, 2026
Merged

docs: add maturity scorecard mirror#91317
kevinslin merged 4 commits into
mainfrom
dev/kevinlin/maturity-scorecard-docs-copy

Conversation

@kevinslin

@kevinslin kevinslin commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

What problem does this PR solve?

  • OpenClaw did not have a repo-local copy of the maturity scorecard docs tree.
  • The docs publish sync would mirror any new docs subtree into openclaw/docs unless it was explicitly excluded.

Why does this matter now?

  • The maturity scorecard docs need to live in the OpenClaw repo without being published to the public docs mirror.

What is the intended outcome?

  • Add docs/maturity-scorecard/** to the OpenClaw repo.
  • Keep docs/maturity-scorecard/** out of openclaw/docs publish syncs.

What is intentionally out of scope?

  • No edits to the scorecard content itself beyond copying the existing tree.
  • No changes to docs navigation or public docs exposure.
  • No translation or publish-repo updates.

What does success look like?

  • docs/maturity-scorecard/** exists in the repo.
  • scripts/docs-sync-publish.mjs excludes and prunes docs/maturity-scorecard/** from publish sync output.

What should reviewers focus on?

  • Whether docs/maturity-scorecard/** is the right repo-local location for this material.
  • Whether the sync exclusion in scripts/docs-sync-publish.mjs is scoped correctly and cannot leak this subtree into openclaw/docs.

Linked context

Which issue does this close?

Closes #

Which issues, PRs, or discussions are related?

Related #

Was this requested by a maintainer or owner?

  • Requested directly by Kevin Lin.

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: keep a repo-local maturity scorecard docs tree in OpenClaw without syncing it to the publish docs repo.
  • Real environment tested: local OpenClaw worktree on macOS.
  • Exact steps or command run after this patch:
    • git diff --check
    • node --input-type=module -e 'await import("./scripts/docs-sync-publish.mjs"); console.log("script-ok");'
    • rsync -anic --delete /Users/kevinlin/code/claw/maintainers/docs/kevinslin/maturity-scorecard/ /Users/kevinlin/.worktrees/openclaw/dev/kevinlin/maturity-scorecard-docs-copy/docs/maturity-scorecard/
  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):
    • git diff --check returned clean.
    • Script import returned script-ok.
    • rsync -anic --delete ... returned no differences after the copy.
  • Observed result after fix:
    • docs/maturity-scorecard/** exists in the worktree.
    • scripts/docs-sync-publish.mjs now treats maturity-scorecard as a publish-excluded subtree alongside internal.
  • What was not tested:
    • Full end-to-end publish sync into an openclaw/docs checkout.
    • CI or remote docs publish jobs.
  • Proof limitations or environment constraints:
    • Validation was limited to local script parsing, diff hygiene, and source-vs-copy parity checks.
    • Remote publication was not exercised in this environment.
  • Before evidence (optional but encouraged):
    • Before the patch, OpenClaw had no docs/maturity-scorecard/** subtree, and the sync script did not exclude maturity-scorecard.

Tests and validation

Which commands did you run?

  • git diff --check
  • node --input-type=module -e 'await import("./scripts/docs-sync-publish.mjs"); console.log("script-ok");'
  • rsync -anic --delete /Users/kevinlin/code/claw/maintainers/docs/kevinslin/maturity-scorecard/ /Users/kevinlin/.worktrees/openclaw/dev/kevinlin/maturity-scorecard-docs-copy/docs/maturity-scorecard/

What regression coverage was added or updated?

  • No automated test coverage was added. This change is a docs-tree import plus a narrow publish-sync exclusion.

What failed before this fix, if known?

  • The subtree did not exist in OpenClaw before this change.
  • If added without a sync exclusion, publish sync would have mirrored it into openclaw/docs.

If no test was added, why not?

  • The touched behavior is isolated to docs content plus a small sync-script exclusion. Local validation covered syntax, diff hygiene, and source-copy parity; there was no existing focused test seam for this exact exclusion path.

Risk checklist

Did user-visible behavior change? (Yes/No)

  • No

Did config, environment, or migration behavior change? (Yes/No)

  • No

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

  • No

What is the highest-risk area?

  • Accidentally publishing docs/maturity-scorecard/** to the public docs mirror.

How is that risk mitigated?

  • scripts/docs-sync-publish.mjs now excludes maturity-scorecard/ during rsync and prunes it from the publish tree if present.

Current review state

What is the next action?

  • Review the copied docs subtree and confirm the publish-exclusion behavior is the intended long-term contract.

What is still waiting on author, maintainer, CI, or external proof?

  • Maintainer confirmation that this private docs subtree belongs in OpenClaw under docs/maturity-scorecard.
  • Optional end-to-end publish-sync verification if reviewers want live proof that the subtree stays out of openclaw/docs.

Which bot or reviewer comments were addressed?

  • None yet.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation scripts Repository scripts size: XS maintainer Maintainer-authored PR labels Jun 8, 2026
@clawsweeper

clawsweeper Bot commented Jun 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 8, 2026, 11:01 AM ET / 15:01 UTC.

Summary
The PR adds a repo-local maturity scorecard docs tree and updates the docs publish sync script so that tree is excluded from the public docs mirror.

Reproducibility: yes. this is source-reproducible by inspecting the latest PR head and diff for /Users/kevinlin, archive status fields, and personal SSH remote aliases. I did not need a runtime repro because the defect is in the committed docs content and publish tooling boundary.

Review metrics: 2 noteworthy metrics.

  • Private/local metadata exposure: 6,678 /Users/kevinlin matches across 479 files; 822 archive status-token fields; 393 personal SSH remote lines. These are not ordinary docs stats; they quantify the source-level privacy review needed before merge.
  • Publish exclusion surface: 1 publish-excluded docs directory added. The PR adds a new source subtree whose absence from the public docs mirror depends on the modified sync script.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Remove or generalize personal filesystem paths, local database paths, credential-presence fields, and personal remote aliases.
  • Get maintainer confirmation on whether this belongs in tracked docs/maturity-scorecard or a private/ignored docs location.
  • [P1] Add redacted real behavior proof for the docs-sync exclusion and prune behavior.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The PR body includes local command claims, but they only cover diff hygiene, script import/parsing, and copy parity; they do not exercise the docs-sync exclusion/prune path that this PR changes. A contributor should add redacted terminal output, logs, or an artifact from a full or focused docs-sync dry run, and update the PR body to trigger a fresh ClawSweeper review or ask a maintainer to comment @clawsweeper re-review.

Risk before merge

  • [P1] Merging as-is would expose private local filesystem paths, archive database locations, credential-presence booleans, and personal SSH remote aliases in the public OpenClaw source repository.
  • [P1] The publish exclusion has not been proven with an end-to-end docs-sync run or focused regression test, so the private subtree’s only public-docs guard is still review-time reasoning.
  • [P1] The PR is labeled maintainer and the body explicitly asks for confirmation that docs/maturity-scorecard is the intended long-term location.

Maintainer options:

  1. Sanitize and prove before merge (recommended)
    Remove private/local metadata from the tracked tree or move it private, then add a focused docs-sync proof or test showing maturity-scorecard is excluded and pruned.
  2. Accept public source exposure intentionally
    A maintainer can choose to merge only after confirming every local path, archive status line, and private remote reference is safe to publish in source.
  3. Pause or close as private docs
    If this is private operator material, pause or close this PR and keep the mirror in the existing private docs or ignored internal-docs workflow.

Next step before merge

  • [P1] Human maintainer judgment is needed on the permanent location and acceptable sanitization level before any automated repair would be safe.

Security
Needs attention: The diff introduces a concrete information-exposure concern by adding private local paths and archive status metadata to the public source tree.

Review findings

  • [P1] Remove private paths from the scorecard mirror — docs/maturity-scorecard/README.md:29-30
Review details

Best possible solution:

Sanitize the scorecard artifacts into repo-root paths/placeholders or move the material to a private/ignored docs location, then add focused proof that docs sync excludes and prunes the subtree before merge.

Do we have a high-confidence way to reproduce the issue?

Yes; this is source-reproducible by inspecting the latest PR head and diff for /Users/kevinlin, archive status fields, and personal SSH remote aliases. I did not need a runtime repro because the defect is in the committed docs content and publish tooling boundary.

Is this the best way to solve the issue?

No; excluding the subtree from openclaw/docs is only a plausible mitigation and does not address public source exposure. The narrower maintainable solution is to sanitize or relocate the mirror and then prove the publish exclusion with a focused sync check.

Full review comments:

  • [P1] Remove private paths from the scorecard mirror — docs/maturity-scorecard/README.md:29-30
    This representative line commits a personal archive path into tracked docs, and the same pattern appears thousands of times across the new tree, including local database paths, credential-presence fields, and personal SSH remote aliases. Because the OpenClaw source repo is public even when this subtree is excluded from the docs mirror, sanitize these to repo-root paths/placeholders or move the material to a private/ignored docs location before merge.
    Confidence: 0.96

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against e3ef136bca85.

Label changes

Label changes:

  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body includes local command claims, but they only cover diff hygiene, script import/parsing, and copy parity; they do not exercise the docs-sync exclusion/prune path that this PR changes. A contributor should add redacted terminal output, logs, or an artifact from a full or focused docs-sync dry run, and update the PR body to trigger a fresh ClawSweeper review or ask a maintainer to comment @clawsweeper re-review.
  • remove rating: 🌊 off-meta tidepool: Current PR rating is rating: 🧂 unranked krab, so this older rating label is no longer current.

Label justifications:

  • P2: This is a normal-priority docs/tooling improvement with limited runtime blast radius but meaningful repository hygiene and privacy risk.
  • merge-risk: 🚨 security-boundary: The PR would commit private local paths and archive tool-status metadata into public source.
  • merge-risk: 🚨 automation: The PR changes docs publish-sync exclusion behavior, and an error could leak or retain the scorecard subtree in the public docs mirror.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body includes local command claims, but they only cover diff hygiene, script import/parsing, and copy parity; they do not exercise the docs-sync exclusion/prune path that this PR changes. A contributor should add redacted terminal output, logs, or an artifact from a full or focused docs-sync dry run, and update the PR body to trigger a fresh ClawSweeper review or ask a maintainer to comment @clawsweeper re-review.
Evidence reviewed

Security concerns:

  • [medium] Private local and archive metadata in public source — docs/maturity-scorecard/inventory/agent-runtime-and-provider-execution/agent-turn-orchestration-and-runtime-lifecycle.md:26
    The new scorecard notes include local database paths, credential-presence booleans, and personal SSH remote aliases; these are not secret values, but they expose private operational metadata if merged into the public repository.
    Confidence: 0.95

What I checked:

Likely related people:

  • Jason (Json): Current-main blame shows this author introduced the docs guide, scripts guide, and the existing docs publish exclusion/prune code that this PR extends. (role: recent area contributor; confidence: high; commits: 57e0bdaabe0a; files: docs/AGENTS.md, scripts/AGENTS.md, scripts/docs-sync-publish.mjs)
  • Mason Huang: The latest history for the changed Chinese glossary file is a docs i18n preservation commit, making this person relevant for glossary churn review. (role: recent docs i18n contributor; confidence: medium; commits: 9a82b60024b5; files: docs/.i18n/glossary.zh-CN.json)
  • kevinslin: The PR body says the work was requested directly by Kevin Lin, and the imported scorecard content points to Kevin Lin's local maturity archive as its source. (role: scorecard artifact owner/proposer; confidence: medium; commits: 0c154a6ebf19, f6011e583960, c265af474459; files: docs/maturity-scorecard/**, scripts/docs-sync-publish.mjs)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jun 8, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 578dcf55ea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/maturity-scorecard/README.md Outdated
# Maturity scorecard process

This directory is an artifact root maintained by the local `claw-score` skill
at [SKILL.md](/Users/kevinlin/code/claw/maintainers/.agents/skills/claw-score/SKILL.md).

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replace the private absolute skill link

This absolute Markdown link is treated by scripts/docs-link-audit.mjs as an internal docs route, so docs link checks now fail in any checkout that does not have the author's /Users/kevinlin/... tree. I ran node scripts/docs-link-audit.mjs on this commit and it reports this line, plus the duplicate at line 213, as broken links; replace it with a repo-relative checked-in target or make it plain text.

Useful? React with 👍 / 👎.

Comment thread ...ventory/agent-runtime-and-provider-execution/provider-auth-profiles-and-credential-health.md

### Docs

- `/Users/kevinlin/code/openclaw/docs/cli/models.md` documents model status/auth overview, Codex OAuth troubleshooting, auth profile listing, login, paste-api-key, OpenAI API versus ChatGPT/OAuth, and Claude CLI notes.

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize scorecard evidence paths

Because this scorecard is committed under docs/maturity-scorecard rather than docs/internal, the docs guide's no-local-path rule applies; in any checkout other than the author's, /Users/kevinlin/... is neither navigable nor a stable repo reference. I checked with rg /Users/kevinlin docs/maturity-scorecard and this pattern appears throughout the new evidence notes, so consumers cannot verify the audit without rewriting paths; render these as repo-root paths such as docs/cli/models.md.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 8, 2026
@kevinslin kevinslin force-pushed the dev/kevinlin/maturity-scorecard-docs-copy branch from 5b4b7d0 to f73f0d4 Compare June 8, 2026 14:53
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels Jun 8, 2026
@kevinslin kevinslin merged commit 6cc6f5e into main Jun 8, 2026
152 of 155 checks passed
@kevinslin kevinslin deleted the dev/kevinlin/maturity-scorecard-docs-copy branch June 8, 2026 15:07
@github-actions github-actions Bot mentioned this pull request Jun 8, 2026
Open
@kevinslin kevinslin mentioned this pull request Jun 8, 2026
Merged
kevinslin added a commit that referenced this pull request Jun 8, 2026
@kevinslin
Revert "docs: add maturity scorecard mirror (#91317)" (#91508)
4c5d8af 
This reverts commit 6cc6f5e.
@clawsweeper clawsweeper Bot mentioned this pull request Jun 8, 2026
Merged
@github-actions github-actions Bot mentioned this pull request Jun 8, 2026
Open
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 9, 2026
@kevinslin
docs: add maturity scorecard mirror (openclaw#91317)
de9d983 
* docs: add maturity scorecard mirror

* docs: format maturity scorecard mirror

* docs: drop stray maturity note

* docs: fix maturity scorecard docs checks
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 9, 2026
@kevinslin
Revert "docs: add maturity scorecard mirror (openclaw#91317)" (opencl…
4da94f6 
…aw#91508)

This reverts commit de9d983.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

Assignees

No one assigned

Labels

docs Improvements or additions to documentation maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. scripts Repository scripts size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kevinslin