Skip to content

fix(e2e): skip lockfiles in snapshot leak scan#3617

Merged
cv merged 6 commits into
mainfrom
fix-snapshot-lockfile-false-positive
May 15, 2026
Merged

fix(e2e): skip lockfiles in snapshot leak scan#3617
cv merged 6 commits into
mainfrom
fix-snapshot-lockfile-false-positive

Conversation

@jyaunches

@jyaunches jyaunches commented May 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a shared helper that defines which snapshot files should be scanned for credential-looking payloads
  • exclude dependency lockfiles from the coarse snapshot E2E credential grep
  • add regression coverage for lockfile exclusion while preserving JSON/env scan coverage

Root cause

Nightly snapshot-commands-e2e failed in runs 25931606052 and 25933526098 because Phase 8 scans all *.json files under snapshot backups for broad patterns like sk-. Dependency lockfiles such as extensions/openclaw-weixin/package-lock.json can contain non-secret package metadata matching those patterns, causing a false positive.

Validation

  • npm test -- --run src/lib/credential-filter.test.ts
  • bash -n test/e2e/test-snapshot-commands.sh
  • npm test -- --run test/snapshot.test.ts src/lib/credential-filter.test.ts
  • npm run build:cli

Fixes nightly E2E snapshot false positive observed in runs 25931606052 and 25933526098.

Summary by CodeRabbit

  • Tests

    • Enhanced end-to-end credential-leak scanning: now skips common dependency lockfiles and more precisely targets environment and JSON snapshot files to reduce false positives.
  • Chores

    • Adjusted CI workflow install step to stabilize dependency resolution timing for E2E runs.

Review Change Stack

jyaunches added 2 commits May 15, 2026 14:29
This test demonstrates the nightly snapshot failure where dependency lockfiles can match broad credential patterns.
Expected: runtime config/env files are scanned, dependency lockfiles are skipped.
Actual: no shared helper exists, so lockfiles are scanned by the E2E script.
Dependency lockfiles can contain non-secret package metadata that matches broad credential patterns such as sk-. Keep scanning runtime JSON and env files, but exclude lockfiles from the coarse snapshot E2E leak check.
@coderabbitai

coderabbitai Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6841e50e-5d61-490e-9be6-1cfd11eb6a19

📥 Commits

Reviewing files that changed from the base of the PR and between 4373a3f and 87c1dbc.

📒 Files selected for processing (2)
  • src/lib/security/credential-filter.test.ts
  • src/lib/security/credential-filter.ts

📝 Walkthrough

Walkthrough

Exclude common package-manager lockfiles from snapshot credential scans, add shouldScanSnapshotFileForCredentials with basename-based exclusions and tests, update the Phase 8 E2E scan script accordingly, and add an npm install --before timestamp to the E2E Advisor workflow.

Changes

E2E Infrastructure

Layer / File(s) Summary
Credential filter implementation and tests
src/lib/security/credential-filter.ts, src/lib/security/credential-filter.test.ts
Adds basename import, SNAPSHOT_CREDENTIAL_SCAN_EXCLUDED_BASENAMES, and exported shouldScanSnapshotFileForCredentials(filename); tests validate env/json scanning and lockfile basename exclusions.
E2E snapshot scan script update
test/e2e/test-snapshot-commands.sh
Phase 8 credential-leak detection find/grep pipeline rewritten to ignore common lockfiles (package-lock.json, npm-shrinkwrap.json, yarn.lock, pnpm lockfiles) while keeping existing credential pattern greps.
Workflow installation timing constraint
.github/workflows/e2e-advisor.yaml
Pi SDK install step adds npm install --before=2026-05-14T00:00:00.000Z to constrain dependency resolution timing.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#3504: Updates to the E2E Advisor workflow's Pi SDK installation step overlap with changes to npm install behavior and the @earendil-works/pi-coding-agent usage.

Suggested labels

E2E

Poem

🐰 I hop through snapshots, sniff each tiny line,
Lockfiles left untouched, env secrets I mind,
A timestamp for installs keeps SDKs in time,
Tests nod and pass, the pipeline sings fine. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(e2e): skip lockfiles in snapshot leak scan' directly describes the main change: excluding dependency lockfiles from credential scanning in E2E snapshot tests.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-snapshot-lockfile-false-positive

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/lib/credential-filter.test.ts (1)

157-162: ⚡ Quick win

Add lockfile-variant and path-form regression assertions.

Please also cover pnpm-lock.yml and a path input case so basename-based exclusion stays guarded.

Suggested test additions
   it("skips dependency lockfiles that can contain non-secret package metadata matches", () => {
     expect(shouldScanSnapshotFileForCredentials("package-lock.json")).toBe(false);
     expect(shouldScanSnapshotFileForCredentials("npm-shrinkwrap.json")).toBe(false);
     expect(shouldScanSnapshotFileForCredentials("yarn.lock")).toBe(false);
     expect(shouldScanSnapshotFileForCredentials("pnpm-lock.yaml")).toBe(false);
+    expect(shouldScanSnapshotFileForCredentials("pnpm-lock.yml")).toBe(false);
+    expect(shouldScanSnapshotFileForCredentials("/tmp/package-lock.json")).toBe(false);
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/credential-filter.test.ts` around lines 157 - 162, Add assertions to
the existing test that verify shouldScanSnapshotFileForCredentials also excludes
the alternative pnpm lockfile name "pnpm-lock.yml" and that basename-based
exclusion still works when a full/relative path is supplied (e.g.,
"path/to/package-lock.json" should return false). Update the test block in
credential-filter.test.ts where shouldScanSnapshotFileForCredentials is
exercised to include these two additional expect checks so both filename
variants and path inputs are covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/credential-filter.ts`:
- Around line 173-177: The function shouldScanSnapshotFileForCredentials
lowercases the full path instead of the file's basename so
SNAPSHOT_CREDENTIAL_SCAN_EXCLUDED_BASENAMES won't match paths like
/tmp/package-lock.json; change the implementation to first obtain the basename
(use Node's path.basename on filename), then lowercase that basename and use it
for the exclusion check
(SNAPSHOT_CREDENTIAL_SCAN_EXCLUDED_BASENAMES.has(basename)) and for the
subsequent checks (basename === ".env" || basename.endsWith(".env") ||
basename.endsWith(".json")). Ensure you import/require path if not already
present.

---

Nitpick comments:
In `@src/lib/credential-filter.test.ts`:
- Around line 157-162: Add assertions to the existing test that verify
shouldScanSnapshotFileForCredentials also excludes the alternative pnpm lockfile
name "pnpm-lock.yml" and that basename-based exclusion still works when a
full/relative path is supplied (e.g., "path/to/package-lock.json" should return
false). Update the test block in credential-filter.test.ts where
shouldScanSnapshotFileForCredentials is exercised to include these two
additional expect checks so both filename variants and path inputs are covered.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2ba309cc-8e0d-44a7-964f-43a5a5847c32

📥 Commits

Reviewing files that changed from the base of the PR and between 0964a7e and 0d1623d.

📒 Files selected for processing (3)
  • src/lib/credential-filter.test.ts
  • src/lib/credential-filter.ts
  • test/e2e/test-snapshot-commands.sh

Comment thread src/lib/security/credential-filter.ts
@github-actions

github-actions Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: credential-sanitization-e2e, snapshot-commands-e2e
Optional E2E: state-backup-restore-e2e, credential-migration-e2e

Dispatch hint: credential-sanitization-e2e,snapshot-commands-e2e

Auto-dispatched E2E: credential-sanitization-e2e, snapshot-commands-e2e via nightly-e2e.yaml at 87c1dbc4c920111144fd30ffcb2fe8fc6b747affnightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • credential-sanitization-e2e (high): Validates credential stripping, auth file deletion, symlink safety, and runtime sandbox credential absence after touching shared credential-filter security code.
  • snapshot-commands-e2e (medium): Directly exercises the modified snapshot create/list/restore E2E script and its snapshot credential-leak check, including the new lockfile exclusions.

Optional E2E

  • state-backup-restore-e2e (high): Adjacent confidence for backup/restore behavior because credential-filter is used by sandbox backup sanitization paths, but the functional diff is primarily scan selection and is already covered by snapshot/credential sanitization E2Es.
  • credential-migration-e2e (medium): Useful extra coverage for credential migration and gateway credential handling when shared credential/security files are touched, but this PR does not appear to alter the migration flow directly.

New E2E recommendations

  • snapshot credential scan parity (medium): The TypeScript helper shouldScanSnapshotFileForCredentials and the shell E2E find exclusions now encode similar lockfile policy separately. Existing E2E validates the shell behavior, but not that runtime/helper policy remains in sync if future code starts using it.
    • Suggested test: Add a focused E2E or scenario assertion that creates snapshot files including package-lock.json plus real .env/openclaw.json credentials and verifies lockfile-like false positives are ignored while real runtime credential leaks still fail.

Dispatch hint

  • Workflow: nightly-e2e.yaml
  • jobs input: credential-sanitization-e2e,snapshot-commands-e2e

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 25935470561
Target ref: 10f3a12ee0b5c12743434c4dfd32b7c147b4e7ab
Workflow ref: main
Requested jobs: snapshot-commands-e2e,credential-sanitization-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success
snapshot-commands-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 25935944925
Target ref: 4373a3ffc9d0f6d0e8f3a59b8f2f745883f3cfe2
Workflow ref: main
Requested jobs: snapshot-commands-e2e,credential-sanitization-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success
snapshot-commands-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 25936516884
Target ref: 87c1dbc4c920111144fd30ffcb2fe8fc6b747aff
Workflow ref: main
Requested jobs: credential-sanitization-e2e,snapshot-commands-e2e
Summary: 2 passed, 0 failed, 0 skipped

Job Result
credential-sanitization-e2e ✅ success
snapshot-commands-e2e ✅ success

@jyaunches jyaunches requested a review from cv May 15, 2026 20:42
@cv cv merged commit 9b7dd6d into main May 15, 2026
28 checks passed
@wscurran wscurran added the bug-fix PR fixes a bug or regression label Jun 8, 2026
@jyaunches jyaunches deleted the fix-snapshot-lockfile-false-positive branch June 12, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants