Skip to content

Restrict walking sibling folders when using a pattern#20263

Merged
RobinMalfait merged 3 commits into
mainfrom
fix/issue-20255
Jun 19, 2026
Merged

Restrict walking sibling folders when using a pattern#20263
RobinMalfait merged 3 commits into
mainfrom
fix/issue-20255

Conversation

@RobinMalfait

@RobinMalfait RobinMalfait commented Jun 19, 2026

Copy link
Copy Markdown
Member

This PR fixes an issue where a @source pointing to a concrete file could result in scanning the entire parent folder instead of only looking for the file we are actually interested in.

When we optimize a @source, we move all the static parts of the pattern to the base. This means that a @source like this:

@source "../../app.config.ts";

Resolves to:

SourceEntry::Pattern { base: "/Users", pattern: "/app.config.ts" }

When walking the base, we would only emit an !app.config.ts rule. This means that everything in the /Users folder is still walked, and the result is then thrown away. If you take a look at the gitignore equivalent (which is what we build behind the scenes), then we would essentially create the following:

!app.config.ts

But if you know how gitignore files work, then you know that this does force app.config.ts to not be ignored, but it doesn't say anything about all the other files/folders.

The fix is to restrict the base so that we ignore everything in the folder, and then explicitly re-include only the pattern we care about. Fixing this would result in the following:

*
!foo.ts

In the reproduction from #20255 this takes the build from appearing to hang (~22s) down to ~20ms on my machine.

There are a few edge cases we have to be careful about:

Multiple patterns for the same base. If we have multiple @source directives for the same folder:

@source "./src/foo.ts";
@source "./src/bar.ts";

Then blindly emitting * for each one would result in this .gitignore equivalent:

*
!foo.ts
*
!bar.ts

Notice that the second * would end up ignoring foo.ts again. To avoid this, we only emit the * rule once per base.

Dynamic parts in intermediate folders. If the pattern still contains a * in one of its folders:

@source "./src/ba*/*.html";

This resolves to:

SourceEntry::Pattern { base: "/src", pattern: "/ba*/*.html" }

If we now inject the * rule for /src, then we would never walk the ba* folders (e.g. bar or baz). To fix this, we add inverse rules for each parent segment of the pattern:

*                      ignore everything
!/ba*/                 except for the `ba*/` folders, so we walk into them
!/ba*/*.html           then scan the `*.html` files in them

Bases already covered by a broader source. If the base is already included (or nested) under an unrestricted source (an Auto/External source, or a Pattern containing **), then we leave it alone. Restricting it would incorrectly hide siblings that the broader source is supposed to pick up. For example:

@source "**/*";
@source "./src/components/button.html";

Here the **/* source should keep auto-detecting every file, so we must not restrict src/components just because there's a more specific @source pointing to it.

Source order is preserved throughout, so later @source not … rules can still override an earlier restricted source.

Fixes: #20255

Test plan

  1. Added unit tests realted to this sources logic
  2. Added integration like tests for the scanner itself
  3. All other tests should pass as-s
  4. Should work on each OS [ci-all]

@RobinMalfait RobinMalfait requested a review from a team as a code owner June 19, 2026 16:34
@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 4e934da5-6a36-42b2-a26f-dc62d0841960

📥 Commits

Reviewing files that changed from the base of the PR and between 60d8a95 and 5abd92d.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • crates/oxide/src/scanner/sources.rs
  • crates/oxide/tests/scanner.rs
✅ Files skipped from review due to trivial changes (1)
  • CHANGELOG.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/oxide/tests/scanner.rs
  • crates/oxide/src/scanner/sources.rs

Walkthrough

crates/oxide/src/scanner/sources.rs gains a new internal expand_restricted_patterns function that rewrites Pattern source entries into gitignore-style sequences: a catch-all Ignored { base, pattern: "*" } entry, parent-directory exception Ignored entries (!/<dir>/), and the original source entry. This expansion is skipped for auto/external bases and patterns containing **. The function is wired into the return path of public_source_entries_to_private_source_entries. Unit tests and scanner integration tests are added to validate the expansion logic, edge cases with @source not overrides, and interaction with gitignore-excluded directories. A CHANGELOG entry records the fix.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: restricting directory walking when using source patterns to prevent scanning unrelated sibling folders.
Description check ✅ Passed The description thoroughly explains the issue, the fix with gitignore semantics, edge cases handled, and provides concrete examples demonstrating the performance improvement from ~22s to ~20ms.
Linked Issues check ✅ Passed The PR fully addresses #20255 by implementing a fix that prevents the scanner from traversing the entire /Users directory when encountering @source "../../app.config.ts", resolving the hang issue.
Out of Scope Changes check ✅ Passed All changes are directly related to fixing the performance issue in #20255. Code modifications to sources.rs implement pattern restriction logic, tests validate the fix, and CHANGELOG documents the correction.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Confidence Score: 5/5

Safe to merge — the change is well-scoped, the logic is correct, and the extensive test suite covers the described edge cases including interleaved sources, wildcard intermediate segments, negation ordering, and coexistence with broad auto-detect sources.

The expand_restricted_patterns function correctly deduplicates the * restriction rule via restricted_roots, correctly generates !/subdir/ allow rules for wildcard intermediate path segments, and correctly skips restriction when a broader source already covers the base. The layered-gitignore approach in create_walker handles interleaved bases naturally because later groups layer on top and last-match-wins semantics apply. No paths through the new code produce incorrect file inclusion or exclusion.

No files require special attention.

Reviews (4): Last reviewed commit: "update changelog" | Re-trigger Greptile

Comment thread crates/oxide/src/scanner/sources.rs
Comment thread PR.md Outdated
@RobinMalfait RobinMalfait force-pushed the fix/issue-20255 branch 2 times, most recently from ee3b3b9 to 60d8a95 Compare June 19, 2026 17:40
@RobinMalfait RobinMalfait merged commit 0fee7b5 into main Jun 19, 2026
41 of 42 checks passed
@RobinMalfait RobinMalfait deleted the fix/issue-20255 branch June 19, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tailwind CSS 4.3.1 hangs in source scanner with @source "../../app.config.ts" used by Docus

1 participant