Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Exclude files in search.largeFiles#45318

Merged
gl-srgr merged 3 commits into
mainfrom
garyl/negate_pattern_searchLargeFiles
Dec 9, 2022
Merged

Exclude files in search.largeFiles#45318
gl-srgr merged 3 commits into
mainfrom
garyl/negate_pattern_searchLargeFiles

Conversation

@gl-srgr

@gl-srgr gl-srgr commented Dec 7, 2022

Copy link
Copy Markdown
Contributor

The site config setting search.largeFiles should allow file exclusions so that known large files can be excluded from the index.

Zoekt PR: sourcegraph/zoekt#480

This PR introduces extra steps in filter for handling interpretation of largeFiles patterns. Although doublestar still handles parsing for us we now handle additional cases:

  1. a pattern is prefixed by !: We use this character to define negation. Any file matching the pattern following the ! is treated as not being on the large files allow list, and is skipped (same as it would be if no pattern in the largeFiles patterns list matched the file). We honor the order of the patterns in the list. That is, if the file matches multiple patterns in the list then the latest entry is honored and it overrides whatever the preceding matches behavior would have been.

  2. a pattern prefixed with an escape '': No interpretation of the ! so no negation is applied. for example: "\!file.txt" will allow us to not skip the file named "!file.txt".

GH Issue: https://github.com/sourcegraph/sourcegraph/issues/40688

Test plan

  • updated filter tests

… to pattern to negate. Acknowledge escaped '!' as literal.
@sourcegraph-bot

sourcegraph-bot commented Dec 7, 2022

Copy link
Copy Markdown
Contributor

Codenotify: Notifying subscribers in CODENOTIFY files for diff 0cac134...c614a74.

Notify File(s)
@beyang cmd/searcher/internal/search/filter.go
cmd/searcher/internal/search/store_test.go
internal/search/backend/index_options_test.go
@camdencheek internal/search/backend/index_options_test.go
@keegancsmith cmd/searcher/internal/search/filter.go
cmd/searcher/internal/search/store_test.go
internal/search/backend/index_options_test.go

@gl-srgr gl-srgr requested a review from a team December 7, 2022 06:25
if strings.HasPrefix(pattern, negate) {
return true, pattern[len(negate):]
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I would strip the escaped backslash here but doublestar handles that for us

@keegancsmith

Copy link
Copy Markdown
Member

Once this all lands, please add a changelog. I think branch cut is soonish, so try and land soon so customers get extra christmas presents :)

@gl-srgr gl-srgr merged commit 535eb04 into main Dec 9, 2022
@gl-srgr gl-srgr deleted the garyl/negate_pattern_searchLargeFiles branch December 9, 2022 00:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants