Skip to content

Filter out repositories with .batchignore at root#509

Merged
mrnugget merged 5 commits into
mainfrom
mrn/batchignore-repo-root
Apr 9, 2021
Merged

Filter out repositories with .batchignore at root#509
mrnugget merged 5 commits into
mainfrom
mrn/batchignore-repo-root

Conversation

@mrnugget

@mrnugget mrnugget commented Apr 8, 2021

Copy link
Copy Markdown
Contributor

This implements https://github.com/sourcegraph/sourcegraph/issues/18330 by querying the locations of .batchignore files in each repository yielded by the on attribute in a batch spec. If locations were found, the repository is ignored.

This can be overwritten by using the -force-override-ignore flag.

Example: given my instance has the following repositories:

  • github.com/sourcegraph-testing/zap
  • github.com/sourcegraph-testing/titan
  • github.com/sourcegraph-testing/tidb
  • github.com/sourcegraph-testing/etcd
  • github.com/sourcegraph-testing/batch-changes-testing-ignore

And I use the following batch spec:

on:
  - repositoriesMatchingQuery: repohasfile:README.md repo:sourcegraph-testing

steps:
  - run: echo "a horse says 'hello'" >> README.md
    container: alpine:3

with this change the batch-changes-testing-ignore repository will be ignored:

  • no archive will be downloaded
  • no steps executed

A message is printed that says it's ignored.

This implements https://github.com/sourcegraph/sourcegraph/issues/18330
in the simplest possible way by modifying all
`repositoriesMatchingQuery` queries to have
`-repohasfile:^.batchignore$` in them.

Given my instance has the following repositories:

- `github.com/sourcegraph-testing/zap`
- `github.com/sourcegraph-testing/titan`
- `github.com/sourcegraph-testing/tidb`
- `github.com/sourcegraph-testing/etcd`
- `github.com/sourcegraph-testing/batch-changes-testing-ignore`

And I use the following batch spec:

```yaml
on:
  - repositoriesMatchingQuery: repohasfile:README.md repo:sourcegraph-testing

steps:
  - run: echo "a horse says 'hello'" >> README.md
    container: alpine:3
```

with this change the `batch-changes-testing-ignore` repository will be
ignored:

- no archive will be downloaded
- no steps executed

It's simply filtered out of the search results.

But I can manually add it:

```yaml
on:
  - repositoriesMatchingQuery: repohasfile:README.md repo:sourcegraph-testing
  - repository: github.com/sourcegraph-testing/batch-changes-testing-ignore
```
@mrnugget mrnugget requested review from a team and malomarrec April 8, 2021 10:13
@mrnugget mrnugget closed this Apr 8, 2021
@mrnugget

mrnugget commented Apr 8, 2021

Copy link
Copy Markdown
Contributor Author

Closing this for now, since I discussed this with @malomarrec on a Zoom call right now. Outcome: we don't want to exclude those repositories from the search results, because we want to print which repositories were ignored. See https://github.com/sourcegraph/sourcegraph/issues/18330#issuecomment-815752987

@mrnugget

mrnugget commented Apr 8, 2021

Copy link
Copy Markdown
Contributor Author

Turns out that the functionality we built for monorepo support — a method that returns the locations of a file in a list of repositories — can be used here: instead of adding a -repohasfile:.batchignore filter to the search query I changed the code to search the locations of .batchignore files in all the yielded repositories.

screenshot_2021-04-08_15.05.48.mp4

@mrnugget mrnugget reopened this Apr 8, 2021

@LawnGnome LawnGnome left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is surprisingly clean! A real testament to your design work when implementing the file matching in the first place, I think.

This would need documentation and a changelog, of course, but otherwise I think this is good to go.

Comment thread cmd/src/batch_common.go Outdated
Comment thread cmd/src/batch_common.go Outdated
Co-authored-by: Adam Harvey <aharvey@sourcegraph.com>
Comment thread internal/batches/graphql/repository.go Outdated
@mrnugget mrnugget merged commit 365eef2 into main Apr 9, 2021
@mrnugget mrnugget deleted the mrn/batchignore-repo-root branch April 9, 2021 09:45
scjohns pushed a commit that referenced this pull request Apr 24, 2023
This implements https://github.com/sourcegraph/sourcegraph/issues/18330 by querying the locations of `.batchignore` files in each repository yielded by the `on` attribute in a batch spec. If locations were found, the repository is ignored.

This can be overwritten by using the `-force-override-ignore` flag.

Example: given my instance has the following repositories:

- `github.com/sourcegraph-testing/zap`
- `github.com/sourcegraph-testing/titan`
- `github.com/sourcegraph-testing/tidb`
- `github.com/sourcegraph-testing/etcd`
- `github.com/sourcegraph-testing/batch-changes-testing-ignore`

And I use the following batch spec:

```yaml
on:
  - repositoriesMatchingQuery: repohasfile:README.md repo:sourcegraph-testing

steps:
  - run: echo "a horse says 'hello'" >> README.md
    container: alpine:3
```

with this change the `batch-changes-testing-ignore` repository will be ignored:

- no archive will be downloaded
- no steps executed

A message is printed that says it's ignored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants