github: add ability to exclude repositories based on size & stars#58377
Conversation
8b6f084 to
b7bb2f9
Compare
b7bb2f9 to
2bf6403
Compare
f3540b7 to
8a67c37
Compare
17a675e to
6b556cb
Compare
6b556cb to
e598ed5
Compare
|
Codenotify: Notifying subscribers in CODENOTIFY files for diff fcc502b...ae67759.
|
| size, err := bytesize.Parse(sizeMatch[2]) | ||
| if err != nil { | ||
| return nil, err | ||
| } |
There was a problem hiding this comment.
Should we special case 0 here, or make it return a pointer to signify absence of data, in case some repo doesn't have it or github messes up?
There was a problem hiding this comment.
Yes! I forgot that, thanks! Well, if I understood it correctly: this does not parse the size of the repo, but what's in the rule. That is already covered by the regex.
But I changed the evaluation to ensure that repos with size == 0 will not get excluded, because that's a special case. And turns out that even empty repositories have 1 from what I've seen.
| const ( | ||
| opLess operator = "<" | ||
| opLessOrEqual operator = "<=" | ||
| opGreater operator = ">" | ||
| opGreaterOrEqual operator = ">=" | ||
| ) | ||
|
|
||
| func newOperator(input string) (operator, error) { | ||
| if input != "<" && input != "<=" && input != ">" && input != ">=" { | ||
| return "", errors.Newf("invalid operator %q", input) | ||
| } | ||
| return operator(input), nil | ||
| } | ||
|
|
||
| func (o operator) Eval(left, right int) bool { |
There was a problem hiding this comment.
FYI, there is a great library https://github.com/antonmedv/expr I've used in Cloud tooling that allows you to write expressions like this with variables, custom functions etc, though it might make sense to roll your own to keep the functionality specific
There was a problem hiding this comment.
Ha, yeah that's a full-on evaluation framework. I think I'll stick with my little fucntion here :)
There was a problem hiding this comment.
Expanding the thinking a bit more: a full evaluation framework you can use a struct as a variable, e.g. Repo with properties about it, which would enable crazy things like "only include repos that start with foo and have label bar and have over 100 stars but less than 150 stars, and also has an odd number of characters in the repo name, and is >0MB and <100MB in size"... in case anyone ever asks for that 😁
This adds the ability to exclude GitHub repositories based their
starsorsize.Main goal: add
exclude: [{"size": "> 1GB", "stars": "< 100"}]to our configuration on dotcom to free up disk space there.It does that by extending the
excludeschema of the GitHub code host connection in the following ways:starsthat accepts a conditional expression. Examples:< 500,>100,<= 2000,>=3333sizethat accepts a conditional expression and multiple size units (see code, docs will follow). Examples:>= 500 MB,<= 4000 KiB,> 1GB,<22 MiBANDed together. But for now this only works forsize/stars. I want to clean this up in a follow-up PR but want to get this one out so I can update the code host connection on dotcom ASAP.Limitations
Example config
Test plan