chore: Add tests documenting error invariants#62992
Conversation
There was a problem hiding this comment.
TIL about rapid 👁️
I think this is a nice-to-have demonstration, but personally I feel like reading this would not strictly help me understand the error behaviours better. Could we include some more verbose docstrings that:
- Explain what
invariants_test.gois for in the first place - Explain each of the helpers in detail
- Explain the behaviours being tested better (
"payloadLessStruct"doesn't seem very explanatory to me)
It might be nice to incorporate coverage of our existing error constructs (like our custom multi-error) as well
I think some of this will make more sense once I add tests with the other types. But yes, I'll add more descriptions.
|
There was a problem hiding this comment.
Q: Does this behavior actually makes sense? Based on my understanding of the expected behavior of HasType this seems wrong.
There was a problem hiding this comment.
Don't want to ignore the comment so just chiming in to say I am unsure 😅 IMO I don't really understand the use case for HasType; if there isn't one, I'm inclined to suggest we do not export it from lib/errors and discourage its use, to keep things simple
There was a problem hiding this comment.
To me, it seems like the implementation of HasType is wrong. It ought to instead be something along the lines of:
typ := reflect.TypeOf(t)
zero := reflect.Zero(typ)
return errors.As(err, &zero)
Or perhaps, the bug is with our multiError type, not sure.
There was a problem hiding this comment.
I think I tracked down the cause of this.
For a multi-error, we mark the last element as the Cause or result of Unwrap. code
func (e *multiError) Cause() error { return e.errs[len(e.errs)-1] }
func (e *multiError) Unwrap() error { return e.errs[len(e.errs)-1] }HasType calls If which calls UnwrapOnce inside its predicate function while doing a depth-first traversal, which has this:
func UnwrapOnce(err error) (cause error) {
switch e := err.(type) {
case interface{ Cause() error }:
return e.Cause()
case interface{ Unwrap() error }:
return e.Unwrap()
}
return nil
}Notice this comment:
// UnwrapOnce treats multi-errors (those implementing the
// `Unwrap() []error` interface as leaf-nodes since they cannot
// reasonably be iterated through to a single cause
This means that HasType will only compare against the last element in a multiError. 🤦🏽
In contrast, Is handles multi-errors explicitly, and As also handles multi-errors explicitly.
There was a problem hiding this comment.
With a slightly different API, here's an alternate implementation of HasType that doesn't suffer from the same problem. It seems like we should be able to tweak this using some comby-fu.
func HasType[T error](err error) bool {
var zero T
return As(err, &zero)
}
There was a problem hiding this comment.
IMO I don't really understand the use case for HasType
The use case, as I understand it, is "is this error type present in the tree" without caring about the actual value.
There was a problem hiding this comment.
Branch fixing the implementation of HasType. https://github.com/sourcegraph/sourcegraph/tree/vg/replace-HasType
Will create a PR with that after merging this.
20e4e5b to
d1e2c88
Compare
d1e2c88 to
5fcbcfe
Compare
5fcbcfe to
cfecca1
Compare
There was a problem hiding this comment.
Is it ignored because you want to be selective toward what gets committed here, amongst what it finds, or is that because we'll always intend this folder content to be ignored?
There was a problem hiding this comment.
I've added a more detailed comment here, does that answer your question?
# Generated by rapid after finding counter-examples.
#
# When re-running tests locally during development,
# this enables faster failure finding.
# https://github.com/flyingmutant/rapid/commit/4db076b4aa4d41036416da11abd5bbfc228a324e
#
# During development, if a check fails, it makes sense
# to disable or refine the check, and add specific,
# minimal tests using the counter-examples found by rapid.
There was a problem hiding this comment.
See https://github.com/sourcegraph/sourcegraph/pull/62992/files#r1622408899
If this is always going to be filled with examples generated locally, then it'll always end up empty in CI.
Additionally, if the folder doesn't exist, the glob eval will break in CI. Best to add a .gitkeep in there if you truly want to keep it.
There was a problem hiding this comment.
Hmm, not sure why this got added, I have no memory of modifying any BUILD files myself. I'll remove this and see if something breaks. AIUI, the only purpose is to record failures, so next time it can try them quickly.
However, in CI, the test won't be re-run, so it doesn't make sense to save the file.
There was a problem hiding this comment.
85ad33f to
0d8f90e
Compare
|
Going to merge this for now; can address other comments post-merge. |
Based on the discussion in Slack, I thought it would be useful to solidify the invariants that hold (and provide counter-examples for things which you think ought to hold but don't actually hold) in code itself, instead of just via docs. So this PR adds property-based tests for quickly generating different kinds of error shapes, and check which invariants hold and which ones don't.
TODO:
Is,AsandHasType.Test plan
Adds new tests