SyncExternalService fails fast if code host does not seem available#44922
Conversation
|
Codenotify: Notifying subscribers in CODENOTIFY files for diff cdf3710...123526a.
|
| } | ||
|
|
||
| if !src.IsAvailable(ctx) { | ||
| return errors.New("code host does not seem to be available") |
There was a problem hiding this comment.
Maybe we should include more context? Perhaps IsAvailable can return an error instead? In that case, maybe we should call it something like CheckConnection?
We should also be careful with how we check the validity of the connection, we probably only want to fail on networking issues. If the issue is caused by an invalid token for example, this is a real failure that should result in repos being removed.
There was a problem hiding this comment.
One thing we can probably do is lookup the IP of the codehost to validate that DNS is working correctly as a first check: https://jameshfisher.com/2017/08/03/golang-dns-lookup/
There was a problem hiding this comment.
I'll put this comment in the issue. @varsanojidan
Yes, I was thinking something like dialing host-port over TCP:
timeout := 1 * time.Second
conn, err := net.DialTimeout("tcp",fmt.Sprintf("%s:433", host), timeout)
if err != nil {
return fmt.Errorf("Site unreachable, error: ", err)
}
conn.Close()
But actually starting with a DNS lookup and then doing this makes a lot of sense.
@ryanslade thank you for the comment - indeed it makes sense to return some meaningful error message here, and then name change you proposed seems very fitting
There was a problem hiding this comment.
This is done now. If you do not mind, I would like to land this with the
func (...) CheckConnection(ctx context.Context) error {
return nil
}
implementation, and open another PR to fill these in. I think what I would do is:
- Implement a DNS lookup utility somewhere in
internal(please let me know your feedback: good idea, bad idea?) - Use that inside
CheckConnectionwhere I can easily get a host name.
There was a problem hiding this comment.
Maybe some of the APIs also offer a status/healthz/ping endpoint that's unauthenticated and that we can use for this.
There was a problem hiding this comment.
👍 @mrnugget. The point is to reuse a native check of the API, and only alternatively craft one.
There was a problem hiding this comment.
@sourcegraph/batchers this pull request extends the Source interface, so there is one change to a fake within batch changes codebase, so wanted to give you an heads up. Please take a look.
mrnugget
left a comment
There was a problem hiding this comment.
Interface looks good to me!
This pull request introduces the availability surfacing method to
Sourceand invokes it withinSyncExternalServiceto allow better error messaging in case of unavailable code host.Subsequent pull requests will introduce actual availability checks for different code hosts.
Test plan
Extend existing syncer unit tests.