Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

SyncExternalService fails fast if code host does not seem available#44922

Merged
cbart merged 10 commits into
mainfrom
cbart/44683-introduce-extra-code-host-dial
Dec 2, 2022
Merged

SyncExternalService fails fast if code host does not seem available#44922
cbart merged 10 commits into
mainfrom
cbart/44683-introduce-extra-code-host-dial

Conversation

@cbart

@cbart cbart commented Nov 29, 2022

Copy link
Copy Markdown
Contributor

This pull request introduces the availability surfacing method to Source and invokes it within SyncExternalService to allow better error messaging in case of unavailable code host.

Subsequent pull requests will introduce actual availability checks for different code hosts.

Test plan

Extend existing syncer unit tests.

@cla-bot cla-bot Bot added the cla-signed label Nov 29, 2022
@cbart cbart linked an issue Nov 29, 2022 that may be closed by this pull request
4 tasks
@sourcegraph-bot

sourcegraph-bot commented Nov 29, 2022

Copy link
Copy Markdown
Contributor

Codenotify: Notifying subscribers in CODENOTIFY files for diff cdf3710...123526a.

Notify File(s)
@LawnGnome internal/extsvc/bitbucketcloud/client.go
@eseliger enterprise/internal/batches/sources/testing/fake.go
internal/extsvc/bitbucketcloud/client.go
internal/extsvc/versions/sync_test.go
@indradhanush internal/repos/awscodecommit.go
internal/repos/bitbucketcloud.go
internal/repos/bitbucketserver.go
internal/repos/gerrit.go
internal/repos/github.go
internal/repos/gitlab.go
internal/repos/gitolite.go
internal/repos/other.go
internal/repos/packages.go
internal/repos/pagure.go
internal/repos/perforce.go
internal/repos/phabricator.go
internal/repos/sources.go
internal/repos/syncer.go
internal/repos/syncer_test.go
internal/repos/testing.go
@ryanslade internal/repos/awscodecommit.go
internal/repos/bitbucketcloud.go
internal/repos/bitbucketserver.go
internal/repos/gerrit.go
internal/repos/github.go
internal/repos/gitlab.go
internal/repos/gitolite.go
internal/repos/other.go
internal/repos/packages.go
internal/repos/pagure.go
internal/repos/perforce.go
internal/repos/phabricator.go
internal/repos/sources.go
internal/repos/syncer.go
internal/repos/syncer_test.go
internal/repos/testing.go
@sashaostrikov internal/repos/awscodecommit.go
internal/repos/bitbucketcloud.go
internal/repos/bitbucketserver.go
internal/repos/gerrit.go
internal/repos/github.go
internal/repos/gitlab.go
internal/repos/gitolite.go
internal/repos/other.go
internal/repos/packages.go
internal/repos/pagure.go
internal/repos/perforce.go
internal/repos/phabricator.go
internal/repos/sources.go
internal/repos/syncer.go
internal/repos/syncer_test.go
internal/repos/testing.go

Comment thread internal/repos/syncer.go Outdated
}

if !src.IsAvailable(ctx) {
return errors.New("code host does not seem to be available")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should include more context? Perhaps IsAvailable can return an error instead? In that case, maybe we should call it something like CheckConnection?

We should also be careful with how we check the validity of the connection, we probably only want to fail on networking issues. If the issue is caused by an invalid token for example, this is a real failure that should result in repos being removed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing we can probably do is lookup the IP of the codehost to validate that DNS is working correctly as a first check: https://jameshfisher.com/2017/08/03/golang-dns-lookup/

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll put this comment in the issue. @varsanojidan

Yes, I was thinking something like dialing host-port over TCP:

timeout := 1 * time.Second
conn, err := net.DialTimeout("tcp",fmt.Sprintf("%s:433", host), timeout)
if err != nil {
    return fmt.Errorf("Site unreachable, error: ", err)
}
conn.Close()

But actually starting with a DNS lookup and then doing this makes a lot of sense.

@ryanslade thank you for the comment - indeed it makes sense to return some meaningful error message here, and then name change you proposed seems very fitting

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done now. If you do not mind, I would like to land this with the

func (...) CheckConnection(ctx context.Context) error {
  return nil
}

implementation, and open another PR to fill these in. I think what I would do is:

  1. Implement a DNS lookup utility somewhere in internal (please let me know your feedback: good idea, bad idea?)
  2. Use that inside CheckConnection where I can easily get a host name.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe some of the APIs also offer a status/healthz/ping endpoint that's unauthenticated and that we can use for this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 @mrnugget. The point is to reuse a native check of the API, and only alternatively craft one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sourcegraph/batchers this pull request extends the Source interface, so there is one change to a fake within batch changes codebase, so wanted to give you an heads up. Please take a look.

@cbart cbart requested a review from a team November 30, 2022 15:24

@mrnugget mrnugget left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interface looks good to me!

Comment thread internal/repos/awscodecommit.go Outdated
Comment thread internal/repos/bitbucketcloud.go
Comment thread internal/repos/bitbucketserver.go
Comment thread internal/repos/github.go Outdated
Comment thread internal/repos/gitlab.go Outdated
Comment thread internal/repos/testing.go
@cbart cbart requested review from a team and removed request for a team December 2, 2022 16:54
@cbart cbart merged commit bc6d97d into main Dec 2, 2022
@cbart cbart deleted the cbart/44683-introduce-extra-code-host-dial branch December 2, 2022 18:33
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make codehost connection issues more clear to the user.

5 participants