Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Retry requests if GitLab external rate limit is hit#48616

Merged
pjlast merged 17 commits into
mainfrom
pjlast/gitlab-external-rate-limiter
Mar 7, 2023
Merged

Retry requests if GitLab external rate limit is hit#48616
pjlast merged 17 commits into
mainfrom
pjlast/gitlab-external-rate-limiter

Conversation

@pjlast

@pjlast pjlast commented Mar 3, 2023

Copy link
Copy Markdown
Contributor

Closes #48614

Before this PR
GitLab requests all share a single global internal rate limiter. This is a very crude and inaccurate approximation of GitLab's actual rate limits.

After this PR
GitLab requests now use GitLab's response headers to keep track of individual user rate limits. This allows Sourcegraph to make a lot more requests and stay up to date faster while staying inside GitLab's rate limits.

Test plan

Unit tests added.

@cla-bot cla-bot Bot added the cla-signed label Mar 3, 2023
@pjlast pjlast force-pushed the pjlast/gitlab-external-rate-limiter branch from 5c47e96 to 30c9851 Compare March 3, 2023 13:17
@pjlast pjlast changed the title Rename internal and external rate limiters Retry requests if GitLab external rate limit is hit Mar 3, 2023
@pjlast pjlast force-pushed the pjlast/gitlab-external-rate-limiter branch from b7851fe to 8cf5b92 Compare March 6, 2023 09:26
@pjlast pjlast marked this pull request as ready for review March 6, 2023 13:51
@pjlast pjlast requested a review from a team March 6, 2023 13:51
@sourcegraph-bot

sourcegraph-bot commented Mar 6, 2023

Copy link
Copy Markdown
Contributor

Codenotify: Notifying subscribers in CODENOTIFY files for diff 62a93a2...d09e3be.

Notify File(s)
@eseliger internal/extsvc/gitlab/client.go
internal/extsvc/gitlab/client_test.go
internal/extsvc/gitlab/merge_requests.go
internal/extsvc/gitlab/notes.go
internal/extsvc/gitlab/pipelines.go
internal/extsvc/gitlab/projects.go
internal/extsvc/gitlab/resource_state_events.go
internal/extsvc/gitlab/util_test.go
internal/extsvc/gitlab/version.go
internal/extsvc/types.go
internal/extsvc/types_test.go
@indradhanush internal/repos/gitlab.go
@sashaostrikov internal/repos/gitlab.go

@sashaostrikov sashaostrikov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
I've added a comment about a test for possible forgotten mutex unlock.

Comment thread internal/extsvc/gitlab/client.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: ignore explicitly?

Suggested change
c.externalRateLimiter.WaitForRateLimit(ctx)
_ = c.externalRateLimiter.WaitForRateLimit(ctx)

Comment thread internal/ratelimit/monitor.go Outdated
Comment thread internal/extsvc/gitlab/client.go Outdated
Comment thread internal/ratelimit/monitor.go Outdated
Comment thread internal/extsvc/gitlab/merge_requests.go Outdated

@vdavid vdavid left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, very nice! I thought it'd be a lot of time before we see this work for GitLab, too!

@pjlast pjlast force-pushed the pjlast/gitlab-external-rate-limiter branch from 7f2af46 to 782b7e0 Compare March 7, 2023 09:40
Comment thread internal/extsvc/gitlab/client.go Outdated
Comment thread internal/extsvc/gitlab/client.go Outdated
Comment thread internal/extsvc/gitlab/client.go Outdated
Comment thread internal/extsvc/gitlab/merge_requests.go Outdated
Comment thread internal/ratelimit/monitor.go Outdated
Comment thread internal/extsvc/gitlab/client.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitLab.com has a lot of different rate limits, depending on which API you use: https://docs.gitlab.com/ee/user/gitlab_com/index.html#gitlabcom-specific-rate-limits

Are we sure that all of the uses of this method here use the same API?

I'm scared that if we use doWithBaseURL for different APIs with different rate limits we'd flip-flop between rate limits depending on the last request.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand, GitLab only responds with rate limit headers if the rate limit has been hit. So the only time in which a rate limit should be relevant is if we're immediately going to retry the request. I also think we only really use the Authenticated API endpoints

@pjlast pjlast force-pushed the pjlast/gitlab-external-rate-limiter branch from 507d35d to 535ec72 Compare March 7, 2023 12:02
@pjlast pjlast merged commit 78c061a into main Mar 7, 2023
@pjlast pjlast deleted the pjlast/gitlab-external-rate-limiter branch March 7, 2023 13:03
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GitLab external rate limiter

6 participants