Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Retry 404 errors from Triton#63315

Merged
rafax merged 7 commits into
mainfrom
rg/retry_400_embeddings
Jun 19, 2024
Merged

Retry 404 errors from Triton#63315
rafax merged 7 commits into
mainfrom
rg/retry_400_embeddings

Conversation

@rafax

@rafax rafax commented Jun 18, 2024

Copy link
Copy Markdown
Contributor

Currently, when SMEGA scales down a pod, it will return 404 for a period of time, and that 404 will get translated into a 500 response from Cody Gateway. This PR implements (exponential) retries (with jitter), attempting to send the same request to a different Triton pod.

Closes AI-86.

Related to AI-31, AI-87.

Test plan

  • tested locally with a backend that throws random 404

"context"
"io"
"math"
"math/rand"

Check warning

Code scanning / Semgrep OSS

Semgrep Finding: security-semgrep-rules.semgrep-rules.golang.math-random-used

Do not use `math/rand`. Use `crypto/rand` instead.
@cla-bot cla-bot Bot added the cla-signed label Jun 18, 2024
@rafax rafax marked this pull request as ready for review June 18, 2024 12:11
@rafax rafax requested a review from a team June 18, 2024 12:31
@rafax rafax enabled auto-merge (squash) June 18, 2024 14:35

@rishabhmehrotra rishabhmehrotra left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@rafax rafax merged commit c68cd52 into main Jun 19, 2024
@rafax rafax deleted the rg/retry_400_embeddings branch June 19, 2024 12:43
This was referenced Jun 19, 2024
rafax referenced this pull request Jun 19, 2024
Switches code from https://github.com/sourcegraph/sourcegraph/pull/63315
to use [math/rand/v2](https://pkg.go.dev/math/rand/v2) as suggested by
@keegancsmith.


## Test plan

- tested locally with a backend throwing random 404
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants