Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

auth: Fix Found page response when redis is down#64063

Merged
eseliger merged 10 commits into
mainfrom
es/07-25-authfixfoundpageresponsewhenredisisdown
Jul 31, 2024
Merged

auth: Fix Found page response when redis is down#64063
eseliger merged 10 commits into
mainfrom
es/07-25-authfixfoundpageresponsewhenredisisdown

Conversation

@eseliger

Copy link
Copy Markdown
Member

At some point, we decided to serve a 500 error instead of a 302 FOUND which redirects to the sign in page when redis is down. This usually happens during a rollout because redis is usually also rolled out at the same time.
When that happens, users would previously get redirected to the signin page, and their session is cleared.

So we decided to instead send a 500 status code, and let them explicitly retry.

However, that didn't stop the execution chain before, so we would send the 500 status code, but then eventually get to the app handler that calls http.Redirect with 302 FOUND (which as a side-effect renders a Found. link as an HTML page for old browsers that don't support 302 natively). Since the http status header has already been sent, it doesn't change the status code to 302 and instead returns a 500 response with the HTML Found. link.

This is the strange error page we've been seeing during rollouts a lot of times and never knew where it's from.

Test plan: Locally stopped redis while the instance was running and no longer saw the Found. message and instead now get a proper error message.

eseliger commented Jul 25, 2024

Copy link
Copy Markdown
Member Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @eseliger and the rest of your teammates on Graphite Graphite

@eseliger eseliger force-pushed the es/07-25-choreremoveredisinitside-effectofapp.newhandler branch from f0a1fa5 to 19ff47f Compare July 30, 2024 23:30
@eseliger eseliger force-pushed the es/07-25-authfixfoundpageresponsewhenredisisdown branch from dd4ef28 to c31f995 Compare July 30, 2024 23:31
@eseliger eseliger force-pushed the es/07-25-choreremoveredisinitside-effectofapp.newhandler branch from 19ff47f to 7678f53 Compare July 31, 2024 01:06
@eseliger eseliger force-pushed the es/07-25-authfixfoundpageresponsewhenredisisdown branch from c31f995 to d664bf4 Compare July 31, 2024 01:07
@eseliger eseliger force-pushed the es/07-25-choreremoveredisinitside-effectofapp.newhandler branch from 7678f53 to 0a608d3 Compare July 31, 2024 01:22
@eseliger eseliger force-pushed the es/07-25-authfixfoundpageresponsewhenredisisdown branch from d664bf4 to f0586b7 Compare July 31, 2024 01:22
eseliger added 9 commits July 31, 2024 01:34
This takes less hoops to find what we support, and we don't extend this any time soon.

Test plan:

CI passes.
These functions return StoreOpts, but that wasn't immediately clear, so adding this small tweak here.

Test plan: Ci passes.
This code was using a strange pattern that isn't actually observed or controlled by our worker mechanisms, so switching it to return proper goroutines.

Test plan:

CI passes, would like a thorough review on the licensecheck code.
Since we don't do the enterprise/oss split anymore, this global package is no longer required and we can move the code to where it's actually used.

Test plan: Go compiler doesn't complain, and integration tests are still passing.
This doesn't blow up, but golangci-lint complains about it so following best practices here.

Test plan: Go test still passes, linter doesn't complain anymore.
This validation was done in the azure oauth provider, which mixes two completely separate domains.

Test plan: CI, code review.
This removes the need to register a global URL watcher. Instead, the conf package now returns a cached version, and also makes a copy of it so it's impossible to accidentally modify it.

This makes it safe in non-cmd/frontend packages to use this.

Test plan: All tests are still passing.
… provider

Just a little cleanup which makes the main function a bit shorter and easier to understand that userpasswd is not different to other authn providers.

Test plan: Authn with builtin still works in CI integration tests.
This current code sets a global variable which couples the session package and the app package and the session package cannot work when the NewHandler method wasn't called yet.

This also used to wait for redis implicitly, without any indication of doing that.

To be more explicit about that, we now do that in the main function of frontend instead, and create the session store where required on the fly.

Test plan: Login still works, integration/E2E tests are passing.
@eseliger eseliger force-pushed the es/07-25-choreremoveredisinitside-effectofapp.newhandler branch from 0a608d3 to a7e2c69 Compare July 31, 2024 01:38
At some point, we decided to serve a 500 error instead of a 302 FOUND which redirects to the sign in page when redis is down. This usually happens during a rollout because redis is usually also rolled out at the same time.
When that happens, users would previously get redirected to the signin page, and their session is cleared.

So we decided to instead send a 500 status code, and let them explicitly retry.

However, that didn't stop the execution chain before, so we would send the 500 status code, but then eventually get to the app handler that calls http.Redirect with 302 FOUND (which as a side-effect renders a `Found.` link as an HTML page for old browsers that don't support 302 natively). Since the http status header has already been sent, it doesn't change the status code to 302 and instead returns a 500 response with the HTML `Found.` link.

This is the strange error page we've been seeing during rollouts a lot of times and never knew where it's from.

Test plan: Locally stopped redis while the instance was running and no longer saw the `Found.` message and instead now get a proper error message.
@eseliger eseliger force-pushed the es/07-25-authfixfoundpageresponsewhenredisisdown branch from f0586b7 to fc04d87 Compare July 31, 2024 01:39
@eseliger eseliger force-pushed the es/07-25-choreremoveredisinitside-effectofapp.newhandler branch from a7e2c69 to 4676485 Compare July 31, 2024 02:10
Base automatically changed from es/07-25-choreremoveredisinitside-effectofapp.newhandler to main July 31, 2024 02:15
@eseliger eseliger merged commit a156ee6 into main Jul 31, 2024
@eseliger eseliger deleted the es/07-25-authfixfoundpageresponsewhenredisisdown branch July 31, 2024 02:15
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla-signed team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants