Skip to content

logging: wire klog backend, but only output request throttling logs#5419

Merged
negz merged 1 commit intocrossplane:masterfrom
sttts:sttts-request-throttling
Feb 28, 2024
Merged

logging: wire klog backend, but only output request throttling logs#5419
negz merged 1 commit intocrossplane:masterfrom
sttts:sttts-request-throttling

Conversation

@sttts
Copy link
Copy Markdown
Contributor

@sttts sttts commented Feb 22, 2024

Description of your changes

Vendoring and wiring crossplane/crossplane-runtime#673.

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable to ensure this PR is ready for review.
  • [ ] Added or updated unit tests.
  • [ ] Added or updated e2e tests.
  • [ ] Linked a PR or a docs tracking issue to document this change.
  • [ ] Added backport release-x.y labels to auto-backport this PR.

Need help with this checklist? See the cheat sheet.

@sttts sttts requested review from a team and turkenh as code owners February 22, 2024 08:49
@sttts sttts requested a review from bobh66 February 22, 2024 08:49
@sttts sttts force-pushed the sttts-request-throttling branch from 0bf32da to e982faa Compare February 22, 2024 08:50
Copy link
Copy Markdown
Member

@jbw976 jbw976 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sttts would it be possible for you to provide more details in the PR description that show what the logs look like with this change? besides helping understand the general experience, I'm also hoping to see how these logs will be able to positively identify slowdowns and if there's anything the person reading the logs can do about it.

@sttts
Copy link
Copy Markdown
Contributor Author

sttts commented Feb 22, 2024

@jbw976 added the logging lines from client-go that now pass.

@sttts
Copy link
Copy Markdown
Contributor Author

sttts commented Feb 22, 2024

As an example, @turkenh would have seen something like

Waited for 1.033772408s due to client-side throttling, not priority and fairness, request: GET:https://api.example.org/apis/pkg.crossplane.io/v1?timeout=32s

in crossplane-contrib/provider-kubernetes#203 like crazy because client-side throttling was aggressively set to 10 requests per second.

@turkenh
Copy link
Copy Markdown
Member

turkenh commented Feb 22, 2024

As an example, @turkenh would have seen something like

@sttts I am not sure, my understanding was we were rate-limiting ourself even before making client side requests 🤔

@sttts
Copy link
Copy Markdown
Contributor Author

sttts commented Feb 22, 2024

Ok, controller rate limiting is even before, yes.

@sttts
Copy link
Copy Markdown
Contributor Author

sttts commented Feb 22, 2024

For background: there are 3 levels of throttling usually:

  1. Controller rate limiting (max-reconcile-rate)
  2. Client-side request throttling
  3. Server-side request throttling (“priority and fairness”).

In Crossplane afaik request rps is 5x Max-reconcile-rate. Any of these 3 can make things slow if load is high enough.

Copy link
Copy Markdown
Member

@negz negz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable. Do you want to go straight to adding it to crossplane-runtime pkg/logging rather than proceeding here?

If you want to proceed here, see comments. There's also some linter errors to address.

WRT the log output - it will use the same format as the rest of our log lines, right? e.g. If we set our logger to use structured logging these would be structured too? I expect that to be the case from the code but it's not clear from the log lines in the description.

@negz
Copy link
Copy Markdown
Member

negz commented Feb 22, 2024

In Crossplane afaik request rps is 5x Max-reconcile-rate. Any of these 3 can make things slow if load is high enough.

FYI I retroactively documented our client-side rate limiters in #5415.

I was thinking debug logging and emitting metrics when the workqueue and/or rate-limiting Reconciler "middleware" were rate limiting would be useful too.

@sttts
Copy link
Copy Markdown
Contributor Author

sttts commented Feb 22, 2024

Would go directly to crossplane-runtime. Will open it there.

@sttts
Copy link
Copy Markdown
Contributor Author

sttts commented Feb 27, 2024

@turkenh @negz update to use the crossplane-runtime helper.

Signed-off-by: Dr. Stefan Schimanski <stefan.schimanski@upbound.io>
@sttts sttts force-pushed the sttts-request-throttling branch from 6e85902 to eb9fe61 Compare February 27, 2024 20:37
Copy link
Copy Markdown
Member

@negz negz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you!

@negz negz merged commit 44e0ee3 into crossplane:master Feb 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants