Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

symbols: canceling ParseRequest does not spam logs#61719

Merged
keegancsmith merged 1 commit into
mainfrom
k/symbols-log
Apr 10, 2024
Merged

symbols: canceling ParseRequest does not spam logs#61719
keegancsmith merged 1 commit into
mainfrom
k/symbols-log

Conversation

@keegancsmith

Copy link
Copy Markdown
Member

We are seeing lots of log spam related to cancelation of parse requests. We haven't fully tracked this down yet, but the behaviour of spamming logs in this case hides the real issue.

This adjusts the implementation in two places:

  • We introduce a parseCanceled metric so we can still track this. This is used instead of parseFailed in case of cancelation. In this case we also do not emit the "Closing failed parser" even though we are. This is due to log spam.
  • We introduce an ErrorFilter just for handleParseRequest that will elide Canceled from the logs. This is called per file. Note: we keep the error logging for canceled in the high level Parse handler which runs across all files.

Test Plan: go test. Unfortunetly that is as good as it gets for now, as I investigate the root cause further I will test this code path (and only merge once I have manually triggered and tested this code path).

@keegancsmith keegancsmith requested a review from a team April 9, 2024 13:07
@cla-bot cla-bot Bot added the cla-signed label Apr 9, 2024
@github-actions github-actions Bot added team/product-platform team/search-platform Issues owned by the search platform team labels Apr 9, 2024
We are seeing lots of log spam related to cancelation of parse requests.
We haven't fully tracked this down yet, but the behaviour of spamming
logs in this case hides the real issue.

This adjusts the implementation in two places:
- We introduce a parseCanceled metric so we can still track this. This
  is used instead of parseFailed in case of cancelation. In this case we
  also do not emit the "Closing failed parser" even though we are. This
  is due to log spam.
- We introduce an ErrorFilter just for handleParseRequest that will
  elide Canceled from the logs. This is called per file. Note: we keep
  the error logging for canceled in the high level Parse handler which
  runs across all files.

Test Plan: go test. Unfortunetly that is as good as it gets for now, as
I investigate the root cause further I will test this code path (and
only merge once I have manually triggered and tested this code path).

@jtibshirani jtibshirani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. Hopefully this will get us closer to root causing the cancellations!

@keegancsmith keegancsmith merged commit ae59c69 into main Apr 10, 2024
@keegancsmith keegancsmith deleted the k/symbols-log branch April 10, 2024 06:29
sourcegraph-release-bot pushed a commit that referenced this pull request Apr 11, 2024
We are seeing lots of log spam related to cancelation of parse requests.
We haven't fully tracked this down yet, but the behaviour of spamming
logs in this case hides the real issue.

This adjusts the implementation in two places:
- We introduce a parseCanceled metric so we can still track this. This
  is used instead of parseFailed in case of cancelation. In this case we
  also do not emit the "Closing failed parser" even though we are. This
  is due to log spam.
- We introduce an ErrorFilter just for handleParseRequest that will
  elide Canceled from the logs. This is called per file. Note: we keep
  the error logging for canceled in the high level Parse handler which
  runs across all files.

Test Plan: go test. Unfortunetly that is as good as it gets for now, as
I investigate the root cause further I will test this code path (and
only merge once I have manually triggered and tested this code path).

(cherry picked from commit ae59c69)
keegancsmith added a commit that referenced this pull request Apr 12, 2024
#61772)

symbols: canceling ParseRequest does not spam logs (#61719)

We are seeing lots of log spam related to cancelation of parse requests.
We haven't fully tracked this down yet, but the behaviour of spamming
logs in this case hides the real issue.

This adjusts the implementation in two places:
- We introduce a parseCanceled metric so we can still track this. This
  is used instead of parseFailed in case of cancelation. In this case we
  also do not emit the "Closing failed parser" even though we are. This
  is due to log spam.
- We introduce an ErrorFilter just for handleParseRequest that will
  elide Canceled from the logs. This is called per file. Note: we keep
  the error logging for canceled in the high level Parse handler which
  runs across all files.

Test Plan: go test. Unfortunetly that is as good as it gets for now, as
I investigate the root cause further I will test this code path (and
only merge once I have manually triggered and tested this code path).

(cherry picked from commit ae59c69)

Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants