Skip to content

Add option to notify Sentry only after the last retry on resque#2087

Merged
sl0thentr0py merged 5 commits intogetsentry:masterfrom
GlauberrBatista:feature/resque_retry_report_after
Oct 10, 2023
Merged

Add option to notify Sentry only after the last retry on resque#2087
sl0thentr0py merged 5 commits intogetsentry:masterfrom
GlauberrBatista:feature/resque_retry_report_after

Conversation

@GlauberrBatista
Copy link
Copy Markdown
Contributor

sentry-sidekiq and sentry-delayed_job have the option to report the failure only after the retry process has finished.

With sentry-resque we can do the same when using the resque-retry gem. The implementation is not as clean as Sidekiq and DelayedJob, which expose the attempts in a better way, but still, I was able to make it work.
I noticed the configuration option was there but not implemented. I added a conditional inside the exception block to check if the job is an instance of Resque::Plugins::Retry and if so, it checks the retry key.

An odd behavior: resque-retry cleans the key on the last try, before being caught on the exception block. Thus, when the key no longer exists on Redis, it means the retries were exhausted.

I understand this change implies using the resque-retry gem, but I'm unaware of a similar gem widely used as resque-retry.

@GlauberrBatista GlauberrBatista changed the title feat: add option to report failure only on the last retry Add option to notify Sentry only after the last retry on resque Aug 7, 2023
@codecov
Copy link
Copy Markdown

codecov bot commented Aug 20, 2023

Codecov Report

All modified lines are covered by tests ✅

Files Coverage Δ
sentry-resque/lib/sentry/resque.rb 97.36% <100.00%> (+0.14%) ⬆️

... and 50 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

@GlauberrBatista
Copy link
Copy Markdown
Contributor Author

Hello, any update on this?

@andreynering
Copy link
Copy Markdown

/cc @sl0thentr0py

@sl0thentr0py sl0thentr0py force-pushed the feature/resque_retry_report_after branch from 36f916f to 27beb99 Compare September 21, 2023 11:14
Copy link
Copy Markdown
Member

@sl0thentr0py sl0thentr0py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey sorry for the delay on this but I don't think this solution is something we can put in the SDK (see comment)

If there's another possibility to fix the logic, I can take that otherwise I'll dig into resque later.


if Sentry.configuration.resque.report_after_job_retries && defined?(::Resque::Plugins::Retry) == 'constant' && klass.is_a?(::Resque::Plugins::Retry)
retry_key = klass.redis_retry_key(payload['args'])
retry_attempt = ::Resque.redis.get(retry_key)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the SDK should never do its own database calls by any chance, that's a side effect that breaks all sorts of contracts.
Is there no way of knowing the retry status from the stuff available to this function?

Copy link
Copy Markdown
Contributor Author

@GlauberrBatista GlauberrBatista Sep 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sl0thentr0py In fact, there is. I didn't realize it, but we are catching an exception here (so the processed job context is available) and when we use constantize against the class name, resque-retry will capture the job context, so methods retry_limit_reached? and retry_limit_attempt will expose the current "status" of the job.

I just pushed a new commit that fixes it. Hopefully it will match the requirements

@sl0thentr0py sl0thentr0py force-pushed the feature/resque_retry_report_after branch from 76fa859 to 2b1ab7d Compare October 10, 2023 13:23
Copy link
Copy Markdown
Member

@sl0thentr0py sl0thentr0py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the delay again and ty @GlauberrBatista !

@sl0thentr0py sl0thentr0py merged commit 71ec4f4 into getsentry:master Oct 10, 2023
@GlauberrBatista
Copy link
Copy Markdown
Contributor Author

@sl0thentr0py not a problem and thank you 🙌

@GlauberrBatista GlauberrBatista deleted the feature/resque_retry_report_after branch October 10, 2023 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants