HueyIntegration TaskLockedException issue #2123
Description
How do you use Sentry?
Sentry Saas (sentry.io)
Version
1.18.0
Steps to Reproduce
Create Retried/Period task, for example:
@huey.db_periodic_task(crontab(minute="*/5", strict=True))
@huey.lock_task("periodic_notify_admins_about_failed_emails")
def periodic_notify_admins_about_failed_emails():
pass
Let's say that the above task can not be locked for whatever reason (concurrent workers/ task takes longer than 5 mins and invoked again etc.), issue occurs when the lock can not be acquired.
Library versions:
Django = "3.2.19"
sentry-sdk = "1.18.0"
huey = "2.4.5"
django-redis = "5.2.0"
redis = {version = "4.5.4", extras = ["hiredis"]}
Expected Result
TaskLockedException raised in each task retry shouldn't be captured and reported to Sentry.
Actual Result
A lot of TaskLockedException issues eat out quota.
Possible solution is to extend
HUEY_CONTROL_FLOW_EXCEPTIONS = (CancelExecution, RetryTask)
with TaskLockedException
HueyIntegration code:
/sentry_sdk/integrations/huey.py
def _capture_exception(exc_info):
# type: (ExcInfo) -> None
hub = Hub.current
if exc_info[0] in HUEY_CONTROL_FLOW_EXCEPTIONS:
hub.scope.transaction.set_status("aborted")
return
def patch_execute():
...
def _sentry_execute(self, task, timestamp=None):
with hub.push_scope() as scope:
...
if not getattr(task, "_sentry_is_patched", False):
task.execute = _wrap_task_execute(task.execute) # wrap task.execute
task._sentry_is_patched = True
...
def _wrap_task_execute(func):
def _sentry_execute(*args, **kwargs):
...
try:
result = func(*args, **kwargs)
except Exception:
exc_info = sys.exc_info()
_capture_exception(exc_info) # TaskLockedException is captured here
reraise(*exc_info)
return result
return _sentry_execute # type: ignore
Where in Huey Task code, TaskLockedException is caught at Huey._execute level and only logged as a warning:
/huey/api.py
class Huey(object):
...
def _execute(self, task, timestamp):
...
try:
self._tasks_in_flight.add(task)
try:
task_value = task.execute() # execute task
finally:
self._tasks_in_flight.remove(task)
duration = time_clock() - start
except TaskLockedException as exc:
logger.warning('Task %s not run, unable to acquire lock.', task.id)
exception = exc
self._emit(S.SIGNAL_LOCKED, task)
...
Could you please confirm if my investigation conclusions are correct?