Visibility timeout set in celery config does not work, celery still using the default 1 hour #7651
Description
Checklist
- I have checked the issues list
for similar or identical bug reports. - I have checked the pull requests list
for existing proposed fixes. - I have checked the commit log
to find out if the bug was already fixed in the master branch. - I have included all related issues and possible duplicate issues in this issue
(If there are none, check this box anyway).
Related Issues and Possible Duplicates
Related Issues
Here is how we write the configs file:
result_expires = 60
task_acks_late = True # enables retry on task failure
worker_prefetch_multiplier = 1 # reserving one task at a time
accept_content = ['json']
result_extended = True
broker_transport_options = {
'queue_order_strategy': 'round_robin',
'visibility_timeout': 7200 # by seconds, 2 hours
}
task_send_sent_event = True
Here is how we configure the worker:
celery = Celery(
__name__,
broker=config.CELERY_BROKER_URL,
backend=config.CELERY_RESULT_BACKEND
)
os.environ.setdefault(
'CELERY_CONFIG_MODULE',
'celeryconfigs.default'
)
celery.config_from_envvar('CELERY_CONFIG_MODULE')
celery.conf.ONCE = {
'backend': 'celery_once.backends.Redis',
'settings': {
'url': config.CELERY_BROKER_URL,
'default_timeout': 60 * 60
}
}
However, tasks running long still get redelivered after 1 hour as the default visibility timeout, not 2 hours as we set. We are following the docs https://docs.celeryq.dev/en/stable/userguide/configuration.html#broker-transport-options What could we possibly miss?
Possible Duplicates
- None