-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Description
Is your feature request related to a problem? Please describe.
ldap_sync_all has created about 6000 ldap_sync tasks on our system. Currently, we are able to process about 8 of them per minute, meaning they will be running for about 16 hours. This isn't an issue for the synchronization in itself, but to my understanding this means that any task that will be added during this time period will not be handled until after ldap_sync has finished.
Describe the solution you'd like
In general, I think it makes sense to put this kind of task (slow task where lots of them are created in bursts) into a low-priority queue.
Describe alternatives you've considered
I don't know if it's enough to just lower the task's priority (at this point, I don't know deeply enough how celery task queues work).
Additional context
We have patched the scheduling of ldap_sync_all to only be called every 12 hours, but that's irrelevant for the actual issue I'm discussing here.