Semaphores release automatically - refresh won't run #4057
Description
What happened:
Semaphores always automatically release after 40 seconds
Example output:
1597944899.78569 1
1597944899.7857327 0
1597944939.7501612 3
1597944939.75026 2
distributed.semaphore - WARNING - Tried to release semaphore but it was already released: name=database, lease_id=52a2ef49dbfa4fafb68d1118adfcb807. This can happen if the semaphore timed out before.
distributed.semaphore - WARNING - Tried to release semaphore but it was already released: name=database, lease_id=08821fab060c4d25b05209d5bfba6716. This can happen if the semaphore timed out before.
finished 0
finished 1
...
What you expected to happen:
Semaphore leases should be automatically refreshed.
Minimal Complete Verifiable Example:
import dask.distributed as dd
import time
if __name__ == '__main__':
client = dd.Client(processes=False, threads_per_worker=10)
sem = dd.Semaphore(max_leases=2, name="database")
def access_limited(val, sem):
with sem:
print(time.time(), val)
# Interact with the DB
time.sleep(60)
print('finished', val)
futures = client.map(access_limited, range(10), sem=sem)
client.gather(futures)
sem.close()Anything else we need to know?:
Workaround would be appreciated!
Environment:
- Dask version: 2.22.0 & 2.23.0 tested
- Python version: 3.7
- Operating System: Linux
- Install method (conda, pip, source): pip