Skip to content

Flaky distributed/tests/test_worker.py::test_lifetime #4862

@fjetter

Description

@fjetter

Random test failure of distributed/tests/test_worker.py::test_lifetime

First observed in https://github.com/dask/distributed/pull/4774/checks?check_run_id=2693532608

Details
    @pytest.mark.slow
    @pytest.mark.asyncio
    async def test_lifetime(cleanup):
        async with Scheduler() as s:
            async with Worker(s.address) as a, Worker(s.address, lifetime="1 seconds") as b:
                async with Client(s.address, asynchronous=True) as c:
                    futures = c.map(slowinc, range(200), delay=0.1)
                    await asyncio.sleep(1.5)
                    assert b.status != Status.running
                    await b.finished()
    
>                   assert set(b.data).issubset(a.data)  # successfully moved data over
E                   assert False
E                    +  where False = <built-in method issubset of set object at 0x7f8134ca3ba0>(Buffer<<LRU: 1176/9019431321 on dict>, <Func: serialize_bytelist<->deserialize_bytes <File: /Users/runner/work/distributed/distributed/dask-worker-space/worker-bi3offm0/storage, mode="a", 0 elements>>>)
E                    +    where <built-in method issubset of set object at 0x7f8134ca3ba0> = {'slowinc-0093af3b7e8e6f16b973e417caa4f85c', 'slowinc-120b294a681d6c5fc1bb125b7dc27a53', 'slowinc-28ec16f2ca5777e76e1e...eeed2f63f7030bb5542ededd', 'slowinc-4b9f7bf1e74f50ba387496336a3ab2de', 'slowinc-4eb81a10f2358a764e70dd9443ac65a8', ...}.issubset
E                    +      where {'slowinc-0093af3b7e8e6f16b973e417caa4f85c', 'slowinc-120b294a681d6c5fc1bb125b7dc27a53', 'slowinc-28ec16f2ca5777e76e1e...eeed2f63f7030bb5542ededd', 'slowinc-4b9f7bf1e74f50ba387496336a3ab2de', 'slowinc-4eb81a10f2358a764e70dd9443ac65a8', ...} = set(Buffer<<LRU: 588/9019431321 on dict>, <Func: serialize_bytelist<->deserialize_bytes <File: /Users/runner/work/distributed/distributed/dask-worker-space/worker-5762snfd/storage, mode="a", 0 elements>>>)
E                    +        where Buffer<<LRU: 588/9019431321 on dict>, <Func: serialize_bytelist<->deserialize_bytes <File: /Users/runner/work/distributed/distributed/dask-worker-space/worker-5762snfd/storage, mode="a", 0 elements>>> = <Worker: 'tcp://10.212.11.232:52856', tcp://10.212.11.232:52856, Status.closed, stored: 21, running: 3/3, ready: 76, comm: 0, waiting: 0>.data
E                    +    and   Buffer<<LRU: 1176/9019431321 on dict>, <Func: serialize_bytelist<->deserialize_bytes <File: /Users/runner/work/distributed/distributed/dask-worker-space/worker-bi3offm0/storage, mode="a", 0 elements>>> = <Worker: 'tcp://10.212.11.232:52850', tcp://10.212.11.232:52850, Status.running, stored: 42, running: 3/3, ready: 155, comm: 0, waiting: 0>.data

distributed/tests/test_worker.py:1588: AssertionError

Metadata

Metadata

Assignees

No one assigned

    Labels

    flaky testIntermittent failures on CI.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions