Description

Enhance compatibility of the StoreExecutor with non-concurrent.futures.Executor types. The specific goal of this PR was to support the Dask Distributed Client within the StoreExecutor.

Fixes

• Fixes Make StoreExecutor compatible with Dask Client #607

Type of Change

• Breaking Change (fix or enhancement which changes existing semantics of the public interface)
• Enhancement (new features or improvements to existing functionality)
• Bug (fixes for a bug or issue)
• Internal (refactoring, style changes, testing, optimizations)
• Documentation update (changes to documentation or examples)
• Package (dependencies, versions, package metadata)
• Development (CI workflows, pre-commit, linters, templates)
• Security (security related changes)

Testing

Added new test cases using mock Dask Client. Also tested using this example script:

from dask.distributed import Client
from proxystore.connectors.file import FileConnector
from proxystore.proxy import Proxy
from proxystore.store import Store
from proxystore.store.executor import StoreExecutor, ProxyType

def concat(base: str, *, num: int) -> str:
    return f'{base}-{num}'

def power(x: int) -> int:
    return x ** 2


if __name__ == '__main__':
    client = Client()
    store = Store('executor-example', FileConnector('./object-cache'))

    with StoreExecutor(
        client,
        store=store,
        should_proxy=ProxyType(str),
        ownership=False,
    ) as executor:
        future = executor.submit(concat, 'foobar', num=42)
        result = future.result()

        assert isinstance(result, Proxy)
        assert result == 'foobar-42'

        x = list(executor.map(power, [1, 2, 3]))
        assert x == [1, 4, 9]

Pull Request Checklist

• I have read the Contributing and PR submission guides.

Please confirm the PR meets the following requirements.

• Tags added to PR (e.g., breaking, bug, enhancement, internal, documentation, package, development, security).
• Code changes pass pre-commit (e.g., mypy, ruff, etc.).
• Tests have been added to show the fix is effective or that the new feature works.
• New and existing unit tests pass locally with the changes.
• Docs have been updated and reviewed if relevant.