Skip to content

Conversation

@gpauloski
Copy link
Collaborator

Description

When passing proxies to and from tasks executed on remote processes (e.g., via ProcessPoolExecutor, Parsl, Dask, etc.), returning an OwnedProxy from the task may not have the intended effect. Intuitively, you might think that this yields ownership from the invoked task that's creating the OwnedProxy to the client which is receiving the result of the task. However, when the invoked task's result gets serialized and then goes out of scope, the destructor of the OwnedProxy will evict the associated data. That means that when the client receives the OwnedProxy, it is now invalid.

The workaround is to return a Proxy, and then convert it to an OwnedProxy via into_owned() on the client side. This PR adds this helper function and documents this edge case in submit().

Fixes N/A

Type of Change

  • Breaking Change (fix or enhancement which changes existing semantics of the public interface)
  • Enhancement (new features or improvements to existing functionality)
  • Bug (fixes for a bug or issue)
  • Internal (refactoring, style changes, testing, optimizations)
  • Documentation update (changes to documentation or examples)
  • Package (dependencies, versions, package metadata)
  • Development (CI workflows, pre-commit, linters, templates)
  • Security (security related changes)

Testing

Added new tests.

Pull Request Checklist

Please confirm the PR meets the following requirements.

  • Tags added to PR (e.g., breaking, bug, enhancement, internal, documentation, package, development, security).
  • Code changes pass pre-commit (e.g., mypy, ruff, etc.).
  • Tests have been added to show the fix is effective or that the new feature works.
  • New and existing unit tests pass locally with the changes.
  • Docs have been updated and reviewed if relevant.

@gpauloski gpauloski added the enhancement New features or improvements to existing functionality label Feb 28, 2024
@gpauloski gpauloski merged commit 6972c32 into main Feb 28, 2024
@gpauloski gpauloski deleted the return-owned-proxy branch February 28, 2024 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New features or improvements to existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants