Skip to content

Conversation

@gpauloski
Copy link
Collaborator

Description

This PR replaces use of lazy_object_proxy.slots.Proxy with a new custom slots-based Proxy implementation. Compared to the prior Proxy implementation with extended lazy_object_proxy.slots.Proxy to add different pickling mechanics, the new proxystore.proxy.Proxy has better handling for __module__ access, a simpler MRO hierarchy (i.e., less inheritance), correct support for __annotations__, and drops magic methods deprecated in Python 3.

Of these changes, the most important is that __module__ now behaves differently if accessed on Proxy directly or an instance of Proxy.

>>> from proxystore.proxy import Proxy
>>> Proxy.__module__
'proxystore.proxy'
>>> Proxy.__name__
'Proxy'
>>> proxy = Proxy(lambda: object)
>>> proxy.__module__
'builtins'
>>> proxy.__name__
'object'

This change addresses many long-standing headaches related to using proxies:

  • help(proxystore.proxy) works correctly and no longer raises a vague TypeError: '<' not supported between instances of 'str' and 'property'.
  • Serializers which inspect the __module__ attribute of object types no longer fail (as in Parsl and Dask).
  • A trampoline function is no longer needed for pickle to correctly reconstruct the type.

Internally, this required a lot of changes. Most importantly is that lazy_object_proxy.slots.Proxy needed to be modified so that source has been copied into ProxyStore's source and modified. The license for lazy-object-proxy is now included in THIRD_PARTY_LICENSES and included in the relevant source files containing the third-party code.

Fixes

Type of Change

  • Breaking Change (fix or enhancement which changes existing semantics of the public interface)
  • Enhancement (new features or improvements to existing functionality)
  • Bug (fixes for a bug or issue)
  • Internal (refactoring, style changes, testing, optimizations)
  • Documentation update (changes to documentation or examples)
  • Package (dependencies, versions, package metadata)
  • Development (CI workflows, pre-commit, linters, templates)
  • Security (security related changes)

Testing

Added a lot of new tests.

Pull Request Checklist

Please confirm the PR meets the following requirements.

  • Tags added to PR (e.g., breaking, bug, enhancement, internal, documentation, package, development, security).
  • Code changes pass pre-commit (e.g., mypy, ruff, etc.).
  • Tests have been added to show the fix is effective or that the new feature works.
  • New and existing unit tests pass locally with the changes.
  • Docs have been updated and reviewed if relevant.

Includes the slots proxy implementation source from lazy-object-proxy.
This is necessary to modify the proxy's metaclass to use our custom
@proxy_property decorator rather than the native @Property.

The corresponding BSD license for lazy-object-proxy has been included
in the associated source file and the THIRD_PARTY_LICENSES files.

The dependency on lazy-object-proxy has been removed.
@gpauloski gpauloski added the enhancement New features or improvements to existing functionality label Apr 19, 2024
@gpauloski gpauloski force-pushed the issue-517 branch 2 times, most recently from d19006b to c2fe293 Compare April 19, 2024 19:15
@gpauloski gpauloski merged commit d6caff6 into main Apr 19, 2024
@gpauloski gpauloski deleted the issue-517 branch April 19, 2024 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New features or improvements to existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve handling of attribute access on Proxy class

2 participants