Skip to content

Implements Python RPC + test suite.#3931

Merged
dom96 merged 1 commit intomainfrom
dominik/python-rpc-testing
May 7, 2025
Merged

Implements Python RPC + test suite.#3931
dom96 merged 1 commit intomainfrom
dominik/python-rpc-testing

Conversation

@dom96
Copy link
Contributor

@dom96 dom96 commented Apr 11, 2025

Relatively large PR which introduces Workers RPC support for Python. The aim is to make it easy to call functions defined in other Python and JS Workers, to do this all arguments are converted to/from JS where appropriate.

The implementation relies on a number of wrappers as well as conversion functions:

  • The binding env object is wrapped via the handler decorator, it gets wrapped in a class that intercepts calls to Fetcher bindings (i.e. env.ServiceBinding) and then another class that intercepts calls made on that binding. All arguments passed to these calls are wrapped in a call to serialize_to_js and the returned value from the call is wrapped in a call to serialize_to_python.
  • The serialize_to_python and serialize_to_js functions convert Python and JS arguments to JS and Python respectively. These mostly use Pyodide's to_js and to_py but with some improvements and restrictions.
  • Arguments received by WorkerEntrypoints also get converted. This is done in the code that manages the entrypoint classes and already intercepts calls.

All of this is supported by a test suite which attempts to send/receive both from and to Python, as well as from and to JS.

Test Plan

$ bazel run @workerd//src/workerd/server/tests/python:python-rpc_development@

@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 5 times, most recently from 70063bd to 62b8370 Compare April 25, 2025 14:44
@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 2 times, most recently from 54bf0c6 to 4451eac Compare April 25, 2025 17:31
@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 6 times, most recently from 949b5c8 to bb0767f Compare April 28, 2025 17:12
@dom96 dom96 marked this pull request as ready for review April 28, 2025 17:14
@dom96 dom96 requested review from a team as code owners April 28, 2025 17:14
@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 2 times, most recently from 61fffba to d2bc3c2 Compare April 28, 2025 17:26
@danlapid
Copy link
Collaborator

I'll review tomorrow
In the meantime, mind adding a description to the PR of your implementation approach?
Thanks!

@dom96 dom96 force-pushed the dominik/python-rpc-testing branch from d2bc3c2 to 619985c Compare April 28, 2025 17:45
@dom96
Copy link
Contributor Author

dom96 commented Apr 28, 2025

@danlapid sure

@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 2 times, most recently from 7785efa to a46583a Compare April 29, 2025 11:12
@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 6 times, most recently from e834101 to bbba8eb Compare May 6, 2025 17:24
Copy link
Contributor

@hoodmane hoodmane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me other than suggestion for using unittest's assertRaises.

@dom96 dom96 force-pushed the dominik/python-rpc-testing branch 2 times, most recently from 97e889f to 172b7a9 Compare May 7, 2025 15:47
@dom96 dom96 force-pushed the dominik/python-rpc-testing branch from 172b7a9 to f88d72a Compare May 7, 2025 16:23
@dom96 dom96 merged commit 20a01f1 into main May 7, 2025
18 checks passed
@dom96 dom96 deleted the dominik/python-rpc-testing branch May 7, 2025 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants