Skip to content

Exploration: Better transferring of TypedArrays used in Webview.postMessage#115664

Merged
mjbvz merged 4 commits intomicrosoft:mainfrom
mjbvz:webview-typed-array-transfer
Mar 30, 2021
Merged

Exploration: Better transferring of TypedArrays used in Webview.postMessage#115664
mjbvz merged 4 commits intomicrosoft:mainfrom
mjbvz:webview-typed-array-transfer

Conversation

@mjbvz
Copy link
Copy Markdown
Collaborator

@mjbvz mjbvz commented Feb 3, 2021

Fixes #115807

This improves optimizes the passing of ArrayBuffers to and from webviews. Previously, these objects were inefficiently serialized and were not properly revived on the webview side

This change makes it so that if a user uses the new transfer argument on Webview.postMessage, any ArrayBuffers in the message are more efficiently transferred and revived

@mjbvz mjbvz requested a review from alexdima February 3, 2021 07:29
@mjbvz mjbvz self-assigned this Feb 3, 2021
@mjbvz mjbvz force-pushed the webview-typed-array-transfer branch from 81a3a9a to cde83c1 Compare February 3, 2021 07:40
Copy link
Copy Markdown
Member

@alexdima alexdima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prototyped this entirely in webview code in order to avoid any changes to our RPC code

👍 I think the approach is very good! Walking all the transferred objects is expensive and we used to do that for all our RPC calls. We used to automatically serialize regular expressions, and on the revive side we would instantiate URIs and RegExps. We then found out that was a real perf bottleneck, especially for some really large intellisense payloads, but it would add a cost to everything. Walking untyped objects is just slow... It took us quite an effort to move away from that and now we revive things manually wherever we need it. @jrieken can fill in more details if I missed anything.

So today the RPC layer only does a single walk over the arguments. It does not go inside the arguments anymore. So I like the approach of doing things outside the RPC layer, and paying the perf cost of walking the object only where we need to support this. If we find out we need this in another place (i.e. for another RPC call) we could add a utility to serialize / deserialize using this format. We have done that before in marshalling.ts, so I think a utility there like serializeTypedArrays/deserializeTypedArrays would be beneficial.

Regarding web, I'm not sure what you mean exactly, since we have so many moving parts. But AFAIK, using VSBuffer is the right thing to do and is supported also on web in extHost.protocol.ts, where we respect and send arguments of type VSBuffer as a transferable in the postMessage.

@jrieken
Copy link
Copy Markdown
Member

jrieken commented Feb 3, 2021

It took us quite an effort to move away from that and now we revive things manually wherever we need it. @jrieken can fill in more details if I missed anything.

Yeah, all of that is gone (and the reason us using URI.revive a lot). Tho, it was a good change because recursively walking all objects that are being send is expensive. From the looks it seems that your solution doesn't require such a walk so that you can find and replace TypedArray instances.

For web, it would be great if we could mark any TypedArrays in postMessage as transferable to improve performance for large arrays. I believe this would require some changes in our core RPC code. I think we are already using transferable with here

On top of that we should investigate if message ports can be used. With that we could connect webviews directly to the extension host and wouldn't need to go through the renderer anymore. (I have done that with the web extension host that is launched through an iframe). Electron now supports MessagePorts between the main and renderer processes (@bpasero explored this for the shared process) and we should figure out if message ports would also work between webview processes and the extension host. Iff so then you would use our RPC only to exchange the ports and use a direct channel from then on

@alexdima
Copy link
Copy Markdown
Member

alexdima commented Feb 3, 2021

Iff so then you would use our RPC only to exchange the ports and use a direct channel from then on

👍 x 💯

@bpasero
Copy link
Copy Markdown
Member

bpasero commented Feb 3, 2021

On top of that we should investigate if message ports can be used. With that we could connect webviews directly to the extension host and wouldn't need to go through the renderer anymore.

I like the idea of exploring this but please keep in mind that with the sandbox architecture, the extension host process maybe something different than what it is today. Today it is a child process of the renderer but in the future it might even just be a normal node.js process.

I am not entirely sure if node.js supports the same concepts (found this: https://nodejs.org/api/worker_threads.html#worker_threads_class_messageport)

PS: we ship with message ports for renderer <-> shared process communication for this stable release and it has proven to be very stable so far.

@jrieken
Copy link
Copy Markdown
Member

jrieken commented Feb 3, 2021

I like the idea of exploring this but please keep in mind that with the sandbox architecture, the extension host process maybe something different than what it is today.

I think that's a very similar question. In the sandbox world how do renderers communicate with the extension host? Replaying messaging on the main-process, use web sockets, or message ports?

@bpasero
Copy link
Copy Markdown
Member

bpasero commented Feb 3, 2021

I think that's a very similar question. In the sandbox world how do renderers communicate with the extension host? Replaying messaging on the main-process, use web sockets, or message ports?

This is something that @deepak1556 will start to explore this month. From the sandbox backlog:

image

I think whatever solution we pick, it cannot go through electron-main but must be a direct communication channel.

@mjbvz
Copy link
Copy Markdown
Collaborator Author

mjbvz commented Feb 3, 2021

Thanks for the feedback @alexdima! I'll continue exploring making this change in the webview code. I will also look into the our marshaling code to see if I can reuse some functions sheer

@jrieken I investigated using message port as well. I think using them should be possible but it requires some more thought for webviews.

In addition to the issues @bpasero brought up, the webview context is destroyed whenever the webview is hidden. This means that every time the webview is recreated (such as switching back to a custom editor), we'd have to go through the message port exchange again. Instead of waiting for this to complete before sending any messages, I think we could do something like:

  1. Webview is recreated
  2. We start passing the message port from the webview to the extension host
  3. If postMessage is called during this time, we pass those messages through the existing flow (extension host -> main -> webview)
  4. Once the extension host receives the message port, we start using it instead

@mjbvz mjbvz force-pushed the webview-typed-array-transfer branch from cde83c1 to 6f0ff98 Compare February 5, 2021 04:46
@mjbvz mjbvz added this to the February 2021 milestone Feb 5, 2021
@joaomoreno joaomoreno changed the base branch from master to main February 15, 2021 08:51
@mjbvz mjbvz modified the milestones: February 2021, March 2021 Feb 26, 2021
@mjbvz mjbvz force-pushed the webview-typed-array-transfer branch from 5f94a8f to ba8fa69 Compare March 18, 2021 03:21
@mjbvz mjbvz marked this pull request as ready for review March 19, 2021 01:39
@mjbvz
Copy link
Copy Markdown
Collaborator Author

mjbvz commented Mar 22, 2021

I'll merge this at the start of April for more testing

@mjbvz mjbvz modified the milestones: March 2021, April 2021 Mar 22, 2021
mjbvz added 4 commits March 29, 2021 23:33
This also makes it so that if you pass the same ArrayBuffer twice in an object, we use a single object on the receiver side too
@mjbvz mjbvz force-pushed the webview-typed-array-transfer branch from df22580 to 05ba482 Compare March 30, 2021 06:34
@mjbvz mjbvz enabled auto-merge (squash) March 30, 2021 06:37
@mjbvz mjbvz merged commit 3499f63 into microsoft:main Mar 30, 2021
@github-actions github-actions bot locked and limited conversation to collaborators May 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve transfer of ArrayBuffers to and from webviews

4 participants