Reduce size of frames sent to websockets to 10MB#5052
Closed
Reduce size of frames sent to websockets to 10MB#5052
Conversation
This also forces us to thread a keyword down through from to_frames to frame_split_size.
Member
Author
|
cc @ian-r-rose |
2 tasks
Member
Author
|
Maybe we should have a different shard default for websockets instead. It would be nice to let people increase this limit if they wanted. |
Collaborator
I would probably appreciate this option, even if it's pretty esoteric. |
Member
Author
|
@ian-r-rose do you want to take this PR over? |
Collaborator
|
Sure, happy to do so |
Member
Author
|
Are there other options for websockets that we might want? If so, maybe distributed:
comm:
websockets:
shard: 5MiB |
2 tasks
mrocklin
pushed a commit
that referenced
this pull request
Jul 15, 2021
Supersedes #5052 . In addition to making the default websocket maximum-frame-size smaller, this makes the specific value configurable. It's somewhat redundant with distributed.comm.shard, but the constraints on websockets are sufficiently different that a separate config seems okay. This does not implement the fix in #5061, as that would read a config value for every frame, which is costly. So the config value will in general not be changed after import time.
Member
Author
|
Superceded by #5070 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This also forces us to thread a keyword down through from to_frames to frame_split_size.
Websockets are often used in situations where large frames are not allowed. This restricts the size of any frame sent through the websocket protocol to 10MB.
This isn't a great solution for a few reasons:
But the work of threading through the keyword is probably valuable, and maybe it helps to inspire a better solution.