-
Notifications
You must be signed in to change notification settings - Fork 7.4k
[Data] Make Zip a properly streaming operator #56300
Copy link
Copy link
Open
Labels
P1Issue that should be fixed within a few weeksIssue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tdataRay Data-related issuesRay Data-related issuesperformance
Description
What happened + What you expected to happen
Right now zip unnecessarily materializes both datasets in memory.
Instead it should just be zipping both operators in a streaming fashion.
Versions / Dependencies
2.49
Reproduction script
NA
Issue Severity
None
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P1Issue that should be fixed within a few weeksIssue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tdataRay Data-related issuesRay Data-related issuesperformance
Type
Projects
Status
In progress