Skip to content

Why RawStream contains an Iterator of Vec<u8> instead of std::io::Read? #7017

@Kixunil

Description

@Kixunil

Question

RawStream is an iterator of heap-allocated chunks of bytes as opposed to std::io::Read which may harm performance in some cases. The only reason that comes to my mind is having a different error type but that could've been implemented by having a custom Read trait that's a copy of std::io::Read except for error bytes.

Is there any other reason I don' see? If not how hard would it be to refactor it?

Additional context and details

I started working on optimizing json objects parsing to not collect the whole stream into string as mentioned in #6979 (comment) and assumed it implements Read which is not the case. I even implemented a bunch of optimizations aiming at significantly reducing heap allocations. 🤦‍♂️ I'm now wondering how to proceed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A:streamingIssues related to streaming data (or collecting data when it should be streamed)deprecated:questionQuestions should be redirected to GitHub discussions

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions