Skip to content

save should stream data whenever possible #7590

@rgwood

Description

@rgwood

Related problem

save collects input data in a few situations where it doesn't always need to:

output.into_value(span)
}
None => input.into_value(span),

input => match input.into_value(span) {

Describe the solution you'd like

save should always stream ListStream input into a file instead of attempting to collect it all.

It should be possible to stream to a file slowly:

1..100 | each { |i| sleep 400ms; $i} | save -f output.txt

And watch individual lines (or batches of lines) arrive with tail --follow in another shell:

tail --follow output.txt

Additional context and details

Hat tip to this comment: #6178 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    A:streamingIssues related to streaming data (or collecting data when it should be streamed)category:enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions