futures-rs icon indicating copy to clipboard operation
futures-rs copied to clipboard

Concurrent stream combinators

Open ibraheemdev opened this issue 3 years ago • 3 comments

Currently we have the Stream::for_each_concurrent, buffer_unordered, and buffered, which use FuturesUnordered to run streams concurrently. Calling the combinators "buffered" is not great for visibility in my experience, and it might be more obvious if they were named concurrent/concurrent_unordered (or similar) with a limit: Option<usize> parameter. Or perhaps just introduce concurrent and make the limit field in buffered non-optional (undoing https://github.com/rust-lang/futures-rs/pull/2429). for_each_concurrent is also trivially implemented as stream.concurrent().for_each(...), so it may not be worth having.

ibraheemdev avatar Jun 05 '22 04:06 ibraheemdev

Note that https://github.com/rust-lang/futures-rs/pull/2429 is not released yet so we could easily add concurrent(_unordered) without breaking changes.

ibraheemdev avatar Jun 05 '22 04:06 ibraheemdev

Both renaming buffered/buffer_unordered and removing for_each_concurrent look like good ideas.

taiki-e avatar Jun 05 '22 04:06 taiki-e

If anyone decides to implement this, concurrently seems like a good name.

ibraheemdev avatar Jul 15 '22 03:07 ibraheemdev