feat: Add a way to reserve many permits on bounded mpsc channel by Totodore · Pull Request #6205 · tokio-rs/tokio

Totodore · 2023-12-09T15:08:05Z

Motivation

I am in a situation where I need to be able to efficiently reserve n permit from the same mpsc::bounded channel. Because the Semaphore API already implement the possibility to reserve n permit. It is quite easy to propagate this API to the mpsc::bounded API. More details on this need can be found on the corresponding issue :

Fixes Add a way to request many permits on a mpsc::channel #6174

Solution

In order to avoid any API breaking change rather than adding a u32 field on the current Permit struct I added :

a ManyPermit struct
a reserve_many(n: u32) fn on the Sender struct
a try_reserve_many(n: u32) fn on the Sender struct

Co-authored-by: Tymoteusz Wiśniewski <tymo@tymo.xyz>

Co-authored-by: Alice Ryhl <aliceryhl@google.com>

…rnal `Semaphore`

Co-authored-by: oliver <151407407+kwfn@users.noreply.github.com>

…count

Totodore · 2023-12-10T22:16:28Z

The scope of this PR has evolved a bit.
The internal batch_semaphore has been changed to take a usize permit count rather than a u32 permit count so that there is no conversion from usize to u32 because it might overflow.

Darksonn · 2023-12-11T12:42:45Z

I think this looks reasonable. However, we will need some tests as well. Here are the tests that come to mind:

A test that just uses it to send n messages, then receives n messages. (You can wrap it in a for loop to try different values of n.)
A test that acquires n permits, sends k messages, and then drops the iterator. The capacity should be correct at the end. (Again with a for loop for a few differen t values of n and k.)
Test that you get a SendError in the right cases.
Test edge-cases, e.g. what happens if n == 0 or n == MAX_PERMITS or n == usize::MAX?

Co-authored-by: Alice Ryhl <aliceryhl@google.com>

Darksonn

This looks good to me. I came up with one minor improvement, but overall I think this is ready to be merged.

Totodore · 2024-01-02T16:24:23Z

Ready to merge :).

Darksonn · 2024-01-02T16:29:08Z

Thanks!

Bumps tokio from 1.35.1 to 1.36.0. Release notes Sourced from tokio's releases. Tokio v1.36.0 1.36.0 (February 2nd, 2024) Added io: add tokio::io::Join (#6220) io: implement AsyncWrite for Empty (#6235) net: add support for anonymous unix pipes (#6127) net: add UnixSocket (#6290) net: expose keepalive option on TcpSocket (#6311) sync: add {Receiver,UnboundedReceiver}::poll_recv_many (#6236) sync: add Sender::{try_,}reserve_many (#6205) sync: add watch::Receiver::mark_unchanged (#6252) task: add JoinSet::try_join_next (#6280) Changed io: make copy cooperative (#6265) io: make repeat and sink cooperative (#6254) io: simplify check for empty slice (#6293) process: use pidfd on Linux when available (#6152) sync: use AtomicBool in broadcast channel future (#6298) Documented io: clarify clear_ready docs (#6304) net: document that *Fd traits on TcpSocket are unix-only (#6294) sync: document FIFO behavior of tokio::sync::Mutex (#6279) chore: typographic improvements (#6262) runtime: remove obsolete comment (#6303) task: fix typo (#6261) #6220: tokio-rs/tokio#6220 #6235: tokio-rs/tokio#6235 #6127: tokio-rs/tokio#6127 #6290: tokio-rs/tokio#6290 #6311: tokio-rs/tokio#6311 #6236: tokio-rs/tokio#6236 #6205: tokio-rs/tokio#6205 #6252: tokio-rs/tokio#6252 #6280: tokio-rs/tokio#6280 #6265: tokio-rs/tokio#6265 #6254: tokio-rs/tokio#6254 #6293: tokio-rs/tokio#6293 #6238: tokio-rs/tokio#6238 #6152: tokio-rs/tokio#6152 #6298: tokio-rs/tokio#6298 #6262: tokio-rs/tokio#6262 #6303: tokio-rs/tokio#6303 #6261: tokio-rs/tokio#6261 ... (truncated) Commits eaf81ed chore: prepare Tokio v1.36.0 (#6312) 53f9e5a ci: make sure dictionary words are sorted and unique (#6316) 9077762 net: expose keepalive option on TcpSocket (#6311) 131e7b4 ci: add spellchecking (#6297) e53b92a io: clarify clear_ready docs (#6304) 7536132 sync: use AtomicBool in broadcast channel future (#6298) b6d0c90 macros: fix trait_method breaking change detection (#6308) 4846959 runtime: remove obsolete comment (#6303) ec30383 net: add UnixSocket (#6290) f80bbec io: simplify check for empty slice (#6293) Additional commits viewable in compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase. Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: @dependabot rebase will rebase this PR @dependabot recreate will recreate this PR, overwriting any edits that have been made to it @dependabot merge will merge this PR after your CI passes on it @dependabot squash and merge will squash and merge this PR after your CI passes on it @dependabot cancel merge will cancel a previously requested merge and block automerging @dependabot reopen will reopen this PR if it is closed @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Totodore added 3 commits December 9, 2023 14:51

mpsc: add a way to reserve many permit on bounded mpsc chan

14d0af6

mpsc: apply clippy rules + fmt

92c26b5

mpsc: export the ManyPermit struct

b684f3c

github-actions Bot added the R-loom-sync Run loom sync tests on this PR label Dec 9, 2023

mpsc: fix fmt

c40178b

ghost reviewed Dec 9, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Darksonn added A-tokio Area: The main tokio crate M-sync Module: tokio/sync labels Dec 9, 2023

Darksonn reviewed Dec 9, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Totodore added 3 commits December 10, 2023 00:00

mpsc: return an Iterator impl for the *_reserve_many fns

53fb399

mpsc: fix rustdoc

242f171

mpsc: fix fmt

81c91da

Totodore requested a review from Darksonn December 10, 2023 00:11

satakuma reviewed Dec 10, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Totodore and others added 2 commits December 10, 2023 10:26

mpsc: add an implementation for ExactSizeIterator

ad21c6c

mpsc: apply doc suggestions

71c48ba

Co-authored-by: Tymoteusz Wiśniewski <tymo@tymo.xyz>

Darksonn reviewed Dec 10, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Darksonn reviewed Dec 10, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Totodore and others added 3 commits December 10, 2023 11:43

mpsc: fix Debug implementation for PermitIterator

8dd3078

Co-authored-by: Alice Ryhl <aliceryhl@google.com>

mpsc: move the u32 n to usize and only cast to u32 for the inte…

9d9435b

…rnal `Semaphore`

mpsc: improve doc

28d8c51

Co-authored-by: oliver <151407407+kwfn@users.noreply.github.com>

Totodore requested a review from Darksonn December 10, 2023 12:20

mpsc: change the internal batch_semaphore API to take usize permit …

08a8622

…count

mpsc: fix usize casting in special features

5f9cb1a

Darksonn reviewed Dec 11, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs

Totodore added 3 commits December 11, 2023 11:46

mpsc: fix usize casting in try_reserve fn for batch_semaphore

5bcc3f9

mpsc: return a SendError if n > MAX_PERMIT for *reserve_many fns

a3fd29e

mpsc: return a SendError if n > max_capacity for *reserve_many fns

419bfd3

Totodore added 11 commits December 23, 2023 11:09

mpsc: test reserve_many_on_closed_channel

ddd44d1

mpsc: doc mention reserve_many for Cancel Safety part

f103641

mpsc: improve doc for reserve_many fn

b688a05

mpsc: fix formatting

3b705be

mpsc: fix formatting

430286b

mpsc: switch to maybe_tokio_test

58f7648

mpsc: add tests for try_reserve_many

a25fb7c

Merge branch 'master' into feat-mpsc-many-permit

2bd5c8b

mpsc: add an early return if n == 0 to avoid acquire mechanism

bd8c3e6

mpsc: improve doc for reserve_many

6988079

mpsc: remove early return for reserve_inner

133f2c1

Totodore requested a review from Darksonn December 23, 2023 16:16

Darksonn reviewed Dec 30, 2023

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Comment thread tokio/tests/sync_mpsc.rs

Comment thread tokio/src/sync/mpsc/bounded.rs Outdated

Totodore and others added 6 commits January 2, 2024 13:35

mpsc: remove useless empty iterator guard for try_reserve_many

c39878e

mpsc: apply doc suggestion

f7025ce

mpsc: Apply suggestions from code review

0d19901

Co-authored-by: Alice Ryhl <aliceryhl@google.com>

mpsc: fix sync_mpsc tests

3fdb5d9

mpsc: fix sync_mpsc tests

e5f1df8

Merge branch 'master' into feat-mpsc-many-permit

830188e

Totodore requested a review from Darksonn January 2, 2024 15:00

Darksonn approved these changes Jan 2, 2024

View reviewed changes

Comment thread tokio/src/sync/mpsc/bounded.rs

mpsc: early return for empty PermitIterator for drop logic

ec8b537

Darksonn enabled auto-merge (squash) January 2, 2024 16:29

Darksonn merged commit 7c606ab into tokio-rs:master Jan 2, 2024

Totodore deleted the feat-mpsc-many-permit branch January 2, 2024 23:54

maminrayej mentioned this pull request Jan 30, 2024

chore: prepare Tokio v1.36.0 #6312

Merged

cedricschwyter mentioned this pull request Feb 2, 2024

chore(deps): bump tokio from 1.35.1 to 1.36.0 in /src-tauri woollygoods/huehuehue#181

Merged

Uh oh!

Conversation

Totodore commented Dec 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Solution

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Totodore commented Dec 10, 2023

Uh oh!

Uh oh!

Darksonn commented Dec 11, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Darksonn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Totodore commented Jan 2, 2024

Uh oh!

Darksonn commented Jan 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Totodore commented Dec 9, 2023 •

edited

Loading