feat: Retain final path stats if a Path is alive, add WeakPathHandle#386
feat: Retain final path stats if a Path is alive, add WeakPathHandle#386
Path is alive, add WeakPathHandle#386Conversation
|
Documentation for this PR has been generated and is available at: https://n0-computer.github.io/quinn/pr/386/docs/iroh_quinn/ Last updated: 2026-02-06T19:32:15Z |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #386 +/- ##
==========================================
+ Coverage 76.97% 77.34% +0.36%
==========================================
Files 81 81
Lines 23030 23154 +124
==========================================
+ Hits 17727 17908 +181
+ Misses 5303 5246 -57 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Performance Comparison Report
|
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5346.6 Mbps | N/A | N/A | 91.9% / 97.6% |
| medium-concurrent | 5336.1 Mbps | N/A | N/A | 90.8% / 96.9% |
| medium-single | 3792.1 Mbps | N/A | N/A | 88.6% / 97.3% |
| small-concurrent | 3706.4 Mbps | N/A | N/A | 90.2% / 99.1% |
| small-single | 3304.4 Mbps | N/A | N/A | 90.6% / 97.9% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2833.5 Mbps | 3714.3 Mbps | -23.7% |
| lan | 768.4 Mbps | 796.4 Mbps | -3.5% |
| lossy | 55.9 Mbps | N/A | N/A |
| wan | 83.8 Mbps | N/A | N/A |
Summary
iroh-quinn is 20.1% slower on average
2f9dafca52a3f874e17fbccfc72516e7eda4b1d0 - artifacts
No results available
062a68a079d24d0f4105fcf46e9c29311da15bac - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5378.1 Mbps | 7862.4 Mbps | -31.6% | 92.8% / 97.9% |
| medium-concurrent | 5312.2 Mbps | 7779.4 Mbps | -31.7% | 91.8% / 97.0% |
| medium-single | 3916.4 Mbps | 4487.5 Mbps | -12.7% | 90.3% / 98.1% |
| small-concurrent | 3734.3 Mbps | 5274.3 Mbps | -29.2% | 93.3% / 103.0% |
| small-single | 3338.3 Mbps | 4734.8 Mbps | -29.5% | 92.9% / 102.0% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2987.5 Mbps | 3685.6 Mbps | -18.9% |
| lan | 782.4 Mbps | 796.4 Mbps | -1.8% |
| lossy | 69.8 Mbps | 69.9 Mbps | ~0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 26.4% slower on average
1fc892fd06bf57fe328e64d1cc276d75baa42da7 - artifacts
No results available
e52d84a2fb6c3b521e42034852fea0d27c74d2fb - artifacts
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2966.3 Mbps | 3603.1 Mbps | -17.7% |
Summary
iroh-quinn is 17.7% slower on average
e4c3a5db6b2eb18ce04d8ea60b918a9f8b076af6 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5455.2 Mbps | 7752.3 Mbps | -29.6% | 93.2% / 97.9% |
| medium-concurrent | 5349.2 Mbps | 7026.4 Mbps | -23.9% | 91.0% / 96.9% |
| medium-single | 3904.3 Mbps | 4020.8 Mbps | -2.9% | 99.9% / 187.0% |
| small-concurrent | 3729.8 Mbps | 5081.4 Mbps | -26.6% | 91.0% / 99.1% |
| small-single | 3293.0 Mbps | 4389.0 Mbps | -25.0% | 85.3% / 95.9% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2862.0 Mbps | 3929.1 Mbps | -27.2% |
| lan | 768.5 Mbps | 804.8 Mbps | -4.5% |
| lossy | 69.8 Mbps | 55.9 Mbps | +25.0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 23.0% slower on average
3475d3be1b3cf0b196c9c0becfcaaa5ccd6996ac - artifacts
No results available
37f697eb5ba52913e248d571c2d8f4a80f38a012 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5356.0 Mbps | 8004.1 Mbps | -33.1% | 94.3% / 111.0% |
| medium-concurrent | 5395.0 Mbps | 7763.4 Mbps | -30.5% | 92.7% / 97.5% |
| medium-single | 3791.7 Mbps | 4189.4 Mbps | -9.5% | 99.8% / 168.0% |
| small-concurrent | 3629.2 Mbps | 4812.6 Mbps | -24.6% | 97.3% / 115.0% |
| small-single | 3305.4 Mbps | 4360.8 Mbps | -24.2% | 96.7% / 115.0% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2835.4 Mbps | 3904.5 Mbps | -27.4% |
| lan | 768.8 Mbps | 810.3 Mbps | -5.1% |
| lossy | 55.8 Mbps | 69.8 Mbps | -20.0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 25.8% slower on average
c932bdddc24887a3623599512d65149040df1612 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5341.8 Mbps | 7518.5 Mbps | -29.0% | 98.0% / 178.0% |
| medium-concurrent | 5460.6 Mbps | 7287.4 Mbps | -25.1% | 95.3% / 118.0% |
| medium-single | 3860.7 Mbps | 4307.7 Mbps | -10.4% | 93.4% / 119.0% |
| small-concurrent | 3877.1 Mbps | 4989.0 Mbps | -22.3% | 92.7% / 100.0% |
| small-single | 3472.7 Mbps | 4364.9 Mbps | -20.4% | 93.5% / 122.0% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2792.0 Mbps | 3631.4 Mbps | -23.1% |
| lan | 768.5 Mbps | 796.5 Mbps | -3.5% |
| lossy | 69.8 Mbps | 69.8 Mbps | ~0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 22.2% slower on average
52265737605d7a214551abb43f3dfc968036fa68 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5320.7 Mbps | 7939.8 Mbps | -33.0% | 93.3% / 101.0% |
| medium-concurrent | 5441.0 Mbps | 7801.7 Mbps | -30.3% | 90.8% / 96.7% |
| medium-single | 3563.1 Mbps | 4725.7 Mbps | -24.6% | 92.0% / 104.0% |
| small-concurrent | 3711.0 Mbps | 5205.0 Mbps | -28.7% | 102.7% / 194.0% |
| small-single | 3335.0 Mbps | 4751.1 Mbps | -29.8% | 89.2% / 97.0% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2807.5 Mbps | 3897.0 Mbps | -28.0% |
| lan | 768.4 Mbps | 796.5 Mbps | -3.5% |
| lossy | 69.8 Mbps | 55.9 Mbps | +25.0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 28.8% slower on average
6ce9604944e949e18cf6e39cf64bd829d0e270f1 - artifacts
No results available
7f68a3ec97150733100894ae0d812f12d869f035 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5638.9 Mbps | 8105.9 Mbps | -30.4% | 95.7% / 100.0% |
| medium-concurrent | 5636.6 Mbps | 7515.7 Mbps | -25.0% | 95.6% / 101.0% |
| medium-single | 4153.8 Mbps | 4363.6 Mbps | -4.8% | 98.2% / 151.0% |
| small-concurrent | 3938.1 Mbps | 4954.8 Mbps | -20.5% | 94.0% / 104.0% |
| small-single | 3579.7 Mbps | 4464.5 Mbps | -19.8% | 93.4% / 103.0% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2873.2 Mbps | 4009.2 Mbps | -28.3% |
| lan | 782.4 Mbps | 810.3 Mbps | -3.4% |
| lossy | 69.8 Mbps | 69.8 Mbps | ~0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 22.2% slower on average
e2007d41fafad049c813effe08ad6c2e3e4eefa0 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5249.2 Mbps | 8088.4 Mbps | -35.1% | 95.8% / 100.0% |
| medium-concurrent | 5277.2 Mbps | 7802.1 Mbps | -32.4% | 95.0% / 101.0% |
| medium-single | 4059.7 Mbps | 4749.8 Mbps | -14.5% | 99.2% / 153.0% |
| small-concurrent | 3738.5 Mbps | 4999.8 Mbps | -25.2% | 91.4% / 99.4% |
| small-single | 3453.9 Mbps | 4836.0 Mbps | -28.6% | 94.3% / 104.0% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2841.5 Mbps | 4090.2 Mbps | -30.5% |
| lan | 770.2 Mbps | 810.3 Mbps | -5.0% |
| lossy | 69.9 Mbps | 55.9 Mbps | +25.0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 28.1% slower on average
93d27db89f3b069ba32eda9e55e1575a250ac595 - artifacts
Raw Benchmarks (localhost)
| Scenario | iroh-quinn | upstream | Delta | CPU (avg/max) |
|---|---|---|---|---|
| large-single | 5923.2 Mbps | 7945.0 Mbps | -25.4% | 97.2% / 98.8% |
| medium-concurrent | 5401.1 Mbps | 7623.9 Mbps | -29.2% | 95.5% / 97.4% |
| medium-single | 4172.4 Mbps | 4749.0 Mbps | -12.1% | 96.5% / 98.5% |
| small-concurrent | 3669.5 Mbps | 5359.0 Mbps | -31.5% | 96.5% / 98.8% |
| small-single | 3561.7 Mbps | 4748.8 Mbps | -25.0% | 96.1% / 98.3% |
Netsim Benchmarks (network simulation)
| Condition | iroh-quinn | upstream | Delta |
|---|---|---|---|
| ideal | 2873.4 Mbps | 4101.9 Mbps | -30.0% |
| lan | 782.3 Mbps | 810.3 Mbps | -3.5% |
| lossy | 69.9 Mbps | 69.8 Mbps | ~0% |
| wan | 83.8 Mbps | 83.8 Mbps | ~0% |
Summary
iroh-quinn is 25.2% slower on average
2f9dafc to
062a68a
Compare
1fc892f to
e52d84a
Compare
Path is alive, add WeakPathHandlePath is alive, add WeakPathHandle
flub
left a comment
There was a problem hiding this comment.
Generally looks fine I think. I also tried to look for some alternative for the expect of the path stats, but it is pretty difficult to do so.
| let path_stats = self.path_stats(path_id).unwrap_or_default(); | ||
| self.path_stats.remove(&path_id); |
There was a problem hiding this comment.
I'm not sure why this needs to move, this whole function has &mut self so moving this a few lines does nothing?
(and also not sure why it's on two lines now..)
There was a problem hiding this comment.
Self::path_stats sets the current RTT onto the stats, whereas the PathStats returned from remove will always have an RTT of 0.
quinn/src/connection.rs
Outdated
| } | ||
|
|
||
| /// Returns `true` if the two [`WeakConnectionHandle`] point at the same connection. | ||
| pub fn ptr_eq(&self, other: &Self) -> bool { |
There was a problem hiding this comment.
Bikeshedding alert: not sure I like this much as a public API. Why do folks suddenly have to worry about pointers?
But also what to suggest? is_same_connection?
Also not entirely sure how much I dislike this, if more folks like this that would be fine by me. I tried to find some precedent for this kind of API in other libs but didn't find any.
There was a problem hiding this comment.
Arc::ptr_eq is where I took the name from. Changing to is_same_connection sgtm.
There was a problem hiding this comment.
Renamed it to is_same_conn
quinn/src/connection.rs
Outdated
| .or_else(|| self.final_path_stats.get(&path_id).copied()) | ||
| } | ||
|
|
||
| /// Increment the application-level reference counter for a path. |
There was a problem hiding this comment.
"the application" is usually the user of quinn, so "application-level" is a bit confusing to me here. Maybe "increments the [Connection]s reference counter for a path" works?
Co-authored-by: Floris Bruynooghe <flub@n0.computer>
…watcher (#3899) ## Description Improves the path watcher for connections to reliably allow accessing the stats of abandoned paths. * Abandoned paths are not removed from the value of the watchable for a connection's paths, as long as there are active watchers (subscribers) * Only once all watchers are dropped, we drop the abandoned paths from the watchable's value on the next update * This means that if you keep a `PathWatcher` alive for the duration of a connection, it will contain *all* paths the connection ever used. * We use the changes from n0-computer/noq#386 and keep a `WeakPathHandle` for all paths in the watchable's value. This prevents the path stats to be dropped within a quinn connection even if the path is abandoned. * We do no longer store path stats anywhere in iroh directly, but only ever access them from quinn. Due to keeping the `WeakPathHandle`, we can ensure that the stats are always available as long as the connection hasn't been dropped. The combination of all this gives us a reliable way to access final path stats for all paths used in a connection, as long as you keep a reference to the connection around, which is quite straightforward to do and document. ## Breaking Changes * `Connection::paths` and `ConnectionInfo::paths` now return a `PathWatcher` (which still implements `n0_watcher::Watcher` but now also is a named struct) * `PathInfo::stats` and `PathInfo::rtt` now return `Option`. They return `None` if the underlying connection has been droped. ## Notes & open questions I spend quite some time going back-and-forth over different approaches. Very open to other ideas, but I'm a bit out of further ideas currently and this is the best I could come up with so far. ## Change checklist <!-- Remove any that are not relevant. --> - [ ] Self-review. - [ ] Documentation updates following the [style guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text), if relevant. - [ ] Tests if relevant. - [ ] All breaking changes documented. - [ ] List all breaking changes in the above "Breaking Changes" section. - [ ] Open an issue or PR on any number0 repos that are affected by this breaking change. Give guidance on how the updates should be handled or do the actual updates themselves. The major ones are: - [ ] [`quic-rpc`](https://github.com/n0-computer/quic-rpc) - [ ] [`iroh-gossip`](https://github.com/n0-computer/iroh-gossip) - [ ] [`iroh-blobs`](https://github.com/n0-computer/iroh-blobs) - [ ] [`dumbpipe`](https://github.com/n0-computer/dumbpipe) - [ ] [`sendme`](https://github.com/n0-computer/sendme)
Description
Once a path is abandoned, its state is immediately dropped from
quinn_proto::Connection. The final path stats are emitted as an event, which is handled inquinn::Connectionand forwarded over the path events broadcast channel.This construction makes sure that no state is kept around for abandoned paths to prevent memory leaks. However, it gives consumers a hard time, as we experience in iroh with the difficulties in accessing the final path stats.
By sending the final path stats over a broadcast channel while they are already removed from the underlying connection, consumers of the API have this weird in-between state where the final stats for an abandoned path are not accessible for the period of time between t1: remove stats from connection and send into broadcats channel and t2: consumer of path event broadcast channel receives the event. Between t1 and t2, the final stats are not accessible at all.
This PR changes quinn such that within a
quinn::Connectionwe keep a reference counter forPathstructs currently alive. As long as aPathstruct is alive, once a path is abandoned we store the path stats within thequinn::Connectionuntil all references to this path are dropped.This means that
Path::statsnow can unconditionally return the stats for that path, even if the path was abandoned in the meantime.Great! However, still not enough for iroh. Keeping a
quinn::Pathstruct around prevents thequinn::Connectionfrom being closed-on-drop. In iroh, we want a "passive" path watcher which does not keep the connection alive, but still ensures that path stats for the watched paths are not dropped.Therefore, this PR additionally adds a
WeakPathHandle. It contains aWeakConnectionHandleand aPathId, so it does not prevent a connection from being closed-on-drop. However, it does take part in the reference counting of path refs, and thus does prevent the stats for abandoned paths to be dropped as long as there is aWeakPathHandlealive.Breaking Changes
Notes & open questions
If there are ideas on how to handle this simpler, I'm all ears. I spent quite some time thinking this through, and this is the best I could come up so far.