Skip to content

feat: Implement custom transports#3845

Merged
rklaehn merged 63 commits intomainfrom
feat-user-transport-2
Mar 5, 2026
Merged

feat: Implement custom transports#3845
rklaehn merged 63 commits intomainfrom
feat-user-transport-2

Conversation

@rklaehn
Copy link
Copy Markdown
Contributor

@rklaehn rklaehn commented Jan 13, 2026

Description

Adds user defined transports.

Transports have to impl a dynable trait, currently named CustomTransport. Implementing this requires implementing another dynable trait CustomSender. Creating a transport requires implementing yet another dynable trait CustomTransportFactory.

A custom addr is just an opaque blob with an u64 tranport id. Small blobs will be inlined, but that is an implementation detail.

Formatting currently is "-"

Replaces #3707

Breaking Changes

Notes & open questions

  • Question: encoding of custom addrs in DNS records.

    • Option 1: addr=1_a1b2c3 (tries to parse first as SocketAddr then as custom addr)
    • Option 2: custom=1_a1b2c3 (separate IrohAttr::Custom)
  • Question: Feature gate or not? User transports require exposing a bit more of the guts, e.g. transmit fields, so we might want to put this behind a feature flag that is documented to be not part of the stable API.

  • Question: Path selection.

    Currently in select_best_path we never select an user path. For some reason it still works, but we probably should put in some logic there.

        Some((transports::Addr::User(_), _)) => {
            // todo: when should we select an user path?
            None
        }
  • DRY the different mapped addrs?

    We could DRY the 2 differnet mapped addrs a bit more. But once you start with this you get into a rabbit hole. Should relay transports just be a special kind of custom transport? If we could abstract the logic when to choose the relay, this might work. Maybe this is best left for later.

Update: path selection is now taken care of. Basically each path gets a 2-tuple consisting of primary/secondary and biased rtt, then sort and take the best. There is an API to add a bias for each addr type and also to configure for each transport if it is a transport of last resort.

Here is how to bias a transport:

Endpoint::builder()
    .transport_bias(
        AddrKind::Custom(TEST_TRANSPORT_ID),
        TransportBias::primary().with_rtt_advantage(CUSTOM_TRANSPORT_RTT_ADVANTAGE))
    .build().await?;

I also added some tests to make sure custom transports work if the ip transport is enabled.

@rklaehn rklaehn closed this Jan 13, 2026
@rklaehn rklaehn reopened this Jan 13, 2026
@rklaehn rklaehn changed the base branch from feat-user-transport to main January 13, 2026 10:50
Missing: the watcher rube goldberg types!
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 13, 2026

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/3845/docs/iroh/

Last updated: 2026-03-04T16:00:35Z

@n0bot n0bot bot added this to iroh Jan 13, 2026
@github-project-automation github-project-automation bot moved this to 🏗 In progress in iroh Jan 13, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 13, 2026

Netsim report & logs for this PR have been generated and is available at: LOGS
This report will remain available for 3 days.

Last updated for commit: b835f2f

@rklaehn rklaehn changed the title Ignore - just trying to fix merge conflicts feat: Implement user transports Jan 13, 2026
Plus some refactoring to make the types a bit less frightening.

Also reduce the amount of code behind the wasm_browser config so you can
work on it without having to compile to wasm.
@rklaehn rklaehn force-pushed the feat-user-transport-2 branch 2 times, most recently from fc0b076 to a6d5dad Compare January 13, 2026 12:25
@rklaehn rklaehn force-pushed the feat-user-transport-2 branch from a6d5dad to aabdc93 Compare January 13, 2026 12:33
@rklaehn rklaehn added this to the iroh: v1.0.0-rc.0 milestone Jan 13, 2026
@rklaehn rklaehn self-assigned this Jan 13, 2026
@matheus23
Copy link
Copy Markdown
Member

  • Question: Path selection.
    Currently in select_best_path we never select an user path. For some reason it still works, but we probably should put in some logic there.
    rust Some((transports::Addr::User(_), _)) => { // todo: when should we select an user path? None }

The examples for user transports that I can come up with are a WebRTC transport, a WifiAware transport, and a BLE transport.
All of these should be preferred over relays, if possible.
If both IP transports and one of these custom transports end up working, then they should compete on RTT.

So that would mean they should basically be regarded as similar to the IP transports?

  • DRY the different mapped addrs?
    We could DRY the 2 differnet mapped addrs a bit more. But once you start with this you get into a rabbit hole. Should relay transports just be a special kind of custom transport? If we could abstract the logic when to choose the relay, this might work. Maybe this is best left for later.

Relays are still too special. Even if we don't have any transports, we can still open relay connections (the relay connection management isn't self-contained in the relay transport). We also use relays for QAD.
I think we're far off from implementing relays as custom transports, and I don't think there's much reason for trying to do that.

@rklaehn
Copy link
Copy Markdown
Contributor Author

rklaehn commented Jan 13, 2026

Relays are still too special. Even if we don't have any transports, we can still open relay connections (the relay connection management isn't self-contained in the relay transport). We also use relays for QAD.
I think we're far off from implementing relays as custom transports, and I don't think there's much reason for trying to do that.

Fully agree. I just had to mention it, since you got a lot of places where relays are being treated in a very similar way to user addrs. But it is perfectly fine to keep them separate for now (or maybe forever... )

Comment on lines +1162 to +1165
Some((transports::Addr::User(_), _)) => {
// todo: when should we select an user path?
None
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This currently just says we never switch away from user transports, even if "better" ones are available.

We need to figure this out before merging. To reason about this, I'm thinking about example user transports:

  • WifiAware
  • WebRTC
  • In-process simulated networks
  • Bluetooth

I'd say that we should prefer IP paths if the work and have lower RTT.
So that'd mean that here we're checking if current_rtt >= new_rtt + RTT_SWITCHING_MIN_IP similar to above. And if that is true, we resolve to best_ip_path, if not, we keep the user path.

Additionally, we should check if we have a user transport available, and switch away from None, Relay or Ip to a user path in the match cases above, if the user transport ends up having a lower RTT.


Tbh, this could become quite complicated quickly. E.g. maybe some user transports actually want to stay in the background (with PATH_BACKUP), and only be used if (once we have an API for that) a stream is specifically attached to that path.
We should probably think about how much wiggle-room we have if we'd set the API in stone for 1.0.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the new API. Basically this is solved by #3870. For each transport (=address type) you can have a rtt bias (positive or negative), and you can configure for each address type if it is a normal transport or a transport of last resort.

Then within each group transports compete according to their biased rtt, and there is stickiness. For switching the group there is no stickiness.

@rklaehn rklaehn force-pushed the feat-user-transport-2 branch from 71b17c9 to 98edf8f Compare February 5, 2026 13:03
/// Only the client closes paths, just like only the client opens paths. This is to
/// avoid the client and server selecting different paths and accidentally closing all
/// paths.
fn close_redundant_paths(&mut self, selected_path: &transports::Addr) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this account for custom transports in some form?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. A custom path is just another path, so I guess this does not have to be special cased for custom paths.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we special case IP paths in here, that's why I am wondering

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh god. I guess you are right.

We are using is_ip as a "this is a good path" stand-in for now. Not just here, but in a lot of places.

I guess we would have to change is_ip with checking if the path is a "primary". But you can't see that from an addr - custom transports can be either. So it would have to be a lookup in bias_map.

Something like this:

+        let is_primary = |addr: &transports::Addr| {
+            self.transport_bias
+                .get(addr)
+                .transport_type == transports::TransportType::Primary
+        };
+
         for (conn_id, conn_state) in self.connections.iter() {
             for (path_id, path_remote) in conn_state
                 .open_paths
                 .iter()
-                .filter(|(_, addr)| addr.is_ip())
+                .filter(|(_, addr)| is_primary(addr))
                 .filter(|(_, addr)| *addr != selected_path)
             {
-                if conn_state.open_paths.values().filter(|a| a.is_ip()).count() <= 1 {
+                if conn_state.open_paths.values().filter(|a| is_primary(a)).count() <= 1 {
                     continue; // Do not close the last direct path.
                 }
                 if let Some(path) = conn_state

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without custom transports is_primary returns true only for ip, so the behaviour would be unchanged.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are only a few non-test places other than this where is_ip is used. BUT... I think custom transports as implemented now introduce the concept of a backup transport, that gets sorted together with the relay transports, but is not a relay.

I wonder if we should just drop this and say that all custom transports are direct. They would have to compete with Ip transports on the basis of their (biased) rtt. That would probably simplify not mixing up all the relay special cases with custom transports.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could flip is_ip to just !is_relay and call it a day...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am honestly not sure what to do here yet, lets make a todo/note in the comment and open an issue about it, so we don't forget about it

@rklaehn rklaehn force-pushed the feat-user-transport-2 branch from b0d5ad8 to 397d0ed Compare February 18, 2026 12:37
For now, backup is reserved for relay.
We treat all non-relay paths as direct paths.

Custom transports compete with Ip paths based on biased rtt.
# Conflicts:
#	iroh/src/endpoint.rs
#	iroh/src/socket.rs
#	iroh/src/socket/remote_map/remote_state.rs
@rklaehn rklaehn marked this pull request as ready for review March 4, 2026 13:23
@rklaehn rklaehn requested a review from dignifiedquire March 4, 2026 15:59
@rklaehn rklaehn added this pull request to the merge queue Mar 5, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 5, 2026
@rklaehn rklaehn added this pull request to the merge queue Mar 5, 2026
Merged via the queue into main with commit d434c85 Mar 5, 2026
28 of 29 checks passed
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in iroh Mar 5, 2026
@rklaehn rklaehn deleted the feat-user-transport-2 branch March 6, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

6 participants