Skip to content

Conversation

@claraphyll
Copy link
Contributor

Purpose

This pull request allows syncthing to request an IPv6 pinhole, addressing issue #7406. This helps users who prefer to use IPv6 for hosting their services or are forced to do so because of CGNAT. Otherwise, such users would have to configure their firewall manually to allow syncthing traffic to pass through while IPv4 users can use UPnP to take care of network configuration already.

Testing

I have tested this in a virtual machine setup with miniupnpd running on the virtualized router. It successfully added an IPv6 pinhole when used with IPv6 only, an IPv4 port mapping when used with IPv4 only and both when dual-stack (IPv4 and IPv6) is used.

Automated tests could be added for SOAP responses from the router but automatically testing this with a real network is likely infeasible.

Documentation

https://docs.syncthing.net/users/firewall.html could be updated to mention the fact that UPnP now works with IPv6, although this change is more "behind the scenes".

Authorship

Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.

Copy link
Member

@acolomb acolomb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't tested it yet but looking forward to doing so.

Some minor questions below just from looking at the code.

Copy link
Member

@imsodin imsodin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

I have some general comments on the code - I didn't check/test the actual upnp procedures.

@bt90
Copy link
Contributor

bt90 commented Jul 30, 2023

Does this also work if temporary addresses are used? The address being published via global discovery would need to match the IP we're requesting the pinhole for. see #8453 for more context.

@claraphyll
Copy link
Contributor Author

Does this also work if temporary addresses are used? The address being published via global discovery would need to match the IP we're requesting the pinhole for. see #8453 for more context.

This seems to work. In my testing setup, it opens a pinhole to the SLAAC-assigned IPv6 which seems to be the one advertised to the discovery service

@claraphyll claraphyll requested review from acolomb and imsodin July 30, 2023 17:02
@acolomb
Copy link
Member

acolomb commented Jul 30, 2023

@maxi0604 I just tested this on my home LAN, with an IPv6 and CGNAT IPv4 connection to the ISP. It seems to work partially, but I couldn't really make sense of the debugging log messages. Would you care to look at the debug logs? How can I send them to you privately?

@claraphyll
Copy link
Contributor Author

@maxi0604 I just tested this on my home LAN, with an IPv6 and CGNAT IPv4 connection to the ISP. It seems to work partially, but I couldn't really make sense of the debugging log messages. Would you care to look at the debug logs? How can I send them to you privately?

@acolomb I just noticed that my e-mail wasn't visible on github, you should be able to mail them to me now

@acolomb
Copy link
Member

acolomb commented Jul 30, 2023

Sent the logs and tried once more with explicit listen addresses set for quic6://[::]:22123 and tcp6://[::]:22123. Strange enough, the router still says the device can open ports on its own but doesn't have any active mappings. The log does mention that the mapping was added, but it gets removed again exactly 7 minutes later.

@acolomb
Copy link
Member

acolomb commented Jul 30, 2023

Now it does open a port, but the router says it's for IPv4 (TCP/22123).

Still wondering about those connect: invalid argument and Error getting new lease on NAT-PMP@192.168.0.1 unexpected result size 12, expected 16 errors by the way.

@claraphyll
Copy link
Contributor Author

claraphyll commented Jul 30, 2023

I haven't changed anything substantial for NAT-PMP so that error message is weird but likely not related.

With UPnP I think the code has to become a little bit more complicated. The assumption I made was that I could just have the code create a NAT device for IPv4 which contacts the gateway via IPv4 and one for IPv6 which contacts the gateway via IPv6. Then I could just use the local IP used to connect to the gateway for the request, like in the IPv4 case.

There's already a comment in the code about how that is a roundabout approach for IPv4 but in the log you sent it breaks completely for IPv6. I am thinking that the reason is that your router advertises the IPv6 UPnP service under a link-local address which a) we can't connect to without a zone index b) even if we could just connect, as mentioned before, the IP guessing mechanism breaks here since connecting to a link-local local address will probably select the link-local address of the interface the syncthing instance is running on, which we can't open pinhole to by definition.

I'm thinking the solution looks something like this:

  1. Allow delivering both port mapping and pinholing requests via IPv4 and IPv6. This is done with commit 5a16ea4.
  2. Somehow get the LocalIP of an UPNP nat.Device to be equal to the GUA IPv6 we're listening on if it's a WANIPv6FirewallControl device or equal to the local IPv4 if it is one of the other types. As mentioned before here, ideally the IPv6 here is the same one sent to the discovery service
  3. Handle zone indices so that UPnP traffic can be delivered via link-local IPv6 addresses, making this work on networks that are truly IPv6 only with a router configured like the one in your logs (This is lower priority as pinholing should already work when the first two are implemented but would depend on either IPv4 being available on the local network or the gateway not sending a link-local IPv6)

@acolomb
Copy link
Member

acolomb commented Nov 16, 2023

Fine with me to get this finally completed, without further digging into the deduplication stuff or the Windows failure for now. Of course, a follow-up will be very welcome and I do want to help there with further testing and discussion. But let's get this out to users in a basic form first. Hope I can find some time to test and review again soon. We just missed the -rc.1 release so I guess there's time until the next cycle.

I'll have to think about your comments on the proposed logic, it's been too long since. Just one thought: The fact that the actual listeners are not considered when starting discovery and contacting mapping services, is something we should fundamentally reconsider. I think that will solve our current problems with duplicate mappings much easier.

@claraphyll
Copy link
Contributor Author

Should we add a GOOS check with a Todo?

I think it might be clever to only check for GOOS after the call has failed and provide a more useful error message. That way, no one needs to remember to change it once the issue in Go is fixed.

@bt90
Copy link
Contributor

bt90 commented Dec 8, 2023

@calmh Where do we go from here? We've missed the release candidate again, and I'm not sure the code has had enough review to be merged for RC2.

@calmh
Copy link
Member

calmh commented Dec 8, 2023

I see a hilarious amount of discussion, no approvals (the opposite, two open "changes requested" reviews), and the last comments hint towards outstanding issues ("should we add" ... "I think ..."). I also actively avoid getting involved in PR review when there's already multiple other people reviewing it, so as to avoid pile-on. RC timing has nothing to do with merging PRs. Especially not the latest RC which is outside the regular schedule.

@bt90
Copy link
Contributor

bt90 commented Dec 8, 2023

I can only judge the functionality, which works as intended on Linux and maybe other Unix OS based on my last tests.

@calmh
Copy link
Member

calmh commented Dec 8, 2023

Well the two of you are loudly declaring that the current state is not good for merging, so maybe fix that then:

Screenshot 2023-12-08 at 13 12 27

@acolomb
Copy link
Member

acolomb commented Dec 8, 2023

My point was very clear, it's better to get it out the door and postpone further improvements until later. Sorry it hasn't worked out to do another thorough review in a timely manner. Will do so as time permits. If anything, the last release with nil dereference panics has shown us that one more long hard stare at the code may help to avoid issues early on. And that's what I was planning to do. No investments in the other open comments from @bt90 .

@calmh
Copy link
Member

calmh commented Dec 9, 2023

I'm not sure if we're maybe talking past each other. It sounds to me like you're both saying "this is fine, why is it not merged, calmh?" while what I'm saying is that you've both put stakes in the ground saying "this shall not pass!" by filing "changes required" reviews and then not dismissing them or changing them to "approved". If you, as reviewers, think this should be merged then you need to file approving pull requests, and then merge it. This is how code review works on GitHub. It says so in bold, red, text down below.

@calmh calmh changed the title IPv6 UPnP support lib/nat, lib/upnp: IPv6 UPnP support Dec 9, 2023
@acolomb
Copy link
Member

acolomb commented Dec 9, 2023

I'm saying that we shouldn't add more features before merging. I want to review again (and test, mainly) with the current "final" state, because I think my last tests were with a prior iteration. When that's done, I will mark it appropriately. Not trying to hurry anyone.

Sorry it has taken so long on my side. I basically backed off when seeing there were open questions from @bt90. Then I got busy with the Android Weblate integration instead, but that should be done now so next thing will be getting back to this PR :-)

@bt90
Copy link
Contributor

bt90 commented Dec 9, 2023

I'm not sure if we're maybe talking past each other. It sounds to me like you're both saying "this is fine, why is it not merged, calmh?" while what I'm saying is that you've both put stakes in the ground saying "this shall not pass!" by filing "changes required" reviews and then not dismissing them or changing them to "approved".

Fair point. I hesitated because my approval doesn't carry much weight, or might even give a false sense of security, to be honest. There is a clear difference between a Go developer being able to properly judge code quality and maintainability, and me being happy with the state I have tested.

In other words, I'm happy with the way it works, but I can't guarantee that the code meets the project's quality standards.

In retrospect, my comment was open to misinterpretation. It wasn't my intention to accuse anyone of inaction, but rather to make sure that my inability to do a proper review wouldn't hold up the progress of this PR.

@imsodin
Copy link
Member

imsodin commented Dec 10, 2023

Fair point. I hesitated because my approval doesn't carry much weight, or might even give a false sense of security, to be honest. There is a clear difference between a Go developer being able to properly judge code quality and maintainability, and me being happy with the state I have tested.

You can also dismiss your review - you don't need to approve if you are not comfortable reviewing. However you shouldn't keep a "red"/"changes requested" review in place if you don't want to block a PR from being merged - that's what this kind of review means. Next to your review there are three dots leaving to a menu that gives you the option to dismiss it.

Copy link
Member

@acolomb acolomb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I tested this again with two network links active, wlan0 and eth0. Same scenario as previously documented. The first try only added one IPv4 mapping successfully. Tried again and got a successful mapping for both IPv4 and IPv6, but using different internal ports (I had configured two listeners). The IPv4 mapping disappeared again at some point. But at least, yes I did see a successful IPv6 pinhole.

I'm going to approve this now to at least get this state out to the users. As previously said, UPnP is a best-effort thing, and the PR enabled something that didn't work before.

HOWEVER, I still think it's far less than optimal and I haven't seen good explanations for what happened during the tests. I'm happy to provide logs and PCAP for the two tests for digging into what has failed, in a follow-up discussion. But overall I think whatever we do regarding IGD mappings, it's either too little or too much or buggy in conjunction with my router's firmware, or various combinations thereof. Some of that could be avoided as I outlined earlier, but that means back to the drawing board and shouldn't hold up this first step any longer.

@bt90 bt90 requested review from bt90 and removed request for bt90 December 10, 2023 21:13
@bt90
Copy link
Contributor

bt90 commented Dec 11, 2023

You can also dismiss your review - you don't need to approve if you are not comfortable reviewing.

Unfortunately, the Github app and the web UI on mobile were not up to the task. The last few days have been a bit chaotic for me. But I guess that's pretty much guaranteed when you become a father 😆

@calmh
Copy link
Member

calmh commented Dec 11, 2023

@bt90 Wow, congratulations!

@calmh calmh merged commit 16db6fc into syncthing:main Dec 11, 2023
@calmh calmh added this to the v1.27.2 milestone Dec 11, 2023
@calmh calmh mentioned this pull request Dec 11, 2023
calmh added a commit that referenced this pull request Dec 11, 2023
calmh added a commit to calmh/syncthing that referenced this pull request Dec 11, 2023
* main:
  lib/nat: Fix test build failure (ref syncthing#9010)
calmh added a commit to cjc7373/syncthing that referenced this pull request Dec 11, 2023
* main: (69 commits)
  lib/nat: Fix test build failure (ref syncthing#9010)
  lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274)
  lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274)
  lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271)
  lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271)
  lib/nat, lib/upnp: IPv6 UPnP support (syncthing#9010)
  gui, man, authors: Update docs, translations, and contributors
  gui: Show folder/device status on small screens (syncthing#8643)
  lib/model: Remove runner during folder cleanup (fixes syncthing#9269) (syncthing#9271)
  build: Update dependencies (syncthing#9265)
  build: Revert specifics for Go 1.21.4, build using Go 1.21.5 (syncthing#9264)
  lib/fs: Reduce memory usage in xattrs handling (syncthing#9251)
  lib/model: Improve LastSeen handling (syncthing#9256)
  lib/scanner: Record inode change time for directories and symlinks (syncthing#9250)
  lib/api: Improve ignore loading error handling (fixes syncthing#9253) (syncthing#9254)
  gui, man, authors: Update docs, translations, and contributors
  lib/fs: Better equality comparison in mtimefs
  cmd/stcrashreceiver: Add metrics for diskstore inventory
  cmd/stcrashreceiver: Minor cleanup, stricter file permissions
  cmd/stcrashreceiver: Add metrics for incoming reports
  ...
calmh added a commit to calmh/syncthing that referenced this pull request Dec 16, 2023
* main: (89 commits)
  build: Update quic-go (fixes syncthing#9287)
  lib/model: Only handle relevant folder summaries (kqueue) (fixes syncthing#9183) (syncthing#9288)
  lib/model: Use a single lock (phase two: cleanup) (syncthing#9276)
  build(deps): bump actions/setup-go from 4 to 5 (syncthing#9279)
  lib/model: Use a single lock (syncthing#9275)
  cmd/syncthing: Better cli stdin handling (ref syncthing#9166) (syncthing#9281)
  cmd/syncthing: Mostly replace urfave/cli command line parser with alecthomas/kong (syncthing#9166)
  lib/nat: Fix test build failure (ref syncthing#9010)
  lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274)
  lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274)
  lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271)
  lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271)
  lib/nat, lib/upnp: IPv6 UPnP support (syncthing#9010)
  gui, man, authors: Update docs, translations, and contributors
  gui: Show folder/device status on small screens (syncthing#8643)
  lib/model: Remove runner during folder cleanup (fixes syncthing#9269) (syncthing#9271)
  build: Update dependencies (syncthing#9265)
  build: Revert specifics for Go 1.21.4, build using Go 1.21.5 (syncthing#9264)
  lib/fs: Reduce memory usage in xattrs handling (syncthing#9251)
  lib/model: Improve LastSeen handling (syncthing#9256)
  ...
@acolomb acolomb mentioned this pull request Jan 9, 2024
calmh added a commit to calmh/syncthing that referenced this pull request Jan 14, 2024
* main:
  lib/model: Use a single lock (phase two: cleanup) (syncthing#9276)
  build(deps): bump actions/setup-go from 4 to 5 (syncthing#9279)
  lib/model: Use a single lock (syncthing#9275)
  cmd/syncthing: Better cli stdin handling (ref syncthing#9166) (syncthing#9281)
  cmd/syncthing: Mostly replace urfave/cli command line parser with alecthomas/kong (syncthing#9166)
  lib/nat: Fix test build failure (ref syncthing#9010)
  lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274)
  lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274)
  lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271)
  lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271)
  lib/nat, lib/upnp: IPv6 UPnP support (syncthing#9010)
  gui, man, authors: Update docs, translations, and contributors
  gui: Show folder/device status on small screens (syncthing#8643)
  lib/model: Remove runner during folder cleanup (fixes syncthing#9269) (syncthing#9271)
@st-review st-review added the frozen-due-to-age Issues closed and untouched for a long time, together with being locked for discussion label Apr 9, 2025
@syncthing syncthing locked and limited conversation to collaborators Apr 9, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

frozen-due-to-age Issues closed and untouched for a long time, together with being locked for discussion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants