-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
lib/nat, lib/upnp: IPv6 UPnP support #9010
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
acolomb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't tested it yet but looking forward to doing so.
Some minor questions below just from looking at the code.
imsodin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this!
I have some general comments on the code - I didn't check/test the actual upnp procedures.
Co-authored-by: Simon Frei <freisim93@gmail.com>
|
Does this also work if temporary addresses are used? The address being published via global discovery would need to match the IP we're requesting the pinhole for. see #8453 for more context. |
This seems to work. In my testing setup, it opens a pinhole to the SLAAC-assigned IPv6 which seems to be the one advertised to the discovery service |
|
@maxi0604 I just tested this on my home LAN, with an IPv6 and CGNAT IPv4 connection to the ISP. It seems to work partially, but I couldn't really make sense of the debugging log messages. Would you care to look at the debug logs? How can I send them to you privately? |
@acolomb I just noticed that my e-mail wasn't visible on github, you should be able to mail them to me now |
|
Sent the logs and tried once more with explicit listen addresses set for |
|
Now it does open a port, but the router says it's for IPv4 (TCP/22123). Still wondering about those |
|
I haven't changed anything substantial for NAT-PMP so that error message is weird but likely not related. With UPnP I think the code has to become a little bit more complicated. The assumption I made was that I could just have the code create a NAT device for IPv4 which contacts the gateway via IPv4 and one for IPv6 which contacts the gateway via IPv6. Then I could just use the local IP used to connect to the gateway for the request, like in the IPv4 case. There's already a comment in the code about how that is a roundabout approach for IPv4 but in the log you sent it breaks completely for IPv6. I am thinking that the reason is that your router advertises the IPv6 UPnP service under a link-local address which a) we can't connect to without a zone index b) even if we could just connect, as mentioned before, the IP guessing mechanism breaks here since connecting to a link-local local address will probably select the link-local address of the interface the syncthing instance is running on, which we can't open pinhole to by definition. I'm thinking the solution looks something like this:
|
|
Fine with me to get this finally completed, without further digging into the deduplication stuff or the Windows failure for now. Of course, a follow-up will be very welcome and I do want to help there with further testing and discussion. But let's get this out to users in a basic form first. Hope I can find some time to test and review again soon. We just missed the -rc.1 release so I guess there's time until the next cycle. I'll have to think about your comments on the proposed logic, it's been too long since. Just one thought: The fact that the actual listeners are not considered when starting discovery and contacting mapping services, is something we should fundamentally reconsider. I think that will solve our current problems with duplicate mappings much easier. |
I think it might be clever to only check for GOOS after the call has failed and provide a more useful error message. That way, no one needs to remember to change it once the issue in Go is fixed. |
|
@calmh Where do we go from here? We've missed the release candidate again, and I'm not sure the code has had enough review to be merged for RC2. |
|
I see a hilarious amount of discussion, no approvals (the opposite, two open "changes requested" reviews), and the last comments hint towards outstanding issues ("should we add" ... "I think ..."). I also actively avoid getting involved in PR review when there's already multiple other people reviewing it, so as to avoid pile-on. RC timing has nothing to do with merging PRs. Especially not the latest RC which is outside the regular schedule. |
|
I can only judge the functionality, which works as intended on Linux and maybe other Unix OS based on my last tests. |
|
My point was very clear, it's better to get it out the door and postpone further improvements until later. Sorry it hasn't worked out to do another thorough review in a timely manner. Will do so as time permits. If anything, the last release with nil dereference panics has shown us that one more long hard stare at the code may help to avoid issues early on. And that's what I was planning to do. No investments in the other open comments from @bt90 . |
|
I'm not sure if we're maybe talking past each other. It sounds to me like you're both saying "this is fine, why is it not merged, calmh?" while what I'm saying is that you've both put stakes in the ground saying "this shall not pass!" by filing "changes required" reviews and then not dismissing them or changing them to "approved". If you, as reviewers, think this should be merged then you need to file approving pull requests, and then merge it. This is how code review works on GitHub. It says so in bold, red, text down below. |
|
I'm saying that we shouldn't add more features before merging. I want to review again (and test, mainly) with the current "final" state, because I think my last tests were with a prior iteration. When that's done, I will mark it appropriately. Not trying to hurry anyone. Sorry it has taken so long on my side. I basically backed off when seeing there were open questions from @bt90. Then I got busy with the Android Weblate integration instead, but that should be done now so next thing will be getting back to this PR :-) |
Fair point. I hesitated because my approval doesn't carry much weight, or might even give a false sense of security, to be honest. There is a clear difference between a Go developer being able to properly judge code quality and maintainability, and me being happy with the state I have tested. In other words, I'm happy with the way it works, but I can't guarantee that the code meets the project's quality standards. In retrospect, my comment was open to misinterpretation. It wasn't my intention to accuse anyone of inaction, but rather to make sure that my inability to do a proper review wouldn't hold up the progress of this PR. |
You can also dismiss your review - you don't need to approve if you are not comfortable reviewing. However you shouldn't keep a "red"/"changes requested" review in place if you don't want to block a PR from being merged - that's what this kind of review means. Next to your review there are three dots leaving to a menu that gives you the option to dismiss it. |
acolomb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I tested this again with two network links active, wlan0 and eth0. Same scenario as previously documented. The first try only added one IPv4 mapping successfully. Tried again and got a successful mapping for both IPv4 and IPv6, but using different internal ports (I had configured two listeners). The IPv4 mapping disappeared again at some point. But at least, yes I did see a successful IPv6 pinhole.
I'm going to approve this now to at least get this state out to the users. As previously said, UPnP is a best-effort thing, and the PR enabled something that didn't work before.
HOWEVER, I still think it's far less than optimal and I haven't seen good explanations for what happened during the tests. I'm happy to provide logs and PCAP for the two tests for digging into what has failed, in a follow-up discussion. But overall I think whatever we do regarding IGD mappings, it's either too little or too much or buggy in conjunction with my router's firmware, or various combinations thereof. Some of that could be avoided as I outlined earlier, but that means back to the drawing board and shouldn't hold up this first step any longer.
Unfortunately, the Github app and the web UI on mobile were not up to the task. The last few days have been a bit chaotic for me. But I guess that's pretty much guaranteed when you become a father 😆 |
|
@bt90 Wow, congratulations! |
* main: lib/nat: Fix test build failure (ref syncthing#9010)
* main: (69 commits) lib/nat: Fix test build failure (ref syncthing#9010) lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274) lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274) lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271) lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271) lib/nat, lib/upnp: IPv6 UPnP support (syncthing#9010) gui, man, authors: Update docs, translations, and contributors gui: Show folder/device status on small screens (syncthing#8643) lib/model: Remove runner during folder cleanup (fixes syncthing#9269) (syncthing#9271) build: Update dependencies (syncthing#9265) build: Revert specifics for Go 1.21.4, build using Go 1.21.5 (syncthing#9264) lib/fs: Reduce memory usage in xattrs handling (syncthing#9251) lib/model: Improve LastSeen handling (syncthing#9256) lib/scanner: Record inode change time for directories and symlinks (syncthing#9250) lib/api: Improve ignore loading error handling (fixes syncthing#9253) (syncthing#9254) gui, man, authors: Update docs, translations, and contributors lib/fs: Better equality comparison in mtimefs cmd/stcrashreceiver: Add metrics for diskstore inventory cmd/stcrashreceiver: Minor cleanup, stricter file permissions cmd/stcrashreceiver: Add metrics for incoming reports ...
* main: (89 commits) build: Update quic-go (fixes syncthing#9287) lib/model: Only handle relevant folder summaries (kqueue) (fixes syncthing#9183) (syncthing#9288) lib/model: Use a single lock (phase two: cleanup) (syncthing#9276) build(deps): bump actions/setup-go from 4 to 5 (syncthing#9279) lib/model: Use a single lock (syncthing#9275) cmd/syncthing: Better cli stdin handling (ref syncthing#9166) (syncthing#9281) cmd/syncthing: Mostly replace urfave/cli command line parser with alecthomas/kong (syncthing#9166) lib/nat: Fix test build failure (ref syncthing#9010) lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274) lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274) lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271) lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271) lib/nat, lib/upnp: IPv6 UPnP support (syncthing#9010) gui, man, authors: Update docs, translations, and contributors gui: Show folder/device status on small screens (syncthing#8643) lib/model: Remove runner during folder cleanup (fixes syncthing#9269) (syncthing#9271) build: Update dependencies (syncthing#9265) build: Revert specifics for Go 1.21.4, build using Go 1.21.5 (syncthing#9264) lib/fs: Reduce memory usage in xattrs handling (syncthing#9251) lib/model: Improve LastSeen handling (syncthing#9256) ...
* main: lib/model: Use a single lock (phase two: cleanup) (syncthing#9276) build(deps): bump actions/setup-go from 4 to 5 (syncthing#9279) lib/model: Use a single lock (syncthing#9275) cmd/syncthing: Better cli stdin handling (ref syncthing#9166) (syncthing#9281) cmd/syncthing: Mostly replace urfave/cli command line parser with alecthomas/kong (syncthing#9166) lib/nat: Fix test build failure (ref syncthing#9010) lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274) lib/model: Add pmut locking for DeviceStatistics (fixes syncthing#9274) lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271) lib/model: Remove spurious "replacing service" failure event (ref syncthing#9271) lib/nat, lib/upnp: IPv6 UPnP support (syncthing#9010) gui, man, authors: Update docs, translations, and contributors gui: Show folder/device status on small screens (syncthing#8643) lib/model: Remove runner during folder cleanup (fixes syncthing#9269) (syncthing#9271)

Purpose
This pull request allows syncthing to request an IPv6 pinhole, addressing issue #7406. This helps users who prefer to use IPv6 for hosting their services or are forced to do so because of CGNAT. Otherwise, such users would have to configure their firewall manually to allow syncthing traffic to pass through while IPv4 users can use UPnP to take care of network configuration already.
Testing
I have tested this in a virtual machine setup with miniupnpd running on the virtualized router. It successfully added an IPv6 pinhole when used with IPv6 only, an IPv4 port mapping when used with IPv4 only and both when dual-stack (IPv4 and IPv6) is used.
Automated tests could be added for SOAP responses from the router but automatically testing this with a real network is likely infeasible.
Documentation
https://docs.syncthing.net/users/firewall.html could be updated to mention the fact that UPnP now works with IPv6, although this change is more "behind the scenes".
Authorship
Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.