htlcswitch: add inbound routing fees receive support#6703
htlcswitch: add inbound routing fees receive support#6703Roasbeef merged 4 commits intolightningnetwork:masterfrom
Conversation
|
Added a scenario that underlines the advantage of inbound fees on the spec pr: lightning/bolts#835 (comment) Lightning Labs, what is your view on this feature? |
|
Created BLIP for the gossip change: lightning/blips#18 |
6a14f1e to
776391b
Compare
|
For the The problem though is the size limit of 256 bytes on the failure message. A Maybe a probabilistic fix is to randomly return either the incoming or the outgoing channel update? And perhaps there is an upside to disincentivizing high frequency channel updates too. See also #6883. |
de3de69 to
e0d6f3f
Compare
Crypt-iQ
left a comment
There was a problem hiding this comment.
I think all that's left here is to ensure that an overall negative fee isn't used. I think that can be done with a sanity check in CheckHtlcForward to ensure that we're not losing money. Alternatively, we might be able to do it when updating the policy but not sure how we can guarantee the condition holds for every possible forwarding amount
htlcswitch/link.go
Outdated
There was a problem hiding this comment.
I think you could change this to if incomingHtlcAmt < amtToForward || actualFee < expectedFee as a belt-and-suspenders check to ensure that the node is not losing money due to an overall negative fee.
There was a problem hiding this comment.
Indeed checking it when setting the policy may get a bit complicated. By checking here the node as you say doesn't lose money but will just be penalized by the sender.
But is it really not desired to allow nodes to set a negative (total) fee as a way to incentivize traffic in a certain direction? Currently they may execute rebalance payments to accomplish this goal, which also makes them lose money.
There was a problem hiding this comment.
I can see why a negative total fee is useful, but I'm not sure how it can be added safely. With rebalancing, I'd say that the node operators are more in control of the cost than with negative total fees. When either the inbound base fee or the inbound fee rate is negative, I think the cost becomes less clear to a node operator.
Certain combinations of the 5-tuple (htlc amt, inbound base fee, inbound fee rate, outbound base fee, outbound fee rate) won't have a cost to the operator, but certain combinations will. Node operators could use the HTLC interceptor to halt negative-total-fee traffic after a certain point and have more fine-grained control over any possible losses. However, this seems like a power user feature and I think less savvy users may end up shooting themselves in the foot if they set a negative inbound fee parameter. Setting a negative inbound parameter could come with a warning, but IMO people tend to ignore warnings.
I'm not familiar with the mission control code, but being penalized by the sender definitely doesn't sound good. If we go with the disallow negative total fee approach, what if the path-finding code ensured that the incoming amount was always greater than the outgoing amount?
There was a problem hiding this comment.
I'm not familiar with the mission control code, but being penalized by the sender definitely doesn't sound good.
If the sender receives a fee insufficient failure repeatedly, despite the (negative) fee calculation being correct, the sender will penalize that routing node. I do think that it is reasonable to say that in most cases this is preferable over seeing channel balance drained because of an accidental negative fee setting.
If we go with the disallow negative total fee approach, what if the path-finding code ensured that the incoming amount was always greater than the outgoing amount?
Could do that, but other implementations may differ.
Overall I think that failing forwards with a negative fee is a good idea in combination with documenting that negative fee policies set by the user won't be honored. Can always relax this later on.
There was a problem hiding this comment.
Restored incomingHtlcAmt < amtToForward condition.
There was a problem hiding this comment.
A good idea to not allow negative forward fees for the time being, although I think it's a good idea as well. But I think we need more research on the necessary conditions to the fees on each channel pair, so that a noderunner doesn't lose money in the end.
There was a problem hiding this comment.
The code looks great in the first pass! I think that this proposal can lead to improved routing efficiency and to keep LN more open by providing compensation for liquidity usage. I wanted to do my best to anticipate any worst case scenarios in case of disagreements between peers (an economic question, so hard to answer). I could think of those scenarios:
Scenario 1:
A charges 3000 PPM because the channel is depleted and B is malicious, sets -3000 PPM to generate failures for A->B. A would be blamed in current pathfinding (assume B has opened many channels to strong sinks). B can do this because payments can't settle via the channel and B is not at risk of losing on inbound fees. We would need to account for this scenario in pathfinding, so that's more a penalization issue. [edit: B is limited in its actions because it is required to have an outgoing channel in the forwarding that charges at least 3000 PPM as well, in order to respect a non-negative total node fee, additionally stronger sender-side penalization could be added should such scenarios appear]
Scenario 2:
A (say at 1000 PPM) likes to have some constant flow over a shared channel with B, because A has determined this earns them optimal fees. B would like to generate more traffic from the channel to stimulate traffic in other channels, so B charges -100 PPM. A discovers that traffic is too high now, the channel may deplete soon because there's not enough backward traffic. A raises fees by 100 PPM. B sees this and lowers their inbound fee to -200 PPM and so on. Where would this process stop (note that B subsidizes A here)? Does it become too risky or less economical for B to go much lower? I think that this would be bounded by the highest fees B charges on their node, so it's just a shift of the base fee-PPM from zero PPM (today) to another low at negative values. So the worst case outcome I'd see here would be an increase in gossip, which would be rate limited naturally. This would incur opportunity costs for A and B as this would generate gossip discrepancies in pathfinding, so they would be encouraged to come to an agreement. [edit: another source on the general fee splitting problem can be found here, demonstrating that an equilibrium is possible for rational actors]
Scenario 3:
If B would like to throttle traffic coming from A (say a wallet gateway charging 1000 PPM) to get reimbursed for accumulating balance from a strong source, B can charge 100 PPM inbound fees such that the effective PPM is 1100 PPM. A would lower their fee rate to 900 PPM to keep the status quo and to keep the cost for their users constant and so on. Where would the equilibrium of that process be? I think it would be when A doesn't earn anything net anymore on the channel. I'd say that this will increase the fees for a strong source, but not indefinitely because B also needs to make sure there's enough traffic generated to realize the inbound earnings. I'd argue that in the case of not having inbound fees, B would be inclined to close the channel at some point, not providing any service at all, which leads to a more permissioned system than with inbound fees, a positive argument for inbound fees.
routing/localchans/manager.go
Outdated
There was a problem hiding this comment.
Seems reasonable to not do any checks here given total negative forwardings are disallowed. The check would also depend on the forwarding amount, for some combinations the total fee may be positive, for others negative.
routing/localchans/manager.go
Outdated
There was a problem hiding this comment.
Wouldn't it be possible to charge for routing over private channels already, given the receiving node doesn't charge inbound fees? It would be only problematic on the receiving end for hop hints of length two or more if inbound fees can't be encoded, is it?
I am assuming that B is advertising a discount and then when the htlc arrives, B returns fee insufficient to the sender? In this case, lnd penalizes the A-B link. Can't B already return this error in the current situation too, without a negative inbound fee, when B's balance is zero? |
Interesting, never thought about how insufficient fee errors are penalized, this seems to be similar. I was rather thinking of B attracting traffic to the A-B channel with a discount but A fails back with a temporary channel failure because they can't forward. |
Ah got it, that's a different one. The temp chan fail is still going to hit B too, because it penalizes the A-B link. But yes, it seems the ability for B to set a negative fee allows them to attract more traffic. It is floored at zero though. B can set a enormous negative fee, but it won't make the link A-B more attractive compared to a zero total fee. Implemented here: e142c8d#r1028989648 |
|
Rebased after move to |
lnrpc/lightning.proto
Outdated
There was a problem hiding this comment.
I agree with this, it would be great to have the third amount recorded as well (net incoming amount without the inbound fees) from which we could compute both fees. Otherwise, one would need to keep track of fee rate changes externally from which the data could be computed, but this may be error-prone. Recording the third amount would be important to make accurate fee adjustment decisions. In case of over-payments one could set the third amount in such a way that we distribute the fees between the in and out fees. The change would have to be done via PaymentCircuit I guess?
bitromortac
left a comment
There was a problem hiding this comment.
LGTM 👍 I've manually tested gossip and sending with positive/negative inbound fees as well.
|
Rebased |
|
Important Auto Review SkippedAuto reviews are limited to the following labels: llm-review. Please add one of these labels to enable auto reviews. Please check the settings in the CodeRabbit UI or the To trigger a single review, invoke the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
|
Rebased |
1 similar comment
|
Rebased |
lnwire/typed_fee.go
Outdated
There was a problem hiding this comment.
Non-blocking given the age/lineage of this PR, but I think we should go ahead and add it to the existing ChannelUpdate struct using the new optional TLV records.
There was a problem hiding this comment.
Will make a follow up issue to track this.
In this commit, the tlv extension of a channel update message is parsed. If an inbound fee schedule is encountered, it is reported in the graph rpc calls.
Ensure that negative fees are backwards compatible.
This PR implements "receive" support for inbound fees. In short this means that a routing node operator gets to set distinct fee schedules for the movement of funds on their incoming channels. This enables more fine-grained flow control, potentially leading to an increase in capital efficiency.
More discussion on this change can be found in the corresponding bolts issue lightning/bolts#835 and blip lightning/blips#18
Mailing list post that elaborates a bit on the upgrade scenario and how this can be a non-breaking change: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-July/003643.html
Note that no network-wide upgrade is required for the inbound fee schedule to propagate.
Changes:
For send support, see #6934.
Implementation details
The
channel_updateextra opaque data is stored in the database without any validation. This means that readers of this data must handle the case where a tlv error is encountered during parsing of the stream.An alternative would be to do strict validation in
ExtraOpaqueData.Decodewhen a message is received, but is more involved. It for example breaks the exisitng fuzz tests. Also, there may already be malformed tlv data in the database.When a
fee_insufficientfailure happens on an intermediate node, still only the outgoing channel update is returned. The 256-byte failure message isn't able to accommodate both the incoming and outgoing channel policies until htlcswitch: relax failure message length check #6913 is widely deployed. For now, this means that senders can only receive updates to the inbound policy via gossip.For code that shows a future extended
fee_insufficient, see htlcswitch: return inbound channel update #6967.Forwards will be declined if the total of inbound and outbound fee is negative. Negative inbound fees can be set, but it must be made sure that it is sufficiently offset by a positive outbound fee.
Inbound fees can be set on private channels, but note must be taken that those fees cannot be communicated in bolt11 hop hints. For a hinted route that spans more than a single hop and where any hop except for the final hop has an inbound fee set, the inbound fee will be ignored by the sender. For a negative inbound fee this means the sender will miss out on the discount. For a positive inbound fee, the payment will fail.
Fixes #4225