Add the option on path creator to specify the incoming node or channel on blinded path #9127

MPins · 2024-09-21T04:16:26Z

Change Description

Add the option on path creator to specify the incoming node setting a list of penultimates nodes

Steps to Test

lncli addinvoice --blind --blinded_path_penultimate_node "node_id" amount
lncli addinvoice --blind --blinded_path_penultimate_node "node_id1" --blinded_path_penultimate_node "node_id2" amount

Pull Request Checklist

Testing

Your PR passes all CI checks.
Tests covering the positive and negative (error paths) are included.
Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

The change obeys the Code Documentation and Commenting guidelines, and lines wrap at 80.
Commits follow the Ideal Git Commit Structure.
Any new logging statements use an appropriate subsystem and logging level.
Any new lncli commands have appropriate tags in the comments for the rpc in the proto file.
There is a change description in the release notes, or [skip ci] in the commit message for small changes.

📝 Please see our Contribution Guidelines for further guidance.

MPins · 2024-09-21T04:22:06Z

Hello @ellemouton if you think I'm on the right way I can add the option to choose a channel also.

MPins · 2024-09-22T13:42:18Z

Maybe change the name to "incoming_node" is a better idea! But I'll wait for others opinion also.

ellemouton · 2024-09-23T01:16:34Z

thanks @MPins - in tokyo at the moment for the LN summit so will take a look at this a bit later this week or next week

ziggie1984 · 2024-10-03T20:14:21Z

I think it makes sense to just specify the incoming channel, I don't see a lot of use cases for the whole blinded route, but maybe we already prepare the code so that it will be easy to just upgrade to the whole blinded path if it has benefits.

ziggie1984

Thank you for your first contribution 🎉

Please squash all commits related to the lnrpc change into one.

Moreover while reviewing this PR I found out that we have a bug in your probabiliy calculation. We basically should have the prob. of full certainty for the last hop however we need to change the logic in our probability estimator which still seems to not account for the blinded usecase see here (only when from == Self do we set the full local probability, that needs to now be inverted for the last hop as well):

lnd/routing/missioncontrol.go

Lines 509 to 526 in 2d33317

    
           func (m *MissionControl) GetProbability(fromNode, toNode route.Vertex, 
        
           	amt lnwire.MilliSatoshi, capacity btcutil.Amount) float64 { 
        
           	m.mu.Lock() 
        
           	defer m.mu.Unlock() 
        
           	now := m.cfg.clock.Now() 
        
           	results, _ := m.state.getLastPairResult(fromNode) 
        
           	// Use a distinct probability estimation function for local channels. 
        
           	if fromNode == m.cfg.selfNode { 
        
           		return m.estimator.LocalPairProbability(now, results, toNode) 
        
           	} 
        
           	return m.estimator.PairProbability( 
        
           		now, results, toNode, amt, capacity, 
        
           	) 
        
           }

ziggie1984 · 2024-10-03T19:31:36Z

lnrpc/lightning.proto

@@ -3894,6 +3894,12 @@ message BlindedPathConfig {
    blinded paths.
    */
    repeated bytes node_omission_list = 4;
+


squash into first commit because it belongs to the lnrpc change.

I did just one commit to all lnrpc changes

And another one with lnrpc/lightning.pb.go . As it is a file created with make rpc I'm not sure if I should commit this change.

routing/pathfind.go

cmd/commands/cmd_invoice.go

MPins · 2024-10-04T02:50:19Z

I think it makes sense to just specify the incoming channel, I don't see a lot of use cases for the whole blinded route, but maybe we already prepare the code so that it will be easy to just upgrade to the whole blinded path if it has benefits.

In most cases it would be sufficient to specify the node, the channel must be specified in cases where there is more than one channel with the same node. Of course, specifying the channel covers both use cases. I'm not sure if we should keep both options for the user, perhaps it would be more intuitive to specify the income node for most users.

ziggie1984 · 2024-10-04T09:57:12Z

I prefer the channel, because than you have more control rather than the pubkey. But at the end not sure if its really worth it to treat different channels to the same peer differently because we have non-strict forwarding. So probably both options are ok ...

MPins · 2024-10-07T12:58:32Z

Thank you for your first contribution 🎉

Please squash all commits related to the lnrpc change into one.

Moreover while reviewing this PR I found out that we have a bug in your probabiliy calculation. We basically should have the prob. of full certainty for the last hop however we need to change the logic in our probability estimator which still seems to not account for the blinded usecase see here (only when from == Self do we set the full local probability, that needs to now be inverted for the last hop as well):

lnd/routing/missioncontrol.go

Lines 509 to 526 in 2d33317

func (m *MissionControl) GetProbability(fromNode, toNode route.Vertex,

amt lnwire.MilliSatoshi, capacity btcutil.Amount) float64 {

m.mu.Lock()

defer m.mu.Unlock()

now := m.cfg.clock.Now()

results, _ := m.state.getLastPairResult(fromNode)

// Use a distinct probability estimation function for local channels.

if fromNode == m.cfg.selfNode {

return m.estimator.LocalPairProbability(now, results, toNode)

}

return m.estimator.PairProbability(

now, results, toNode, amt, capacity,

)

}

Thank you for your carefully revision. I'll be working on it some time on the following days. About the probability, I invest some time trying to understanding it, but I didn't get the bug on it! From my understanding, probability is about sending the payment, right? How the payment node would even know that we are choosing the income channel/node? I'm probably missing something, I would appreciate if you give me directions to expand my understanding.

MPins · 2024-10-07T13:37:56Z

I prefer the channel, because than you have more control rather than the pubkey. But at the end not sure if its really worth it to treat different channels to the same peer differently because we have non-strict forwarding. So probably both options are ok ...

I think we can move forward with both options, but in my opinion they should be exclusive. I mean, if the channel is chosen, the node cannot be chosen. (just to avoid redundant info)

ziggie1984 · 2024-10-08T06:29:07Z

So my initial idea was, that we can know the probability of the last channel in a blinded path with very high accuracy because its our own channel and we know the liquidity distribution when creating the blinded path. However looking into the codebase I think we should keep it as is, we do allow MPP payments for blinded paths so the amount can be splitted among the incoming channels in general. Moreover when we specify the incoming channel to receive on (via this PR) we already made the decision that this is the right channel to receive on the full amount, so maybe we can keep the probability estimation of the last hop/channel as is. Given the fact that we include all routes with a minimum route prob. of 1% we are good as is I think.

MPins · 2024-10-08T09:26:11Z

So my initial idea was, that we can know the probability of the last channel in a blinded path with very high accuracy because its our own channel and we know the liquidity distribution when creating the blinded path. However looking into the codebase I think we should keep it as is, we do allow MPP payments for blinded paths so the amount can be splitted among the incoming channels in general. Moreover when we specify the incoming channel to receive on (via this PR) we already made the decision that this is the right channel to receive on the full amount, so maybe we can keep the probability estimation of the last hop/channel as is. Given the fact that we include all routes with a minimum route prob. of 1% we are good as is I think.

Got it ... besides that, as it is blinded I can't think a way of considering that it is our own node without breaking the main reason for using a blinded path, that is not revealing our own node on a invoice.

ziggie1984 · 2024-10-08T09:54:21Z

Got it ... besides that, as it is blinded I can't think a way of considering that it is our own node without breaking the main reason for using a blinded path, that is not revealing our own node on a invoice.

Hmm not sure what you mean, but I was just referring to the creator of the blinded path taking the incoming channel distribution directly into account without really relying on the MC data. The path would still be blinded for the sender so the sender would just have a sorted list of blinded paths in the bolt11 invoice.

MPins · 2024-10-08T10:00:29Z

Got it ... besides that, as it is blinded I can't think a way of considering that it is our own node without breaking the main reason for using a blinded path, that is not revealing our own node on a invoice.

Hmm not sure what you mean, but I was just referring to the creator of the blinded path taking the incoming channel distribution directly into account without really relying on the MC data. The path would still be blinded for the sender so the sender would just have a sorted list of blinded paths in the bolt11 invoice.

Never mind ... I was thinking that the GetProbability func would be called when the payment is being processed by the sender.

MPins · 2024-10-08T12:10:52Z

So my initial idea was, that we can know the probability of the last channel in a blinded path with very high accuracy because its our own channel and we know the liquidity distribution when creating the blinded path. However looking into the codebase I think we should keep it as is, we do allow MPP payments for blinded paths so the amount can be splitted among the incoming channels in general. Moreover when we specify the incoming channel to receive on (via this PR) we already made the decision that this is the right channel to receive on the full amount, so maybe we can keep the probability estimation of the last hop/channel as is. Given the fact that we include all routes with a minimum route prob. of 1% we are good as is I think.

I just saw that it is called by FindBlindedPaths ... Thank you, I have a better understanding of the whole process now.

I think you right ... maybe it should be changed ... not sure if it should be included in this PR or maybe on a specific one for that.

… path

…path

MPins · 2024-10-12T01:13:40Z

go.mod

@@ -35,7 +35,7 @@ require (
 	github.com/lightningnetwork/lightning-onion v1.2.1-0.20240712235311-98bd56499dfb


Those files were changed when I did go get -u github.com/lightningnetwork/lnd/fn to update the Fn to use isEmpty function once #9159 is merged

saubyk · 2024-10-13T17:25:55Z

Have a basic question here.
The problem statement in the linked issue is regarding specifying an incoming channel...

to have better control over payment success probability
to better control the liquidity distribution available on the node

So why are we then giving the user the control of selecting a node in the cli, instead of directly specifying a channel?
I believe the user should only specify the channel not the node (which I find confusing)

MPins · 2024-10-13T19:25:37Z

Have a basic question here. The problem statement in the linked issue is regarding specifying an incoming channel...

to have better control over payment success probability

to better control the liquidity distribution available on the node

So why are we then giving the user the control of selecting a node in the cli, instead of directly specifying a channel? I believe the user should only specify the channel not the node (which I find confusing)

The idea to have the option to specify the node instead of the channel is because in the most cases it would be enough and from the node runner perspective it would be more intuitive specifying just the income node.

ziggie1984 · 2024-10-13T20:26:05Z

So why are we then giving the user the control of selecting a node in the cli, instead of directly specifying a channel?
I believe the user should only specify the channel not the node (which I find confusing)

Good question, I also tend to only support the incoming channel id why:

When creating the blinded path we do not create a Unified Edge meaning that we do NOT take all the different policies for all the channels with the same peer into account. Moreover we cannot be sure that all policies are the same, other implementations iirc can have different policies for different channels, therefore I recommend only using the channel ID here.
Although the lightning network supports non-strict forwarding which means the forwarder can basically select another channel if he wants to, but as mentioned before, we are not unifying the edges to create the worst case policy to the last node. So I think we should stick to the incoming channel id here and should not allow the node id.

ziggie1984 · 2024-10-13T20:47:09Z

Imagine your peer has 2 channels with you and different policies, now when creating the blinded path and using the nodeid, you cannot control the particular channel and the sender will probably use the route with the better constraints. We already have something similar when sending a payment where we can select the outgoing channel. We should prob. stick to this form.

MPins · 2024-10-13T21:14:34Z

I think that

Imagine your peer has 2 channels with you and different policies, now when creating the blinded path and using the nodeid, you cannot control the particular channel and the sender will probably use the route with the better constraints. We already have something similar when sending a payment where we can select the outgoing channel. We should prob. stick to this form.

When sending payment you can also choose the last hop (penultimate node in the path) to route through for the payment, but it is other use case. So I think you both are right, I'm going to take the node option out.

ziggie1984 · 2024-10-13T21:20:27Z

Good observation that we also allow the last hop in the payment flow, but LND does unify the edges in the sending flow and selects the most expensive policy that's why it makes sense to have this last hop setting.

MPins · 2024-10-13T21:34:06Z

Good observation that we also allow the last hop in the payment flow, but LND does unify the edges in the sending flow and selects the most expensive policy that's why it makes sense to have this last hop setting.

Perfect ... I'm going to take the node option out. Thank you @saubyk for starting this exchange of ideas.

saubyk · 2024-10-13T21:37:54Z

Good observation that we also allow the last hop in the payment flow, but LND does unify the edges in the sending flow and selects the most expensive policy that's why it makes sense to have this last hop setting.

Perfect ... I'm going to take the node option out. Thank you @saubyk for starting this exchange of ideas.

Thanks. Not to add the confusion, but I do understand the logic of selecting the node from a UX standpoint and especially when you imagine a user doing it in a UI. Selecting a pub key (with an alias) is much easier than selecting a channel ID. But I think that problem can be addressed at the application level, where the UI can establish that node-channel linkage and present a user friendly way for the user to make a selection.

ellemouton · 2024-10-16T09:28:21Z

I prefer the channel, because than you have more control rather than the pubkey.

The problem is that you dont actually every have this control due to non-strict forwarding. At the end of the day, the peer may choose any channel to fwd on as long as the peer on the other end is the same peer.

Regarding the impl here and if we only want to specify final hop or list of hops, I think things should be made general enough for both:

so if user specifies 1 peer, then LND takes that as a fixed final hop and uses its current logic to build on that given the other config values
use can specify a list of hops and LND will just do the same thing as above.

MPins · 2024-10-16T10:37:31Z

use can specify a list of hops and LND will just do the same thing as above.

The way it is done here the user can specify the last hop many times (peers or channels, not both). If I understand you correctly it should be kept this way. Right?

ziggie1984 · 2024-10-16T11:04:41Z

@ellemouton what's your take on this, we cannot really know whether peers have a universal policy to us?

Imagine your peer has 2 channels with you and different policies, now when creating the blinded path and using the nodeid, you cannot control the particular channel and the sender will probably use the route with the better constraints. We already have something similar when sending a payment where we can select the outgoing channel. We should prob. stick to this form.

ellemouton · 2024-10-16T12:55:52Z

@ziggie1984 - yes that makes sense but my point is that nothing stops the peer from in any case just choosing the channel they want to choose. If you peer has channel X and Y with you, and you tell them to use channel X, nothing stops them from using channel Y

ziggie1984 · 2024-10-16T13:14:45Z

yes that makes sense but my point is that nothing stops the peer from in any case just choosing the channel they want to choose. If you peer has channel X and Y with you, and you tell them to use channel X, nothing stops them from using channel Y

Correct that's for the perspective of the forwarding node, but let's look at it from the receiving node:

You have channel X and Y, both have different Policies, now you create your invoice specifying the last hop via the nodeid, you get 2 blinded paths most likely (or more). The sender will probably select the path with the best constraints, hence the receiver lost a bit of control while specifying the invoice. If he could have specified the channelid he would not put both paths into the invoice.

I agree with you statement when policies are equal the sender can choose both channel X or channel Y and specifying the nodeid here makes sense. However when the policies are different it makes sense for the receiver to have more control specifying the channelid not the nodeid. Thats why I proposed only going with the chanid, which is basically valid for both options, rather than going with only nodeid, or both (which increases complexity).

ziggie1984 · 2024-10-16T13:17:45Z

Do you favour the approach of providing a whole path not just the last channel ? Could not think of a use case where we would like to specify the whole path ?

MPins · 2024-10-16T14:15:30Z

use can specify a list of hops and LND will just do the same thing as above.

Do you mean a list of final hops or all the blinded path? As @ziggie1984 said I can't imagine also a use case for that. 🤔

MPins changed the title ~~Add the option on path creator to specify the incoming node~~ Add the option on path creator to specify the incoming node on blinded path Sep 21, 2024

saubyk assigned MPins Oct 3, 2024

ziggie1984 requested changes Oct 3, 2024

View reviewed changes

MPins added 5 commits October 11, 2024 14:48

lnrpc: add the options to set the income node or channel on a blinded…

7632da5

… path

commands: add option to set the income node or channel on blinded path

8fbb4dd

lnd: add the option to set the income node or channel on blinded path

31f2bac

routing: add the option to set the income node or channel on blinded …

fdff391

…path

routing: add the option to set the income node or channel on blinded …

adc5c80

…path

MPins force-pushed the issue-8993 branch from 65a9c7f to adc5c80 Compare October 12, 2024 00:49

MPins added 2 commits October 11, 2024 21:59

updating go.mod and go.sum (getting the IsEmpty on Fn)

1ab658f

lnrpc/lightning.pb.go

14475d8

MPins commented Oct 12, 2024

View reviewed changes

MPins changed the title ~~Add the option on path creator to specify the incoming node on blinded path~~ Add the option on path creator to specify the incoming node or channel on blinded path Oct 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the option on path creator to specify the incoming node or channel on blinded path #9127

Add the option on path creator to specify the incoming node or channel on blinded path #9127

MPins commented Sep 21, 2024 •

edited

Loading

MPins commented Sep 21, 2024

MPins commented Sep 22, 2024

ellemouton commented Sep 23, 2024

ziggie1984 commented Oct 3, 2024

ziggie1984 left a comment •

edited

Loading

ziggie1984 Oct 3, 2024

MPins Oct 12, 2024

MPins Oct 12, 2024 •

edited

Loading

MPins commented Oct 4, 2024

ziggie1984 commented Oct 4, 2024

MPins commented Oct 7, 2024

MPins commented Oct 7, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

MPins commented Oct 8, 2024 •

edited

Loading

MPins Oct 12, 2024

saubyk commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

saubyk commented Oct 13, 2024 •

edited

Loading

ellemouton commented Oct 16, 2024

MPins commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024 •

edited

Loading

ellemouton commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024

MPins commented Oct 16, 2024

	func (m *MissionControl) GetProbability(fromNode, toNode route.Vertex,
	amt lnwire.MilliSatoshi, capacity btcutil.Amount) float64 {

	m.mu.Lock()
	defer m.mu.Unlock()

	now := m.cfg.clock.Now()
	results, _ := m.state.getLastPairResult(fromNode)

	// Use a distinct probability estimation function for local channels.
	if fromNode == m.cfg.selfNode {
	return m.estimator.LocalPairProbability(now, results, toNode)
	}

	return m.estimator.PairProbability(
	now, results, toNode, amt, capacity,
	)
	}

		@@ -35,7 +35,7 @@ require (
		github.com/lightningnetwork/lightning-onion v1.2.1-0.20240712235311-98bd56499dfb

Add the option on path creator to specify the incoming node or channel on blinded path #9127

Are you sure you want to change the base?

Add the option on path creator to specify the incoming node or channel on blinded path #9127

Conversation

MPins commented Sep 21, 2024 • edited Loading

Change Description

Steps to Test

Pull Request Checklist

Testing

Code Style and Documentation

MPins commented Sep 21, 2024

MPins commented Sep 22, 2024

ellemouton commented Sep 23, 2024

ziggie1984 commented Oct 3, 2024

ziggie1984 left a comment • edited Loading

Choose a reason for hiding this comment

ziggie1984 Oct 3, 2024

Choose a reason for hiding this comment

MPins Oct 12, 2024

Choose a reason for hiding this comment

MPins Oct 12, 2024 • edited Loading

Choose a reason for hiding this comment

MPins commented Oct 4, 2024

ziggie1984 commented Oct 4, 2024

MPins commented Oct 7, 2024

MPins commented Oct 7, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

MPins commented Oct 8, 2024 • edited Loading

MPins Oct 12, 2024

Choose a reason for hiding this comment

saubyk commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

saubyk commented Oct 13, 2024 • edited Loading

ellemouton commented Oct 16, 2024

MPins commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024 • edited Loading

ellemouton commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024

MPins commented Oct 16, 2024

MPins commented Sep 21, 2024 •

edited

Loading

ziggie1984 left a comment •

edited

Loading

MPins Oct 12, 2024 •

edited

Loading

MPins commented Oct 8, 2024 •

edited

Loading

saubyk commented Oct 13, 2024 •

edited

Loading

ziggie1984 commented Oct 16, 2024 •

edited

Loading