add lnurl

[callebtc]

This PR adds the LNURL protocol.

[brianoflondon]

For reasons I've explained in detail elsewhere (see this twitter thread) LNURL is almost uniquely unsuited to this use unless you can demonstrate it's ability to cope with high volumes of small transactions.

It would be far more preferable to show a proof of concept with a system for receiving funds and a working podcast player (or an adaption of one of the open source players) to demonstrate how you can make the complicated, slow, multi step LNURL system suitable for streaming payments.

[dergigi]

Nice! +1 from my side for LNURL support. Lightning addresses are all kinds of awesome, would be great to have support for them (and LNURL in general) in Podcasting 2.0.

One of the main benefits would be more self-custody and more self-sovereign use, since way more people have a lightning address than a keysend-enabled full node. Since we are moving towards custodial hell anyway, with all the inevitable implications of state interference, this is very much needed.

I don't think additional requests are that big of an issue. Some requests can be cached, payments can be batched. The value spec is very clear about this:

The "time interval" for calculating payments is always 1 minute. However, the actual interval between when payments are sent can be longer.

[callebtc]

Thank you for considering this PR.

For reasons I've explained in detail elsewhere (see this twitter thread) LNURL is almost uniquely unsuited to this use unless you can demonstrate it's ability to cope with high volumes of small transactions.

I see no reason why LNURL shouldn't be able to achieve the same throughput as keysend. It's important to realize that the LNURL-pay scheme adds two additional http-only requests per payment, there is no additional burden on the Lightning level. A payment consists of a single Lightning round trip, like with keysend. Running an http endpoint for this does not require any meaningful resources compared to running the Lightning backend. Furthermore, since this is a purely opt-in and open protocol upgrade, you don't need to support LNURL if you can't or don't want.

On client-side optimization: In a private conversation, @kingonly from Breez explained to me how optimization is important for mobile devices since battery / cpu constraints can be very aggressive. I agree with his view. To optimize for this, in principle, the LNURL spec could be extended to indicate in the first request that the endpoint for reaching the second one is permanent. This way, the http-only extra burden would be reduced by half. This is an optional optimization and should not be assumed by the client and should not be a requirement for this PR. Whether this makes sense should be discussed within the LNURL spec conversation.

To summarize: LNURL adds only two (or potentially one) additional http-only requests per payment compared to keysend. These requests do not require any meaningful computation and are solely for the purpose of exchanging payment metadata. In my view, this additional burden will not affect performance and is quasi-negligible compared to what is required to initiate and settle a payment via Lightning.

[brianoflondon]

One of the main benefits would be more self-custody and more self-sovereign use, since way more people have a lightning address than a keysend-enabled full node. Since we are moving towards custodial hell anyway, with all the inevitable implications of state interference, this is very much needed.

I'm a bit surprised at this statement from you Gigi. LN Addresses don't help with custody! Somewhere there is a server running which has to generate a lightning invoice. The Lightning goes to a Node. If the user doesn't have the keys and isn't running that node, it's not self-custody.

[brianoflondon]

To summarize: LNURL adds only two (or potentially one) additional http-only requests per payment compared to keysend. These requests do not require any meaningful computation and are solely for the purpose of exchanging payment metadata. In my view, this additional burden will not affect performance and is quasi-negligible compared to what is required to initiate and settle a payment via Lightning.

Other than making the Value blocks look nicer (and they'd look WAY nicer if we used Hive which has user readable full self-custody payment addresses) what benefit does LNURL bring?

If you don't have your own node, everything in Lightning is custodial. My Hive solution (because the conversion and pay out is near instant) is paradoxically the only self custody solution WITHOUT having your own Lightning Node.

[callebtc]

Other than making the Value blocks look nicer (and they'd look WAY nicer if we used Hive which has user readable full self-custody payment addresses) what benefit does LNURL bring?

If you don't have your own node, everything in Lightning is custodial. My Hive solution (because the conversion and pay out is near instant) is paradoxically the only self custody solution WITHOUT having your own Lightning Node.

I don't know what Hive is and I'm not going to engage in a discussion about the general utility of LNURL. LNURL can be custodial and non-custodial, it doesn't matter for the spec. It has proven itself de facto, insofar that almost all wallets support it since it is undoubtedly the most-used static identifier in Lightning.

If you have technical arguments against it, we can discuss them here. If you don't like it or don't want to support it, for example due to esthetical issues in the value block, feel free to just ignore it.

[brianoflondon]

If you have technical arguments against it, we can discuss them here. If you don't like it or don't want to support it, for example due to esthetical issues in the value block, feel free to just ignore it.

How can LNURL and a Lightning Address be non-custodial (I hate this term I prefer the affirmative Self Custody) without your own Lightning Node. Please explain to me because I'm missing something.

My technical argument is that I built and operate a Lighting Address Gateway for 10's of 1,000's of Hive account owners and quite a few of them are using it on a regular basis.

https://api.v4v.app/docs

This API call returns a QR code which will send me Hive:

https://api.v4v.app/v1/lnurlp/qrcode/brianoflondon?cache_flush=false

any Hive account @v4v.app is a valid Lightning Address because of my work.

So I'm quite familiar with the resources needed to handle these calls and I know for a FACT that if we tried to use Lightning addresses instead of keysend to a node, my system would die.

I dare say I could bring down ANY current Lightning Address implementation with 100 calls per minute to get invoices for 5 sats.

[kingonly]

To summarize: LNURL adds only two (or potentially one) additional http-only requests per payment compared to keysend.

In our private discussion I also said that even one additional http call can impact performance. For sats streaming, there's a one minute timeframe to complete all the payments and there are episodes with 10+ splits (i.e. more than 10 different destinations). So it's not one http call, it could be more than 10 calls in a low bandwidth environment (we serve clients all across the world). If there's a split limit - maybe. But an invoiceless solution is a better tool for this use case.

[kingonly]

Let's decouple Lightning address from lnurlp. The spec can easily support Lightning address if it's extended to support keysend. The v4v has custom key/value for these use cases, but it doesn't require custodian nodes to enable keysend support.

It's true the 1min timeframe is a recommendation, but less batching the better because it ensures podcasters get paid even if the use stops listening. It's less streaming as the timeframe is longer.

[brianoflondon]

Because I'm not a bitcoin maxi and I am intellectually open to all things NOT bitcoin IF they're made by people smarter than me (usually the case) and not owned and operated by VC's and big Silicon Valley deuchebags, I can also spot the problems with Lighting.

Here's a concrete example:

This is my system receiving a keysend payment of 30 sats. Notice how I first get notified of an un-settled invoice at time 273.906826. My logs don't even have the resolution to spot when the notification of payment comes in at 273.907441 which is 0.000615s later which is under a milisecond.

I have to say my opinion of keysend went up dramatically when I actually implemented both the receiving side (shown here) and the sending side (which nobody appears to have ever published a Python implementation of before and which I have made open source). Its only when you dig in to these protocols and actually use them that you fully understand them.

2022-08-22 05:31:34,003 INFO      lnd_scanner      189 : New invoice received: Value: 30 | add_index: 4892 | Settled: False | settle_index: 0 | B4+C1vWq9UDHqdgoe2m0jE90sHkDw5bIX+1zWRANrAk= |  | No broadcast: False | Timer: 273.906826
2022-08-22 05:31:34,003 INFO      lnd_scanner      213 : Invoice has no htlcs: 4892 | Wait for next alert with htlcs: B4+C1vWq9UDHqdgoe2m0jE90sHkDw5bIX+1zWRANrAk= | Timer: 273.907030
2022-08-22 05:31:34,003 INFO      lnd_scanner      189 : New invoice received: Value: 30 | add_index: 4892 | Settled: True | settle_index: 3444 | B4+C1vWq9UDHqdgoe2m0jE90sHkDw5bIX+1zWRANrAk= |  | No broadcast: False | Timer: 273.907441

The song and dance involved for LNURL always takes 500ms or so and more usually a second. I don't have particularly great logs on the system generating the LNURL invoice callbacks to compare but the simple fact is Keysend is actually damn good.

[callebtc]

I dare say I could bring down ANY current Lightning Address implementation with 100 calls per minute to get invoices for 5 sats.

Assuming that's true (which I strongly doubt): That's not because of LNURL but because Lightning is an interactive payment protocol. If you can bring down an LNURL endpoint because of this, you'll be able to do the same thing with keysend.

Obviously, were confusing two very different issues in this thread: Server-side optimization and client-side optimization. I think the former is negligible (as per my previous comments). The latter is an annoyance and I won't pretend that I know how mobile clients behave under these circumstances.

I am almost certain though that the ability of LNURL handling 10x6 payments per minute can be demonstrated and I will prove it.

LNURL isn't the bottleneck, Lightning is.

[cnixbtc]

It seems like it'll be difficult to reach a consensus on how many HTTP requests exactly are too many for a client in bad networking conditions to handle.

As mentioned by others in this thread, while keysend certainly seems to be the more efficient option from this point of view, LNURL-p may make adoption easier on the receiver side. It certainly won't replace keysend, and it shouldn't. But it might still be worthwhile to consider it as an additional way to receive payments. If it can be implemented without a huge toll on performance on the sender side, that is.

In cases where the additional HTTP requests of LNURL-p are too much of a burden for podcast clients, then this boils down to the question of whether it's feasible to batch payments together. As @dergigi mentioned, the spec does allow this:

The "time interval" for calculating payments is always 1 minute. However, the actual interval between when payments are sent can be longer.

The concern from @kingonly is valid, though:

It's true the 1min timeframe is a recommendation, but less batching the better because it ensures podcasters get paid even if the use stops listening. It's less streaming as the timeframe is longer.

Essentially it is now a question of how a client could handle this.

If a receiver supports keysend, payments can be streamed in short intervals just as it is done currently. But if a receiver only supports LNURL-p, then a podcast client would need to be smart about it and, in cases where it's needed (bad network conditions, lots of value splits), resort to batching payments over longer time intervals, or even cache payments for a later time (e.g. for when the network comes back online). Batching payments is certainly not ideal, but maybe it's still worthwhile.

Everything's always a tradeoff. If a receiver wants to receive via LNURL-p then they also need to accept that payments won't always be streamed in such regular intervals as they would be when using keysend. The question is if this tradeoff is worth it. I reckon it might be. Any thoughts?

[kingonly]

Alternatively, custodians can support keysend/AMP...
From a client standpoint, I hope you do understand the complexity of adding support for it.

I guess I don't fully understand

LNURL-p may make adoption easier on the receiver side.

[kiwiidb]

I think adding an extra layer of indirection (like LNURL) would be a good thing.

• Hardcoding the public key in the RSS feed as is done currently is bad practice and is incompatible with a high-availability setup. A service could have multiple receiving nodes, or it could include routehints with private channels in an invoice to be robust against any single node failing.
• I also believe that an extra layer will lead to more self-custodial usage, leveraging private channels and out-of-band communication by LSP's. This could be done by lnurl today but doesn't exist yet afaik.

I also believe that the current focus on being able to do a constant stream of ultra small payments is misguided and clients should batch more aggresively. You can still market it as streaming value, nobody cares. The whole system is based on voluntary donations anyway. Calculate donations once every 10 to 30 minutes, every minute is overkill.

[kingonly]

You don't need to use the pubkey if we add keysend support to Lightning address. I wrote it several times in the thread.
The other remarks aren't really technical, not sure I share the sentiment, definitely not about streaming payments.

p.s. you can use routing hints with keysend or use AMP.
Also, bolt12 will have an invoiceless solution to support offline payments to mobile nodes.

[brianoflondon]

For all the LNURL people here there is a need for one thing you could put in the spec and I see a necessity for it.

I'm not sure how well we'd do at getting it adopted now that Keysend is baked in but problems this week with LNPay have show a need.

We need a VERY SIMPLE spec that just says for a specific lightning address, in the current format which we're all using which unfortunately looks like an email address, to return the correct Keysend routing info.

Example:

lightning:[email protected]

Returns just the json:

            {
                "name": "Brian of London",
                "type": "node",
                "address": "0266ad2656c7a19a219d37e82b280046660f4d7f3ae0c00b64a1629de4ea567668",
                "customKey": "818818",
                "customValue": "brianoflondon"
            },

That would allow for some ability to dynamically move to a new node.

Then in our XML we would have something like:

  <podcast:value type="lightning" method="lnaddress" suggested="0.00000015000">
      <podcast:valueRecipient
          name="Brian of London"
          type="lnaddress"
          address="[email protected]"
          split="100"
      />
  </podcast:value>

This adds one Get request but that can probably happen at the index side. You'd have to write the spec first and then we could see if there was a push to make this happen across the existing app landsacape.

[kingonly]

That's similar to the spec discussed in the lnurl mafia group. You're missing routing hints, but generally it's the same.

[brianoflondon]

As far as I know, nobody has added routing hints to any XML value block but I think they're allowed. I don't think anyone on the Podcasting 2.0 side will do any work on this until you've got a working LNURL spec and then LNURL proponents will need to push it. I'm just suggesting what it would take.

[kingonly]

LNURL ≠ Lightning Address

[brianoflondon]

I've implemented Lightning Addresses so I'm aware that there are a bunch of things lumped under LNURL. It's the only part I can see as having anything useful for us here. The other parts of generating invoices and passing them to podcast players would add so much more complexity. I was trying to make a constructive suggestion of an area the LNURL guys can work on if they want to help.

[bumi]

+1 for an internet identifier / lightning address that returns keysend information.
At Alby we have this enabled for all our lightning addresses. for example [email protected] => https://getalby.com/.well-known/keysend/bumi or here one with custom data: [email protected] => https://getalby.com/.well-known/keysend/michael

For LNURL in podcasting 2.0 we also need to think about boostagrams. Those are essential to the p2.0 ecosystem I think.
Currently afaik LNURL does not provide a way to send additional custom data. We would need an open field that allows clients to send the same JSON data as specified in bLIP 10
(imo LNURL should not know about p2.0 but it needs a way to accept any JSON encoded additional data from the client)

[kingonly]

Why do you need something in lnurl? Send the same TLV as part of the payment.

[bumi]

I guess so. true ideally that is then still the same if possible.

[MerryOscar]

Both Fountain and Alby now have .well-known/keysend/${username} keysend lookups:

https://fountain.fm/.well-known/keysend/merryoscar
https://getalby.com/.well-known/keysend/bumi

Could we proceed by:

1. Agreeing on the above keysend lookup format
2. Have 2 or more apps support the new "type"="lnurl" and "address"="[email protected]" fields in the <podcast:valueRecipient> tag whilst keeping the parent <podcast:value> tag as "method"="keysend"
3. Merge this PR with the small change of keeping the parent <podcast:value> tag as "method"="keysend"
4. Add this keysend lookup to the lightning address spec

I think this is really important so we reduce user friction of adding new splits from:

"give me your value block info so I can add you to the splits"

to:

"give me your lightning address so I can add you to the splits"

The value block is difficult to understand whereas people more intuitively understand lightning addresses. This also solves the issue of having pubkeys in the feeds.

@daveajones @bumi @kiwiidb @callebtc @kingonly what do you think?

I should also note that hosting providers RSS Blue and Sovereign Feeds are already using the above keysend lookups from Fountain and Alby to add splits into feeds.

[brianoflondon]

I support this format already. Any Hive account name will give a valid keysend return from v4v.app.

[kingonly]

@MerryOscar I think first step is to add get the extension PR approved in the lightning address spec (that the way to deal with step 1). Then, decide on migration path. Not sure having 2 apps support it is enough. I think we need to get most of the apps to support it and start documenting it in an table. Ideally it should also be optional tag in valueRecipient till most apps support it. Podcasters are not aware of partial compatibility. I think the best way to it is add it as an optional value, something like:

<podcast:valueRecipient
      name="Carol (Producer)"
      type="node"
      address="02dd306e68c46681aa21d88a436fb35355a8579dd30201581cefa17cb179fc4c15"
      lnaddress="[email protected]"
      split="15"
  />

We don't need to change the value tag imo.
That way clients can start supporting it w/o breaking backward compatibility of other clients. We track client support in a table and once we're happy with the support, we change the optional to "either address or lnaddress are mandatory".

[brianoflondon]

If we add lndaddress to the value block spec (which I agree with as a good idea) I would also like to see a chart with the following info answering the following questions.

Perhaps we also need a spec number for Value Blocks? If we're on 1.0.0 now, will this be 1.1.0 or even 2.0.0 because this WILL break stuff if someone puts out an RSS feed with only an lnaddress and without the matching custom key and value records which all the custodial services (and my non-custodial service) use to route payments. Should we declare Value Block spec in this Value Block?

With a spec number, if a client sees no spec number in an RSS feed it knows what to do. If it sees 2.0.0 it will know to either make a call direct to resolve the lnaddress if it finds one or calls PI API.

For my part I can add lnaddress to all my 3speak feeds in a heartbeat, it's on the sat sending (i.e. listening apps) side that stuff needs to be checked.

All the apps which currently act on Value Block info.
Do they refer to the RSS feed?
Do they refer to the PodcastIndex API?
Which is the ultimate source of truth?

Obviously Dave can make the API work to return the correct values based ONLY on an lnaddress because that's a simple API call ONCE at the center.

But any app which looks at RSS feed is going to have to make that API call for the lnaddress and then cross reference with the PI API?

[brianoflondon]

BTW I have this:

https://v4v.app/.well-known/keysend/brianoflondon

returning my custom key value data for all 3speak shows.

{
  "status": "OK",
  "tag": "keysend",
  "pubkey": "0266ad2656c7a19a219d37e82b280046660f4d7f3ae0c00b64a1629de4ea567668",
  "customData": [
    {
      "customKey": 818818,
      "customValue": "brianoflondon"
    }
  ]
}

[kingonly]

I don't think versioning should be discussed as part of this PR, and generally think it's a overkill. I rather KISS.

[brianoflondon]

I don't think versioning should be discussed as part of this PR, and generally think it's a overkill. I rather KISS.

I'm sure the sleeping yanks will have something to say when they wake up but if we're going to break existing stuff we kind of do need a version system I think.

Every single app currently processing V4V from an RSS feed alone (I don't think anybody does this now but this should be the goal when we can shuffle PI out of the shim position) will need to react differently if there is ONLY and lnaddress in an RSS feed.

[jamescridland]

if there is ONLY and lnaddress in an RSS feed

This spec isn't intended for humans - so a podcast hosting company (aka "Entity Writing The RSS") should be able to run the .well-known lookup and discover the correct values, surely? My lnaddress address, of [email protected], should return the correct info at https://crid.land/.well-known/keysend/james which would then be added to the RSS feed - so I'm unclear of the benefit of supporting [email protected] in the RSS feed.

Perhaps advice for RSS generators might be: "If a user has only supplied their lnaddress address, we recommend periodically retrieving the pubkey and customdata from /.well-known, since they may change over time, and it also verifies that the lnaddress address is accurate. You may wish to check them when generating an updated RSS feed, perhaps when publishing a new show."

Alternatively, an optional lnaddress could be added to the spec, and our advice for apps could be "where a podcast publisher has specified an lnaddress within the podcast:value tag, you should retrieve the pubkey and customdata directly from their ./well-known address, which may be different to the given data in the RSS feed for routing or compatibility reasons with your app.

Keen to avoid potential breakage.

[kingonly]

My point was not to have the versioning discussion in this PR

[kingonly]

I was suggesting lnaddress an an optional parameter at first till we achieve implementation consensus

[brianoflondon]

I was suggesting lnaddress an an optional parameter at first till we achieve implementation consensus

I think we can do this properly one and done.

[MerryOscar]

I think first step is to add get the extension PR approved in the lightning address spec (that the way to deal with step 1). Then, decide on migration path.

@kingonly yep that makes sense

---

Ideally it should also be optional tag in valueRecipient till most apps support it. Podcasters are not aware of partial compatibility.

Alternatively, an optional lnaddress could be added to the spec, and our advice for apps could be "where a podcast publisher has specified an lnaddress within the podcast:value tag, you should retrieve the pubkey and customdata directly from their ./well-known address, which may be different to the given data in the RSS feed for routing or compatibility reasons with your app.

@kingonly @jamescridland ok yes this makes sense. So we add an optional lnaddress and then both hosts and apps can use this at their discretion to make sure the pubkey / customKey / customValue data is correct / up to date.

@daveajones what do you think?

[daveajones]

@daveajones what do you think?

Looking through this now.

[kiwiidb]

I agree on making the smallest change possible, so just adding an optional lnaddress field without changing anything else here seems good to me.
Unrelated, but if there is a LUD PR for keysend I would also like to see route hints supported there if possible.
And versioning seems like a good idea to me but should be it's own PR, yes.

[bumi]

To me the current spec is very generic and universal. Very well designed. We have the universal type and address attribute that is very flexible for future developments like this lnurl option. Adding an additional attribute which is special for one type/option feels a bit wrong to me.
If a hoster allows entering the details through the lightning address then they still need to do the lookup to be backwards compatible and if a client supports the ln address it could also load it from the type=lnurl.

The ln address would also only work if it supports the keysend option. If this validation does not happen on the hoster side listeners will be frustrated when it does not work.
I don't see the a path to adoption through this attribute.

I like the recommendation to periodically refresh the pubkey and customdata from /.well-known and update it in the RSS feed. And developers should be aware that the data might need to be changed.

The type option seems favorable to me and then we could promote this option to make developers of players/hosters aware of this. If it is helpful and advantageous it will find adoption.

[kingonly]

We need a migration path in any case. Since I don't control the hosters, I rather do something optional in the clients.

Also the period refresh seems complicated and isn't robust.

And a nit to the current spec: lnaddress isn't lnurl.

[bumi]

And a nit to the current spec: lnaddress isn't lnurl.
👍

is the goal of the migration to fully support LNURL-pay? or we want to get the keysend data out of the RSS feed and behind a HTTP request?

[kingonly]

The latter

[bumi]

how do clients currently deal with multiple value blocks? I think this came up somewhere else, too.
If it is possible to add multiple tags then we can keep the format and simple have both options for the client to decide.

[kingonly]

lnaddress isn't another form of payment in my mind. I think the whole idea here is to migrate from keysend to lnaddress, so I'm just offering a migration path. We can add another optional value block, but I don't see the point if we want to deprecate keysend. When bolt12 comes, we'll add another block? It's super complicated imo. It's better to standardize on one Lightning value block.
Introducing another value block w/o a proper migration process (i.e. the lnaddress value block can exist w/o the keysend block) would be bad, and if that's the case, I think an optional lnaddress attribute is both simpler and clearer.