BOLT12

[era-coder]

Please add BOLT12 to LND
he BOLT 11 invoice format has proven popular, but has several limitations:

The entangling of bech32 encoding makes it awkward to send in other forms (i.e. inside the lightning network itself).
The signature applying to the entire invoice makes it impossible to prove an invoice without revealing its entirety.
Fields cannot generally be extracted for external use: the h field was a boutique extraction of the d field, only.
The lack of 'it's OK to be odd' rule makes backwards compatibility harder.
The 'human-readable' idea of separating amounts proved fraught: p was often mishandled, and amounts in pico-bitcoin are harder than the modern satoshi-based counting.
The bech32 encoding was found to have an issue with extensions, which means we want to replace or discard it anyway.
The payment_secret designed to prevent probing by other nodes in the path was only useful if the invoice remained private between the payer and payee.
Invoices must be given per-user, and are actively dangerous if two payment attempts are made for the same user.

please Read BOLT12.org for details

[Fittiboy]

Big fan of bolt12 offers. I've been working with lnurl for a while now, and I love being able to present a static QR code that handles multiple payments or withdrawals. Lnurl-pay also allowed for lightning addresses to happen via this shockingly simple spec.

The most common gripe people have with lnurl is the fact that an http server is required, and either a domain, or an extra tor hidden service.

There are workarounds for lightning addresses, like satdress or bridgeaddr, which allow users to get a lightning address that points to their own node (or LNbits instance), without requiring their own http server, or even their own domain (in the case of satdress).
But even this still requires that users provide their node's IP address, and an invoice macaroon.

With the introduction of bolt12, users can simply provide an offer that is served at their lightning address. This is a significant improvement in privacy, especially as bolt12 supports blinded paths.
Furthermore, bolt12 offers allow users to gain access to payment flows equivalent to lnurl-pay (without a lightning address) and lnurl-withdraw, without requiring any third party to host an http server, or provide a domain.

Bolt12 also has other advantages like enabling recurring payments, and, importantly, extensibility.

[drvnoo]

BOLT12 would be epic!

[JimmyMow]

Is this the correct place to track LND's progress in supporting BOLT12?

We're beginning to look into the best way to support BOLT12 for various efforts at Strike

[Roasbeef]

Is this the correct place to track LND's progress in supporting BOLT12?

Yeah you can view this as a tracking/brain storming issue. Also important to keep in mind that BOLT 12 is still evolving as the design meets real world design.

lnd development priorities

Our current set of priorities in terms of lnd development (roughly in order) are: improving the stability+security+scalability of lnd (should just run w/ minimal baby sitting, easy to run in a secure config, and be able to scale to meet the new levels of adoptions we're seeing w/ LN around the world), integrating taproot initially from an on-chain perspective -- ultimately leading to the first set of taproot native channels as they really expand the types of things we can do off-chain with channels, and path finding/routing improvements (better algos, trampoline, adding a payment level ACK, packet switching stuff, etc).

Most of our development resources currently allocated to the above items, along with more ad-hoc pushes to fix serious/critical bugs. Given the large design space of both LN the protocol, as well as LN implementations, each team needs to make tradeoffs to work particular items, as given the amount of innovation/ideas/proposals, it's difficult for a team to implement+review+develop everything while also making sure an adequate amount of maintenance+stability is in place. It's also somewhat unavoidable that all implementations won't roll out new feature in lock step (as an example only this year did another node join lnd in using anchor channels by default). Even today there're several protocol extensions that only a single implementation has fully implemented + rolled out, and in many cases due to the extensibility of the protocol, they're able to use those new features without requiring the entire network to adopt them.

In the past we admittedly moved a bit too quickly on new features (both at the protocol and RPC/API level), which resulted in existing bugs and stability improvements not being properly prioritized. My aim is to start to reverse that trend this year, slowing down on new features and instead ensuring that lnd the software is ready to shine in primetime and won't buckle under the load and is easy to run securely.

bolt 12's surface area

Given the above, on the topic of BOLT 12, it admittedly isn't a very high priority for the project atm. What people colloquially refer to as BOLT 12 is actually the composition of several (possibly independent protocol features) including:

• a new invoice format
• a new blinded path scheme
• onion messaging
• an invoice negotiation protocol
• a recurring payments scheme
• a subscription scheme
• a new custom merkle tree/proof format
• protocol-level fiat pricing negotiation

and much more. In short, there's a lot packed in there and at this point, no one implementation has actually implemented it all to my knowledge.

At the current cross roads, if we had to pick between BOLT 12 and taproot (alongside making lnd more stable/secure/scalable), we'd pick taproot, as partial solutions to everything BOLT 12 is trying to tackle already exist, but taproot allows entirely new interactions/protocols off-chain. I think that given BOLT 12 encompasses so many layers of the protocol stack (from internal routing messages all the way up to wallet/service payment negotiation), at times conversations about the proposal end up missing a lot of nuance w.r.t what it "competes" with in the wild. For example, it's entirely possible for LN-URL to actually serve BOLT 12 invoices in the confines of the existing protocol. A static AMP invoice can even be served over the BOLT 12 invoice negotiation protocol.

decomposing bolt 12 + iterative deployment

I think in order for the proposal to move forward, the community and implementations should adopt of piece-wise approach. The entire proposal all together is a massive under taking, but it can be broken up into initial phases like just adding that new invoice format (we get pure TLV, more extensible, no weird bech32 stuff -- all great stuff). Even just that may take some time as BOLT 11 is the current default across all wallets/services. Should wallets be ready to scan/display both invoices? Just switch to the new format? Convert between BOLT 11 and 12? I'm not sure what the best path forward is there. I also wouldn't underestimate the timeline to fully convert to the new format. With segwit it took years for services to update, and even now w/ taproot, even though the "new" address was just a constant change (bech32m), some services still aren't able to send to taproot addresses.

onion messaging concerns

The part of BOLT 12 that gives me personally the greatest pause is the onion messaging component. In theory, it sounds dope: any node can just send any other node a message tunneled over LN using our existing onion encryption stuff. In the past I even advocated for similar schemes, like streaming video of the network using HORNET. Several years later, I'm a bit more apprehensive to introducing generalized messaging as, then we'd need to still ensure LN is an acceptable payment network and an arbitrary messaging protocol. Can LN be LN, and also a better version of Tor? It seems like a lot to tackle at once, and getting DoS and rate limiting right isn't going to be a walk in the park.

Today with onion messaging the idea is you just sling out the message (maybe across concurrent paths) to any other node on the network. The lack of session creation means that any node can spam any other node, as you don't need to "accept" the circuit creation. As the message is meant to be forwarded on a best effort basis, it can be dropped or even prematurely rate limited across the entire network, meaning even just something like fetching an invoice can fail. Compare this to the situation today, where a user can reliably obtain an invoice and execute a payment immediately. If the BOLT 12 we talk about today just uses the inner invoice format as the default (fetching isn't always the default) we can achieve a similar flow and take advantage of the new bits of the invoice.

adding metered payments and explicit circuit creation to onion messaging

I've been working in the background on a payment metered way to allow explicit session creation, and also allow nodes to natively charge for any onion messages forwarded. The end protocol resembles HORNET, but fits pretty nicely into our existing protocol. IMO if given the choice to forward arbitrary messages in the network for free vs be paid for that service, I think most node operators would choose the latter. Given that we still dealing with all the various griefing vectors today in the protocol as a result of the initial design, we should be careful to roll our new similarly exploitable extensions without thoroughly considering how to enforce rate limiting spam, and incentive alignment in the first version.

does onion messaging actually enable direct p2p mobile payments?

One last thing I think is lost in the current discourse is that the onion messaging component of BOLT 12 is meant primarily to allow direct p2p payments. In theory, if your phone can just fetch an invoice directly from my phone (or w/e) then that's ideal. Ignoring the fact that you can just fetch the invoice if you're scanning the invoice from me directly (or another site), the missing link is a way to accept payments while remaining offline. If my phone goes to sleep right when you send, the payment will fail. What's needed is a way to wake up my phone to have it both respond to the invoice fetch request and also sign the updates needed to receive funds. If you lean towards things like Firebase or apple push notifications, well then you're relying on "servers" once again. From a protocol design perspective, imo it's pragmatic to lean on higher level infrastructure when needed vs trying to do everything within our desired medium. If node implementations all packaged tor onion services, then maybe we would just use that instead and be able to ignore all the implications of making our own generalized messaging system?

[drvnoo]

Relying on Google/Apple to wakeup the app is not optimal, but it's the only solution we got. The data sent in the notification will ofc be encrypted and you can further improve the privacy by mixing in random fake notifications.

It might also be possible to encrypt a timer in the notification for when the real payment is expected and have the wallet wake up itself.

It's not optimal, but users will be able to receive offers in sleep mode while Google only sees encrypted uncorrelateable crap.

[kingonly]

Relying on FCM to wakeup the app is unreliable. Behavior is unpredictable and dependent on the specific os. Also it doesn't provide a solution to other use cases (e.g. browser nodes). Offline receiving is one the problems we must tackle within Lightning if we want it to scale.

[saubyk]

@Roasbeef thanks for sharing your perspective on BOLT12, which I think was sorely needed. Couple of points to consider:

1. Agree that BOLT12's scope is significant, but implementations can break it down and develop it gradually, you've remarked that as well. E.g. only supporting onion messaging is a good start, as it helps other implementations, without adding significant dev load on your team.
2. Lack of good solution for mobile offline payments, shouldn't mean that BOLT12 implementation should be held off for the nodes as well. As it's a significant improvement of the payment functionality aiding tipping and donations campaigns run with nodes.

[Ziya-Sadr]

but it can be broken up into initial phases like just adding that new invoice format

wouldn't such an approach (breaking it into initial phases) mess up compatibility and confuse users and their experience, even more? (I mean in case BOLT 12 gets implemented by wallets and other LN implementations in the future)

[joostjager]

I think it should be possible to implement bolt12 externally? Use custom peer messages to handle the bolt12 traffic in a separate process.

Maybe the onion message routing is more complicated, but just implementing the end point behavior looks to be feasible.

[Roasbeef]

I've been working in the background on a payment metered way to allow explicit session creation, and also allow nodes to natively charge for any onion messages forwarded. The end protocol resembles HORNET, but fits pretty nicely into our existing protocol.

Hi y'all, I just posted a draft proposal to the ML detailing this idea: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-February/003498.html. If everything holds up, then this can add a way to allow nodes to be paid for forwarding onion messages, and also let senders accept/reject them.

@saubyk

Agree that BOLT12's scope is significant, but implementations can break it down and develop it gradually, you've remarked that as well

Yep that's what I suggest in my initial comment.

Lack of good solution for mobile offline payments, shouldn't mean that BOLT12 implementation should be held off for the nodes as well

Totally, I didn't entail that it should be. Instead I was trying to draw attention to the fact most proponents of onion message really like it for the mobile p2p use case, but it alone doesn't fully support that use case in isolation.

@Ziya-Sadr

wouldn't such an approach (breaking it into initial phases) mess up compatibility and confuse users and their experience, even more?

Such initial fragmentation is unavoidable IMO. Even if all the major implementations release new versions that support the bolt 12 invoice format tomorrow, there'll still be older nodes that are unable to scan/parse/pay those invoices.

[carlaKC]

Following up on some conversations in Oakland, I'd like to outline a proposal for adding the ability to pay BOLT12 offers in lnd. While this is only half of the work, my hope is that adding the ability to pay offers in lnd will get us on the road towards deprecating BOLT11, and a common invoice format.

I'd like to implement this in an experimental offers subserver, so that the code is (so far as possible) isolated to a single location. I'm aware that this is a non-trivial review and maintenance cost for the team, so would also be open to initially implementing this externally to lnd if preferred, and limiting the change set to those required to support offers using only the APIs (eg, a carve-out to allow custommessage to relay onion messages) if there would be more enthusiasm for those changes? Would like to get some direction on this so that I don't plunder on in the wrong direction 🙏

Order of Operations

1. Support forwarding blinded routes
2. Add ability to pay bolt 12 invoices
3. Add ability to decode bolt 12 offers
4. Direct Connection Offers
5. Add support for paying blinded routes

Blinded Route Forwarding

Adding support for forwarding htlcs in blinded routes allows lnd nodes to support network payments to BOLT12 invoices with blinded routes. This is good for reliability, since having more nodes that support blinded routes increases your selection options.

Adding support here is also fairly lightweight, because all we need to do is grab an ephemeral key out of update_add_htlc and decode the encoded_data provided in the onion and use the fees/timelock suggested. This change would need to be internal to lnd, because we don't have incoming link htlc interception (and I don't think it would be valuable to add).

Pay BOLT12 Invoices

Add understanding of the new invoice structure, initially without support for blinded routes. There is quite a lot of functionality embedded in BOLT12 invoices, so I'll start with a minimal subset to keep the change small (excluding fields like currency, for example). While the typical offer user wouldn't directly interact with BOLT12 invoices directly (they scan an offer and the invoice is retrieved), this is a useful building block for breaking the change down into parts.

Add BOLT12 Offer Decoding

Add the ability to decode offers, implementing a minimal subset of TLVs required for basic invoice_request functionality.

Direct Connection Offers

When presented with an offer, add the ability to make an ad-hoc connection to retrieve an invoice and pay it. Once we have this functionality, we can always switch out the "direct connect" part with full onion messages in the background without any changes to the API.

1. Directly connect to peer: Make an ad-hoc connection to the peer to relay single-hop onion messages.
2. Create invoice request, handle response: Create an onion message with an invoice_request and handle the invoice_response.
3. Pay BOLT 12 invoice: Pay the invoice returned, using the funcionality above.

Add support for blinded routes

Once offers are working, we can come back and add support for blinded routes. This change will require the following:

1. Update BOLT12 supported TLVs: add blinded routes to the set of known even TLVs in our invoices.
2. Update pathfinding to support blinded routes: pathfinding needs to be able to start with the aggregate CLTV and fees indicated by the blinded route.
3. Update route construction to include blobs: payments to blinded routes need to include encrypted data and ephemeral keys for the blinded route.

Out of scope (for now)

I've omitted the ability to create/receive BOLT12 offers/invoices because the above is a large enough bite of the elephant already, and I'm mindful that LND has ongoing work in the invoice registry (#6288 / #6176, afaik) which needs to be completed before we make any more large changes there. (Will help with review where I can!)

My hope is that sender-support will provide wallets/applications with the opportunity to demonstrate their commitment to supporting offers through the "proof of work" of adding it to their application, and we can confidently proceed with the large changes required to add receiver-side functionality.