Attribution data (feature 36/37)

[joostjager]

Failure attribution is important to properly penalize nodes after a payment failure occurs. The goal of the penalty is to give the next attempt a better chance at succeeding. In the happy failure flow, the sender is able to determine the origin of the failure and penalizes a single node or pair of nodes.

Unfortunately it is possible for nodes on the route to hide themselves. If they return random data as the failure message, the sender won't know where the failure happened.

This PR proposes a new failure message format that lets each node commit to the failure message. If one of the nodes corrupts the failure message, the sender will be able to identify that node.

For more information, see https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-October/003723.html and https://delvingbitcoin.org/t/latency-and-privacy-in-lightning/1723.

Furthermore, the htlc fail and fulfill flows are extended to convey self-reported htlc hold times to the sender.

Fail flow implementations

LND implementation: lightningnetwork/lnd#9888

LDK implementation: lightningdevkit/rust-lightning#2256

Eclair implementation: ACINQ/eclair#3065

CLN implementation: ElementsProject/lightning#8291

Fulfill flow implementations

LDK implementation: lightningdevkit/rust-lightning#3801

Eclair implementation: ACINQ/eclair#3100

[thomash-acinq]

I've started implementing it in eclair, do you have some test vectors so we can check that we are compatible?
The design seems good to me, but as I've said previously, I think keeping hop payloads and hmacs for 8 nodes only (instead of 27) is enough for almost all use cases and would give us huge size savings.

[joostjager]

I don't have test vectors yet, but I can produce them. Will add them to this PR when ready.

Capping the max hops at a lower number is fine to me, but do you have a scenario in mind where this would really make the difference? Or is it to more generally that everything above 8 is wasteful?

[joostjager]

@thomash-acinq added a happy fat error test vector.

[joostjager]

I think this big gap in the bits has emerged here because of tentative spec changes that may or may not make it. Not sure why that is necessary. I thought for unofficial extensions, the custom range is supposed to be used?

I can see that with unofficial features deployed in the wild, it is easier to keep the same bit when something becomes official. But not sure if that is worth creating the gap here? An alternative is to deploy unofficial features in the custom range first, and then later recognize both the official and unofficial bit. Slightly more code, but this feature list remains clean.

[joostjager]

Added fat error signaling to the PR.

[thomash-acinq]

I've spent a lot of time trying to make the test vector pass and I've finally found what was wrong:
In the spec you write that the hmac covers

• failure_len, failuremsg, pad_len and pad.

• The first y+1 payloads in payloads. For example, hmac_0_2 would cover all three payloads.

• y downstream hmacs that correspond to downstream node positions relative to x. For example, hmac_0_2 would cover hmac_1_1 and hmac_2_0.

implying that we need to concatenate them in that order. But in your code you follow a different order:

// Include payloads including our own.
_, _ = hash.Write(payloads[:(NumMaxHops-position)*payloadLen])

// Include downstream hmacs.
var hmacsIdx = position + NumMaxHops
for j := 0; j < NumMaxHops-position-1; j++ {
	_, _ = hash.Write(
		hmacs[hmacsIdx*sha256.Size : (hmacsIdx+1)*sha256.Size],
	)

	hmacsIdx += NumMaxHops - j - 1
}

// Include message.
_, _ = hash.Write(message)

I think the order message + hop payloads + hmacs is more intuitive as it matches the order of the fields in the packet.

[joostjager]

Oh great catch! Will produce a new vector.

[joostjager]

@thomash-acinq updated vector

[joostjager]

Updated LND implementation with sender-picked fat error structure parameters: lightningnetwork/lnd#7139

[joostjager]

I did a rough implementation in LDK of hold times for the success case. It turned out to be straight-forward, mostly reusing code from the failure case. Test vectors in #1261

[carlaKC]

Logic all looks good, but would like to restructure the Returning Errors section to follow the requirements / rationale structure that we have elsewhere in the spec - as is the two are interspersed.

Also needs to refer to option_attributable_failures rather than "attributable failures" in a few place.

[carlaKC]

Doesn't need to be in invoices (9)?

[joostjager]

It is indeed the question whether there are scenarios where senders would refuse a payment that pays to a node that doesn't support attribution data? If they'd return random data, and their predecessor adds attribution data, blame still lands on the final node pair.

Maybe attribution data is unnecessary for the final hop? 🤔 Of course they still want to pass back something to not stand out as a recipient.

For hold times, they will probably always report zero anyway.

[carlaKC]

We don't have any instructions in this PR for how to react to the presence / absence of this feature bit in an invoice, so I think we can take it out?

Seems easy enough to come back and add signaling in invoices (+ handling instructions) if we want it in the future?

[carlaKC]

nit: note that nodes on -> note that nodes in?

[joostjager]

Fixed. Language skills insufficient for me to flag 'on the route' as incorrect.

[carlaKC]

I think that we can more formally specify this:

• if option_attributable_failure is advertised:
  • if path_key is not set in the incoming update_add_htlc:
    • MUST include htlc_hold_times in payload.

[joostjager]

Added

[carlaKC]

Duplicated here and in BOLT02 - perhaps just link to the BOLT02 description?

[carlaKC]

General comment on this section: I'm missing the structure of requirements/rationale that we have elsewhere in the specification.

For example, the section for the erring node notes "The sender can use this information to score nodes on latency". To me this should be in the origin node section or all contained in a single rationale at the end. Ditto with the game theory of guessing HMACs.

[joostjager]

I see what you mean. I've added a commit trying to address this and improve it a little, but it remains difficult to capture this in requirements and still have explanation too I found.

[brh28]

I didn't see the start time and end time of the htlc_hold_time defined. I assume these are upon receiving the update_add_htlc and before sending the update_fulfill_htlc/update_fail_htlc, respectively?

Overall, I like the idea of htlc_hold_time but suspect it'll be gamed by routing nodes.

[Roasbeef]

Overall, I like the idea of htlc_hold_time but suspect it'll be gamed by routing nodes.

A motivated sender can always point point what the actual forwarding delays are by bisecting the route. They also always know how long the entire route took as well. You are correct though that it's intended mainly on a best effort basis, the overall assumption is that this can be used to allow a "fast" node to distinguish themselves, and be rewarded for that.

[joostjager]

Overall, I like the idea of htlc_hold_time but suspect it'll be gamed by routing nodes.

As @Roasbeef says, in any case the sender knows how long the entire route took. The total penalty to apply is known. Without additional information, the sender might apply that penalty equally across all nodes.

The only thing the reported hold time does, is shift the penalty to better match reality. If a node reports zero, it doesn't mean that it doesn't get any penalty. It just means that the sender concludes that if that zero is true, that the outgoing side of that node is fast, and all of the to-be-distributed penalty for that node needs to be directed to its incoming side.

There is some gaming potential in there, but I think it is limited and also in a routing node's interest to report realistically.

[Roasbeef]

Ok, looks like this is ready for final review now.