BOLT 1: Clarify optionality of message format `extension`

[tnull]

BOLT 1 currently states:

## Lightning Message Format

After decryption, all Lightning messages are of the form:

1. `type`: a 2-byte big-endian field indicating the type of message
2. `payload`: a variable-length payload that comprises the remainder of
   the message and that conforms to a format matching the `type`
3. `extension`: an optional [TLV stream](#type-length-value-format)

The `type` field indicates how to interpret the `payload` field.
The format for each individual type is defined by a specification in this repository.

as well as

The messages are grouped logically into five groups, ordered by the most significant bit that is set:
...
  - Custom (types `32768`-`65535`): experimental and application-specific messages

A few observations here:

• Stating that payload comprises the remainder of the message directly conflicts with there possibly being an extension in addition to said payload. If there is an extension, payload won't be the entire remainder.
• Stating that the type field indicates how the payload should be interpreted doesn't say anything about whether it also defines how extension should be interpreted.
• It's not entirely clear what 'an optional TLV stream' means. For who is it optional? Does the type define whether a reader should expect/support an optional extension, or is it expected that any application making use of custom messages is able to deal with such an extension being present (according to 'it's okay to be odd')?
• In contrast what's stated at the end of the first excerpt, (custom) message types are not necessarily defined in the BOLTs. At the very least there a those listed in the mentioned issue as well as over at https://github.com/lightning/blips/blob/master/blip-0002.md#messages

It would be great to get some insights into these details, and maybe even clarify them in the spec. In particular, I'd like to understand whether custom messages are expected to span the remainder of the message or whether they are expected to be, e.g., length-prefixed to always be able to handle an extension that could potentially be added by some other protocol.

For context, I'm raising these questions since the LSPS0 protocol as defined in bLIP-50 currently uses custom messages with a type|payload scheme, assuming that payload is always the remainder of the custom message and valid JSON. Is this in compliance with BOLT 1 or would we need to be able to handle the optional TLV extension?

[Roasbeef]

Stating that payload comprises the remainder of the message directly conflicts with there possibly being an extension in addition to said payload

Yeah I think the wording here is off.

This was added during a pass to migrate some of the older ad-hoc message extensions to be pure TLV.

payload is intended to be the base message type. With extension holding any additional TLV extensions. Some newer messages however are nearly entirely TLV messages, so in that case, the payload and extension fields are one and the same.

Stating that the type field indicates how the payload should be interpreted doesn't say anything about whether it also defines how extension should be interpreted.

The type field governs how both the payload and extension are to be interpreted.

It's not entirely clear what 'an optional TLV stream' means. For who is it optional? Does the type define whether a reader should expect/support an optional extension, or is it expected that any application making use of custom messages is able to deal with such an extension being present (according to 'it's okay to be odd')?

It depends on the parity of the TLV field itself. All messages can have extra TLV payloads appended to them. The reader is expected to retain and transmit that information, even if they don't understand it, as the extra payload may be covered by signatures (eg: for channel updates, etc).

IMO for uniformity, any custom messages added as part of bLIPs, should likely just be pure TLV messages.