From c6521e4105968c4de64611b779eeeb78f1ee2b2a Mon Sep 17 00:00:00 2001 From: seedhammer Date: Sun, 17 Dec 2023 11:38:56 -0600 Subject: [PATCH 1/7] add bip-psbt-descriptors --- bip-psbt-descriptors.mediawiki | 129 +++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 bip-psbt-descriptors.mediawiki diff --git a/bip-psbt-descriptors.mediawiki b/bip-psbt-descriptors.mediawiki new file mode 100644 index 0000000000..74a1307e15 --- /dev/null +++ b/bip-psbt-descriptors.mediawiki @@ -0,0 +1,129 @@ +
+  BIP: ?
+  Layer: Applications
+  Title: PSBT Encoded Output Descriptors
+  Author: SeedHammer 
+  License: BSD-2-Clause
+
+ +==Introduction== + +===Abstract=== + +A BIP 174 PSBT may contain an extended key for deriving input and output +addresses. This document proposes an additional field for PSBTs to represent +arbitrary BIP 380 output script descriptors. + +To support transfer of output descriptors outside signing flows, the proposal +makes the unsigned transaction optional. + +===Copyright=== + +This BIP is licensed under the BSD 2-clause license. + +===Motivation=== + +A BIP 380 output descriptor is a textual representation of a set of output +scripts, such that Bitcoin wallets may agree on the scripts and addresses +to provide the user. However, a descriptor string by itself is not ideal for +transferring between wallets: it lacks a machine recognizable header and cannot +represent metadata such as name and birth block. + +The PSBT encoding gives us a self-describing file format, metadata as well as a +compact, binary representation of keys. Assuming most wallets already implements +the PSBT format, the additional implementation overhead of this extension is +expected to be low. + +==Specification== + +The new global type is defined as follows: + +{| +! Name +! +! +! Description +! +! Description +|- +| Output Descriptor +| PSBT_GLOBAL_OUTPUT_DESCRIPTOR = 0x07 +| +| The earliest block height that may contain transactions for the descriptor, optionally followed by the UTF-8 encoded name of the descriptor. +| +| The output descriptor in BIP380 format without inline keys. +|- +|} + +When PSBT_GLOBAL_OUTPUT_DESCRIPTOR is present, multiple PSBT_GLOBAL_XPUB entries are +allowed. + +When PSBT_GLOBAL_OUTPUT_DESCRIPTOR is present, the presence of PSBT_GLOBAL_UNSIGNED_TX +is optional. Note that this is a relaxation of the PSBT specification. + +All key expressions in the PSBT_GLOBAL_OUTPUT_DESCRIPTOR descriptor string must be +specified as references on the form @ where index is +the 0-based index into the ordered list of PSBT_GLOBAL_XPUB entries. An index +out of range is invalid. + +A PSBT_GLOBAL_OUTPUT_DESCRIPTOR with inline keys is invalid'''Why not allow inline +keys?''' +Allowing inline keys risks incompatible implementations that omit parsing of referenced +keys.'''What about named pk(NAME) references?''' +Named references would allow Miniscript descriptors as-is in PSBT_GLOBAL_OUTPUT_DESCRIPTOR. +They are left out because they complicate decoders and can trivially be replaced by indexed +references. However, if key names are deemed desirable for display purposes, they could be +squeezed into of PSBT_GLOBAL_XPUB entries.. + +Key references may be followed by derivation paths as specified in BIP 389. + +==Test Vectors== + +===Invalid Cases=== + +A descriptor with a key reference out of bounds. + Descriptor: wpkh(@0/*) + Hex encoded PSBT: 70736274ff0207000a77706b682840302f2a29000000 + +A descriptor with an invalid UTF-8 name. + Hex encoded PSBT: 70736274ff05070061c57a0a77706b682840302f2a2953010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 + +A descriptor with an inline key. + Hex encoded PSBT: 70736274ff0207008e77706b68285b64633536373237362f3438682f30682f30682f32685d7870756236446959726652774e6e6a655834764873574d616a4a56464b726245456e753867415739764475517a675457457345484531367347576558585556314c42575145317943546d657072534e63715a3357373468715664674462745948557633654d3457325445556870616e2f2a29000000 + +===Valid Cases=== + +A 2-of-3 multisig descriptor + Descriptor: wsh(sortedmulti(2,@0/<0;1>/*,@1/<0;1>/*,@2/<0;1>/*)) + Name: Satoshi's Stash + Birth block: 123456789012345 + Key 0: [dc567276/48h/0h/0h/2h]xpub6DiYrfRwNnjeX4vHsWMajJVFKrbEEnu8gAW9vDuQzgTWEsEHE16sGWeXXUV1LBWQE1yCTmeprSNcqZ3W74hqVdgDbtYHUv3eM4W2TEUhpan + Key 1: [f245ae38/48h/0h/0h/2h]xpub6DnT4E1fT8VxuAZW29avMjr5i99aYTHBp9d7fiLnpL5t4JEprQqPMbTw7k7rh5tZZ2F5g8PJpssqrZoebzBChaiJrmEvWwUTEMAbHsY39Ge + Key 2: [c5d87297/48h/0h/0h/2h]xpub6DjrnfAyuonMaboEb3ZQZzhQ2ZEgaKV2r64BFmqymZqJqviLTe1JzMr2X2RfQF892RH7MyYUbcy77R7pPu1P71xoj8cDUMNhAMGYzKR4noZ + Hex encoded PSBT: 70736274ff1907ff79df0d86487000005361746f73686927732053746173683477736828736f727465646d756c746928322c40302f3c303b313e2f2a2c40312f3c303b313e2f2a2c40322f3c303b313e2f2a292953010488b21e0418f8c2e7800000026b3a4cfb6a45f6305efe6e0e976b5d26ba27f7c344d7fc7abef7be2d06d52dfd021c0b479ecf6e67713ddf0c43b634592f51c037b6f951fb1dc6361a98b1e5735e0f8b515314dc5672764800008000000080000000800200008053010488b21e04221eb5a080000002c887c72d9d8ac29cddd5b2b060e8b0239039a149c784abe6079e24445db4aa8a0397fcf2274abd243d42d42d3c248608c6d1935efca46138afef43af08e971289657009d2b14f245ae384800008000000080000000800200008053010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 + +==Rationale== + + + +==Compatibility== + +PSBTs without a PSBT_GLOBAL_UNSIGNED_TX will be rejected by software expecting +it. However, such PSBTs are not intended to be used in a signing flow and so do +not pose a compatibility risk. + +PSBTs with multiple PSBT_GLOBAL_XPUBs may be rejected by software that expects a +single extended key whose derived addresses match (some of) the input and output +addresses. + +== Reference Implementation== + +There is a [https://github.com/seedhammer/bip-psbt-descriptors Go implementation] +for development and testing purposes. Don't use it in production, because it only +validates the PSBT format, not the descriptor itself. + +==Acknowledgments== + +This specification builds upon and supercedes the bip-wallet-policies draft BIP +by specifying a serialization format for compact descriptors. It also uses the +indexed key references from that BIP, as well as examples and test vectors. From f9eff32736faf601fefdc4d824ad29098378d3df Mon Sep 17 00:00:00 2001 From: seedhammer Date: Sat, 27 Apr 2024 15:09:01 +0200 Subject: [PATCH 2/7] rename bip-psbt-descriptors and separate format from BIP 174 (PSBT) --- bip-bod-descriptors.mediawiki | 139 +++++++++++++++++++++++++++++++++ bip-psbt-descriptors.mediawiki | 129 ------------------------------ 2 files changed, 139 insertions(+), 129 deletions(-) create mode 100644 bip-bod-descriptors.mediawiki delete mode 100644 bip-psbt-descriptors.mediawiki diff --git a/bip-bod-descriptors.mediawiki b/bip-bod-descriptors.mediawiki new file mode 100644 index 0000000000..b6d7635b24 --- /dev/null +++ b/bip-bod-descriptors.mediawiki @@ -0,0 +1,139 @@ +
+  BIP: ?
+  Layer: Applications
+  Title: Binary Output Descriptors
+  Author: SeedHammer 
+  License: BSD-2-Clause
+
+ +==Introduction== + +===Abstract=== + +This document proposes an efficient encoding format for output descriptors as +defined by [[bip-0380.mediawiki|BIP 380]] and its extensions. The format is +based on key-value maps as defined by [[bip-0174.mediawiki|BIP 174]] with a +distinct header to avoid confusing PSBT decoders. + +===Copyright=== + +This BIP is licensed under the BSD 2-clause license. + +===Motivation=== + +A BIP 380 output descriptor is a textual representation of a set of output +scripts, such that Bitcoin wallets may agree on the scripts and addresses +to provide the user. However, a descriptor string by itself is not ideal for +transferring between wallets: it lacks a machine recognizable header and cannot +represent metadata such as name and birth block. + +BIP 174 gives us a self-describing file format, metadata as well as a compact, +binary representation of keys. Assuming most wallets already implements the PSBT +format, the implementation overhead of this BIP is expected to be low. + +==Specification== + +The Binary Output Descriptor (BOD) format consists of a fixed header, a +key-value map describing the output descriptor and its metadata followed by a +map for each key referenced by the descriptor. + + := * + := 0x62 0x6F 0x64 0xFF + := * 0x00 + := * 0x00 + +The definition for follows [[bip-0174.mediawiki#specification|BIP 174]]. + +The defined global types are as follows: + +{| +! Name +! +! +! Description +! +! Description +|- +| Output Descriptor +| BOD_GLOBAL_OUTPUT_DESCRIPTOR = 0x00 +| +| The earliest block height that may contain transactions for the descriptor, optionally followed by the UTF-8 encoded name of the descriptor. +| +| The output descriptor in BIP 380 format without inline keys. +|} + +It is an error to omit the BOD_GLOBAL_OUTPUT_DESCRIPTOR entry. + +All key expressions in the BOD_GLOBAL_OUTPUT_DESCRIPTOR descriptor string must be +specified as references on the form @ where index is +the 0-based index into the ordered list of entries. An index +out of range is invalid. + +A BOD_GLOBAL_OUTPUT_DESCRIPTOR with inline keys is invalid'''Why not allow inline +keys?''' +Allowing inline keys risks incompatible implementations that omit parsing of referenced +keys.'''What about named pk(NAME) references?''' +Named references would allow Miniscript descriptors as-is in BOD_GLOBAL_OUTPUT_DESCRIPTOR. +They are left out because they complicate decoders and can trivially be replaced by indexed +references.. + +Key references may be followed by derivation paths as specified in [[bip-0389.mediawiki#specification|BIP 389]]. + +The defined key types are as follows: + +{| +! Name +! +! +! Description +! +! Description +|- +| Extended Public Key +| BOD_KEY_XPUB = 0x00 +| +| The 78 byte serialized extended public key as defined by BIP 32. +| <4 byte fingerprint> <32-bit little endian uint path element>* +| The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. +|} + +==Test Vectors== + +===Invalid Cases=== + +A descriptor with a key reference out of bounds. + Descriptor: wpkh(@0/*) + Hex encoded BOD: 70736274ff0207000a77706b682840302f2a29000000 + +A descriptor with an invalid UTF-8 name. + Hex encoded BOD: 70736274ff05070061c57a0a77706b682840302f2a2953010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 + +A descriptor with an inline key. + Hex encoded BOD: 70736274ff0207008e77706b68285b64633536373237362f3438682f30682f30682f32685d7870756236446959726652774e6e6a655834764873574d616a4a56464b726245456e753867415739764475517a675457457345484531367347576558585556314c42575145317943546d657072534e63715a3357373468715664674462745948557633654d3457325445556870616e2f2a29000000 + +===Valid Cases=== + +A 2-of-3 multisig descriptor + Descriptor: wsh(sortedmulti(2,@0/<0;1>/*,@1/<0;1>/*,@2/<0;1>/*)) + Name: Satoshi's Stash + Birth block: 123456789012345 + Key 0: [dc567276/48h/0h/0h/2h]xpub6DiYrfRwNnjeX4vHsWMajJVFKrbEEnu8gAW9vDuQzgTWEsEHE16sGWeXXUV1LBWQE1yCTmeprSNcqZ3W74hqVdgDbtYHUv3eM4W2TEUhpan + Key 1: [f245ae38/48h/0h/0h/2h]xpub6DnT4E1fT8VxuAZW29avMjr5i99aYTHBp9d7fiLnpL5t4JEprQqPMbTw7k7rh5tZZ2F5g8PJpssqrZoebzBChaiJrmEvWwUTEMAbHsY39Ge + Key 2: [c5d87297/48h/0h/0h/2h]xpub6DjrnfAyuonMaboEb3ZQZzhQ2ZEgaKV2r64BFmqymZqJqviLTe1JzMr2X2RfQF892RH7MyYUbcy77R7pPu1P71xoj8cDUMNhAMGYzKR4noZ + Hex encoded BOD: 70736274ff1907ff79df0d86487000005361746f73686927732053746173683477736828736f727465646d756c746928322c40302f3c303b313e2f2a2c40312f3c303b313e2f2a2c40322f3c303b313e2f2a292953010488b21e0418f8c2e7800000026b3a4cfb6a45f6305efe6e0e976b5d26ba27f7c344d7fc7abef7be2d06d52dfd021c0b479ecf6e67713ddf0c43b634592f51c037b6f951fb1dc6361a98b1e5735e0f8b515314dc5672764800008000000080000000800200008053010488b21e04221eb5a080000002c887c72d9d8ac29cddd5b2b060e8b0239039a149c784abe6079e24445db4aa8a0397fcf2274abd243d42d42d3c248608c6d1935efca46138afef43af08e971289657009d2b14f245ae384800008000000080000000800200008053010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 + +==Rationale== + + + +== Reference Implementation== + +There is a [https://github.com/seedhammer/bip-bod-descriptors Go implementation] +for development and testing purposes. Don't use it in production, because it only +validates the BOD format, not the descriptor itself. + +==Acknowledgments== + +This specification builds upon the draft BIP 388 by specifying +a serialization format for compact descriptors. It also uses the indexed key +references from that BIP, as well as examples and test vectors. diff --git a/bip-psbt-descriptors.mediawiki b/bip-psbt-descriptors.mediawiki deleted file mode 100644 index 74a1307e15..0000000000 --- a/bip-psbt-descriptors.mediawiki +++ /dev/null @@ -1,129 +0,0 @@ -
-  BIP: ?
-  Layer: Applications
-  Title: PSBT Encoded Output Descriptors
-  Author: SeedHammer 
-  License: BSD-2-Clause
-
- -==Introduction== - -===Abstract=== - -A BIP 174 PSBT may contain an extended key for deriving input and output -addresses. This document proposes an additional field for PSBTs to represent -arbitrary BIP 380 output script descriptors. - -To support transfer of output descriptors outside signing flows, the proposal -makes the unsigned transaction optional. - -===Copyright=== - -This BIP is licensed under the BSD 2-clause license. - -===Motivation=== - -A BIP 380 output descriptor is a textual representation of a set of output -scripts, such that Bitcoin wallets may agree on the scripts and addresses -to provide the user. However, a descriptor string by itself is not ideal for -transferring between wallets: it lacks a machine recognizable header and cannot -represent metadata such as name and birth block. - -The PSBT encoding gives us a self-describing file format, metadata as well as a -compact, binary representation of keys. Assuming most wallets already implements -the PSBT format, the additional implementation overhead of this extension is -expected to be low. - -==Specification== - -The new global type is defined as follows: - -{| -! Name -! -! -! Description -! -! Description -|- -| Output Descriptor -| PSBT_GLOBAL_OUTPUT_DESCRIPTOR = 0x07 -| -| The earliest block height that may contain transactions for the descriptor, optionally followed by the UTF-8 encoded name of the descriptor. -| -| The output descriptor in BIP380 format without inline keys. -|- -|} - -When PSBT_GLOBAL_OUTPUT_DESCRIPTOR is present, multiple PSBT_GLOBAL_XPUB entries are -allowed. - -When PSBT_GLOBAL_OUTPUT_DESCRIPTOR is present, the presence of PSBT_GLOBAL_UNSIGNED_TX -is optional. Note that this is a relaxation of the PSBT specification. - -All key expressions in the PSBT_GLOBAL_OUTPUT_DESCRIPTOR descriptor string must be -specified as references on the form @ where index is -the 0-based index into the ordered list of PSBT_GLOBAL_XPUB entries. An index -out of range is invalid. - -A PSBT_GLOBAL_OUTPUT_DESCRIPTOR with inline keys is invalid'''Why not allow inline -keys?''' -Allowing inline keys risks incompatible implementations that omit parsing of referenced -keys.'''What about named pk(NAME) references?''' -Named references would allow Miniscript descriptors as-is in PSBT_GLOBAL_OUTPUT_DESCRIPTOR. -They are left out because they complicate decoders and can trivially be replaced by indexed -references. However, if key names are deemed desirable for display purposes, they could be -squeezed into of PSBT_GLOBAL_XPUB entries.. - -Key references may be followed by derivation paths as specified in BIP 389. - -==Test Vectors== - -===Invalid Cases=== - -A descriptor with a key reference out of bounds. - Descriptor: wpkh(@0/*) - Hex encoded PSBT: 70736274ff0207000a77706b682840302f2a29000000 - -A descriptor with an invalid UTF-8 name. - Hex encoded PSBT: 70736274ff05070061c57a0a77706b682840302f2a2953010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 - -A descriptor with an inline key. - Hex encoded PSBT: 70736274ff0207008e77706b68285b64633536373237362f3438682f30682f30682f32685d7870756236446959726652774e6e6a655834764873574d616a4a56464b726245456e753867415739764475517a675457457345484531367347576558585556314c42575145317943546d657072534e63715a3357373468715664674462745948557633654d3457325445556870616e2f2a29000000 - -===Valid Cases=== - -A 2-of-3 multisig descriptor - Descriptor: wsh(sortedmulti(2,@0/<0;1>/*,@1/<0;1>/*,@2/<0;1>/*)) - Name: Satoshi's Stash - Birth block: 123456789012345 - Key 0: [dc567276/48h/0h/0h/2h]xpub6DiYrfRwNnjeX4vHsWMajJVFKrbEEnu8gAW9vDuQzgTWEsEHE16sGWeXXUV1LBWQE1yCTmeprSNcqZ3W74hqVdgDbtYHUv3eM4W2TEUhpan - Key 1: [f245ae38/48h/0h/0h/2h]xpub6DnT4E1fT8VxuAZW29avMjr5i99aYTHBp9d7fiLnpL5t4JEprQqPMbTw7k7rh5tZZ2F5g8PJpssqrZoebzBChaiJrmEvWwUTEMAbHsY39Ge - Key 2: [c5d87297/48h/0h/0h/2h]xpub6DjrnfAyuonMaboEb3ZQZzhQ2ZEgaKV2r64BFmqymZqJqviLTe1JzMr2X2RfQF892RH7MyYUbcy77R7pPu1P71xoj8cDUMNhAMGYzKR4noZ - Hex encoded PSBT: 70736274ff1907ff79df0d86487000005361746f73686927732053746173683477736828736f727465646d756c746928322c40302f3c303b313e2f2a2c40312f3c303b313e2f2a2c40322f3c303b313e2f2a292953010488b21e0418f8c2e7800000026b3a4cfb6a45f6305efe6e0e976b5d26ba27f7c344d7fc7abef7be2d06d52dfd021c0b479ecf6e67713ddf0c43b634592f51c037b6f951fb1dc6361a98b1e5735e0f8b515314dc5672764800008000000080000000800200008053010488b21e04221eb5a080000002c887c72d9d8ac29cddd5b2b060e8b0239039a149c784abe6079e24445db4aa8a0397fcf2274abd243d42d42d3c248608c6d1935efca46138afef43af08e971289657009d2b14f245ae384800008000000080000000800200008053010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 - -==Rationale== - - - -==Compatibility== - -PSBTs without a PSBT_GLOBAL_UNSIGNED_TX will be rejected by software expecting -it. However, such PSBTs are not intended to be used in a signing flow and so do -not pose a compatibility risk. - -PSBTs with multiple PSBT_GLOBAL_XPUBs may be rejected by software that expects a -single extended key whose derived addresses match (some of) the input and output -addresses. - -== Reference Implementation== - -There is a [https://github.com/seedhammer/bip-psbt-descriptors Go implementation] -for development and testing purposes. Don't use it in production, because it only -validates the PSBT format, not the descriptor itself. - -==Acknowledgments== - -This specification builds upon and supercedes the bip-wallet-policies draft BIP -by specifying a serialization format for compact descriptors. It also uses the -indexed key references from that BIP, as well as examples and test vectors. From 5151e3f8e965f50874d7e3f29a1e5c8eeab29d02 Mon Sep 17 00:00:00 2001 From: seedhammer Date: Mon, 29 Apr 2024 17:55:09 +0200 Subject: [PATCH 3/7] bip-bod-descriptors: add mandatory headers and compatibility section --- bip-bod-descriptors.mediawiki | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/bip-bod-descriptors.mediawiki b/bip-bod-descriptors.mediawiki index b6d7635b24..5d87208a2b 100644 --- a/bip-bod-descriptors.mediawiki +++ b/bip-bod-descriptors.mediawiki @@ -3,6 +3,10 @@ Layer: Applications Title: Binary Output Descriptors Author: SeedHammer + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-seedhammer-binary-output-descriptors + Status: Draft + Type: Standards Track + Created: 2024-01-31 License: BSD-2-Clause @@ -126,6 +130,10 @@ A 2-of-3 multisig descriptor +==Compatibility== + +This is a new format and as such have no campatibility burden. + == Reference Implementation== There is a [https://github.com/seedhammer/bip-bod-descriptors Go implementation] From e483f82999aee97cc4530ab35d627003157a8180 Mon Sep 17 00:00:00 2001 From: seedhammer Date: Mon, 29 Apr 2024 18:51:38 +0200 Subject: [PATCH 4/7] bip-bod-descriptors: expanded rationale section --- bip-bod-descriptors.mediawiki | 43 +++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/bip-bod-descriptors.mediawiki b/bip-bod-descriptors.mediawiki index 5d87208a2b..dee9e7f80d 100644 --- a/bip-bod-descriptors.mediawiki +++ b/bip-bod-descriptors.mediawiki @@ -15,9 +15,9 @@ ===Abstract=== This document proposes an efficient encoding format for output descriptors as -defined by [[bip-0380.mediawiki|BIP 380]] and its extensions. The format is -based on key-value maps as defined by [[bip-0174.mediawiki|BIP 174]] with a -distinct header to avoid confusing PSBT decoders. +defined by [[bip-0380.mediawiki|BIP 380]] and its extensions. Based on +key-value maps as defined by [[bip-0174.mediawiki|BIP 174]], the encoding +supports descriptor and key metadata. ===Copyright=== @@ -28,12 +28,34 @@ This BIP is licensed under the BSD 2-clause license. A BIP 380 output descriptor is a textual representation of a set of output scripts, such that Bitcoin wallets may agree on the scripts and addresses to provide the user. However, a descriptor string by itself is not ideal for -transferring between wallets: it lacks a machine recognizable header and cannot -represent metadata such as name and birth block. +transferring between wallets: it lacks a machine recognizable header, cannot +represent metadata such as name and birth block, and its use of base58 for +extended keys is inefficient. BIP 174 gives us a self-describing file format, metadata as well as a compact, binary representation of keys. Assuming most wallets already implements the PSBT -format, the implementation overhead of this BIP is expected to be low. +format, the implementation overhead of this BIP is expected to be low + '''Why not extend BIP 174?''' +The PSBT format is not a general purpose container format, and descriptors +are useful outside of signing flows (backup, initializing wallet +software). + '''Why not use the older Blockchain Commons BCR-2020-010 format?''' +The format was [[https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2020-010-output-desc.md#deprecated-superseded-by-version-3-output-descriptors|recently deprecated]], +because its use of reserved CBOR tags. +'''Why not use the new Blockchain Commons BCR-2023-010 format?''' +The [[https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2023-010-output-descriptor.md|BCR-2023-010 format]] +is roughly equivalent to this proposal, but was designed and released outside of +the usual BIP proposal process. + +In particular, we believe BCR-2023-010 would have faced opposition +in the BIP process because it is based on the [[https://cbor.io/spec.html|CBOR encoding]]. +CBOR is designed for generality and compactness and is therefore significantly +more complex than the PSBT encoding. In contrast, the PSBT key-value maps are +supported by all wallet software that interact with PSBT files, are easier to +review and fit in embedded devices. + +Release in late 2023, BCR-2023-010 doesn't event have the advantage of widespread +support.. ==Specification== @@ -63,7 +85,14 @@ The defined global types are as follows: | | The earliest block height that may contain transactions for the descriptor, optionally followed by the UTF-8 encoded name of the descriptor. | -| The output descriptor in BIP 380 format without inline keys. +| The output descriptor in BIP 380 format without inline keys''Why not encode the descriptor in binary?''' +The BIP 380 descriptor language is complicated and expanding and we believe +the efforts of designing a parallel binary encoding does not outweigh the +space savings. + +The major source of bloat, base58 encoded keys, are binary encoded for compactness and +can be re-used multiple times in a single descriptor. +. |} It is an error to omit the BOD_GLOBAL_OUTPUT_DESCRIPTOR entry. From 710fe837a6d6d702e196cb33c1e1b56f854ce73d Mon Sep 17 00:00:00 2001 From: seedhammer Date: Tue, 30 Apr 2024 14:44:12 +0200 Subject: [PATCH 5/7] bip-bod-descriptors: add proprietary fields --- bip-bod-descriptors.mediawiki | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/bip-bod-descriptors.mediawiki b/bip-bod-descriptors.mediawiki index dee9e7f80d..3a2997851e 100644 --- a/bip-bod-descriptors.mediawiki +++ b/bip-bod-descriptors.mediawiki @@ -93,6 +93,13 @@ space savings. The major source of bloat, base58 encoded keys, are binary encoded for compactness and can be re-used multiple times in a single descriptor. . +|- +| Proprietary Use Type +| BOD_GLOBAL_PROPRIETARY = 0xFC +| +| Compact size unsigned integer of the length of the identifier, followed by identifier prefix, followed by a compact size unsigned integer subtype, followed by the key data itself. +| +| Any value data as defined by the proprietary type user. |} It is an error to omit the BOD_GLOBAL_OUTPUT_DESCRIPTOR entry. @@ -128,6 +135,13 @@ The defined key types are as follows: | The 78 byte serialized extended public key as defined by BIP 32. | <4 byte fingerprint> <32-bit little endian uint path element>* | The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. +|- +| Proprietary Use Type +| BOD_KEY_PROPRIETARY = 0xFC +| +| Compact size unsigned integer of the length of the identifier, followed by identifier prefix, followed by a compact size unsigned integer subtype, followed by the key data itself. +| +| Any value data as defined by the proprietary type user. |} ==Test Vectors== From c6c727ccc8336fb06964a85d0a4cfa30c6938fba Mon Sep 17 00:00:00 2001 From: SeedHammer <93251679+seedhammer@users.noreply.github.com> Date: Thu, 2 May 2024 10:47:21 +0200 Subject: [PATCH 6/7] bip-bod-descriptors: apply suggested edits Suggestions by @jonatack. Co-authored-by: Jon Atack --- bip-bod-descriptors.mediawiki | 42 +++++++++++++++++------------------ 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/bip-bod-descriptors.mediawiki b/bip-bod-descriptors.mediawiki index 3a2997851e..58a6165931 100644 --- a/bip-bod-descriptors.mediawiki +++ b/bip-bod-descriptors.mediawiki @@ -28,21 +28,21 @@ This BIP is licensed under the BSD 2-clause license. A BIP 380 output descriptor is a textual representation of a set of output scripts, such that Bitcoin wallets may agree on the scripts and addresses to provide the user. However, a descriptor string by itself is not ideal for -transferring between wallets: it lacks a machine recognizable header, cannot +transferring between wallets: it lacks a machine-recognizable header, cannot represent metadata such as name and birth block, and its use of base58 for extended keys is inefficient. -BIP 174 gives us a self-describing file format, metadata as well as a compact, -binary representation of keys. Assuming most wallets already implements the PSBT +BIP 174 gives us a self-describing file format, metadata, and a compact, +binary representation of keys. Assuming most wallets already implement the PSBT format, the implementation overhead of this BIP is expected to be low '''Why not extend BIP 174?''' -The PSBT format is not a general purpose container format, and descriptors +The PSBT format is not a general-purpose container format, and descriptors are useful outside of signing flows (backup, initializing wallet software). '''Why not use the older Blockchain Commons BCR-2020-010 format?''' The format was [[https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2020-010-output-desc.md#deprecated-superseded-by-version-3-output-descriptors|recently deprecated]], -because its use of reserved CBOR tags. -'''Why not use the new Blockchain Commons BCR-2023-010 format?''' +due to its use of reserved CBOR tags. +'''Why not use the new Blockchain Commons BCR-2023-010 format?'''. The [[https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2023-010-output-descriptor.md|BCR-2023-010 format]] is roughly equivalent to this proposal, but was designed and released outside of the usual BIP proposal process. @@ -52,10 +52,10 @@ in the BIP process because it is based on the [[https://cbor.io/spec.html|CBOR e CBOR is designed for generality and compactness and is therefore significantly more complex than the PSBT encoding. In contrast, the PSBT key-value maps are supported by all wallet software that interact with PSBT files, are easier to -review and fit in embedded devices. +review, and they fit in embedded devices. -Release in late 2023, BCR-2023-010 doesn't event have the advantage of widespread -support.. +Released in late 2023, BCR-2023-010 doesn't even have the advantage of widespread +support at the time of this writing.. ==Specification== @@ -86,18 +86,18 @@ The defined global types are as follows: | The earliest block height that may contain transactions for the descriptor, optionally followed by the UTF-8 encoded name of the descriptor. | | The output descriptor in BIP 380 format without inline keys''Why not encode the descriptor in binary?''' -The BIP 380 descriptor language is complicated and expanding and we believe -the efforts of designing a parallel binary encoding does not outweigh the +The BIP 380 descriptor language is complicated and expanding, and we believe +the effort of designing a parallel binary encoding does not outweigh the space savings. -The major source of bloat, base58 encoded keys, are binary encoded for compactness and +The major source of bloat, base58-encoded keys, are binary encoded for compactness and can be re-used multiple times in a single descriptor. . |- | Proprietary Use Type | BOD_GLOBAL_PROPRIETARY = 0xFC | -| Compact size unsigned integer of the length of the identifier, followed by identifier prefix, followed by a compact size unsigned integer subtype, followed by the key data itself. +| A compact size unsigned integer of the length of the identifier, followed by an identifier prefix, followed by a compact size unsigned integer subtype, followed by the key data itself. | | Any value data as defined by the proprietary type user. |} @@ -105,7 +105,7 @@ can be re-used multiple times in a single descriptor. It is an error to omit the BOD_GLOBAL_OUTPUT_DESCRIPTOR entry. All key expressions in the BOD_GLOBAL_OUTPUT_DESCRIPTOR descriptor string must be -specified as references on the form @ where index is +specified as references in the form of @ where index is the 0-based index into the ordered list of entries. An index out of range is invalid. @@ -132,14 +132,14 @@ The defined key types are as follows: | Extended Public Key | BOD_KEY_XPUB = 0x00 | -| The 78 byte serialized extended public key as defined by BIP 32. +| The 78-byte serialized extended public key, as defined by BIP 32. | <4 byte fingerprint> <32-bit little endian uint path element>* -| The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. +| The master key fingerprint, as defined by BIP 32, concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. The number of 32-bit unsigned integer indexes must match the depth provided in the extended public key. |- | Proprietary Use Type | BOD_KEY_PROPRIETARY = 0xFC | -| Compact size unsigned integer of the length of the identifier, followed by identifier prefix, followed by a compact size unsigned integer subtype, followed by the key data itself. +| A compact size unsigned integer of the length of the identifier, followed by an identifier prefix, followed by a compact size unsigned integer subtype, followed by the key data itself. | | Any value data as defined by the proprietary type user. |} @@ -153,10 +153,10 @@ A descriptor with a key reference out of bounds. Hex encoded BOD: 70736274ff0207000a77706b682840302f2a29000000 A descriptor with an invalid UTF-8 name. - Hex encoded BOD: 70736274ff05070061c57a0a77706b682840302f2a2953010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 + Hex-encoded BOD: 70736274ff05070061c57a0a77706b682840302f2a2953010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 A descriptor with an inline key. - Hex encoded BOD: 70736274ff0207008e77706b68285b64633536373237362f3438682f30682f30682f32685d7870756236446959726652774e6e6a655834764873574d616a4a56464b726245456e753867415739764475517a675457457345484531367347576558585556314c42575145317943546d657072534e63715a3357373468715664674462745948557633654d3457325445556870616e2f2a29000000 + Hex-encoded BOD: 70736274ff0207008e77706b68285b64633536373237362f3438682f30682f30682f32685d7870756236446959726652774e6e6a655834764873574d616a4a56464b726245456e753867415739764475517a675457457345484531367347576558585556314c42575145317943546d657072534e63715a3357373468715664674462745948557633654d3457325445556870616e2f2a29000000 ===Valid Cases=== @@ -167,7 +167,7 @@ A 2-of-3 multisig descriptor Key 0: [dc567276/48h/0h/0h/2h]xpub6DiYrfRwNnjeX4vHsWMajJVFKrbEEnu8gAW9vDuQzgTWEsEHE16sGWeXXUV1LBWQE1yCTmeprSNcqZ3W74hqVdgDbtYHUv3eM4W2TEUhpan Key 1: [f245ae38/48h/0h/0h/2h]xpub6DnT4E1fT8VxuAZW29avMjr5i99aYTHBp9d7fiLnpL5t4JEprQqPMbTw7k7rh5tZZ2F5g8PJpssqrZoebzBChaiJrmEvWwUTEMAbHsY39Ge Key 2: [c5d87297/48h/0h/0h/2h]xpub6DjrnfAyuonMaboEb3ZQZzhQ2ZEgaKV2r64BFmqymZqJqviLTe1JzMr2X2RfQF892RH7MyYUbcy77R7pPu1P71xoj8cDUMNhAMGYzKR4noZ - Hex encoded BOD: 70736274ff1907ff79df0d86487000005361746f73686927732053746173683477736828736f727465646d756c746928322c40302f3c303b313e2f2a2c40312f3c303b313e2f2a2c40322f3c303b313e2f2a292953010488b21e0418f8c2e7800000026b3a4cfb6a45f6305efe6e0e976b5d26ba27f7c344d7fc7abef7be2d06d52dfd021c0b479ecf6e67713ddf0c43b634592f51c037b6f951fb1dc6361a98b1e5735e0f8b515314dc5672764800008000000080000000800200008053010488b21e04221eb5a080000002c887c72d9d8ac29cddd5b2b060e8b0239039a149c784abe6079e24445db4aa8a0397fcf2274abd243d42d42d3c248608c6d1935efca46138afef43af08e971289657009d2b14f245ae384800008000000080000000800200008053010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 + Hex-encoded BOD: 70736274ff1907ff79df0d86487000005361746f73686927732053746173683477736828736f727465646d756c746928322c40302f3c303b313e2f2a2c40312f3c303b313e2f2a2c40322f3c303b313e2f2a292953010488b21e0418f8c2e7800000026b3a4cfb6a45f6305efe6e0e976b5d26ba27f7c344d7fc7abef7be2d06d52dfd021c0b479ecf6e67713ddf0c43b634592f51c037b6f951fb1dc6361a98b1e5735e0f8b515314dc5672764800008000000080000000800200008053010488b21e04221eb5a080000002c887c72d9d8ac29cddd5b2b060e8b0239039a149c784abe6079e24445db4aa8a0397fcf2274abd243d42d42d3c248608c6d1935efca46138afef43af08e971289657009d2b14f245ae384800008000000080000000800200008053010488b21e041c0ae906800000025afed56d755c088320ec9bc6acd84d33737b580083759e0a0ff8f26e429e0b77028342f5f7773f6fab374e1c2d3ccdba26bc0933fc4f63828b662b4357e4cc3791bec0fbd814c5d8729748000080000000800000008002000080000000 ==Rationale== @@ -175,7 +175,7 @@ A 2-of-3 multisig descriptor ==Compatibility== -This is a new format and as such have no campatibility burden. +This is a new format and as such has no compatibility burden. == Reference Implementation== From 36403bec0124addaca245c50f7d8dd44fae21319 Mon Sep 17 00:00:00 2001 From: seedhammer Date: Thu, 2 May 2024 10:59:14 +0200 Subject: [PATCH 7/7] bip-bod-descriptors: apply more edits suggested during review Suggestions by @jonatack. --- bip-bod-descriptors.mediawiki | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/bip-bod-descriptors.mediawiki b/bip-bod-descriptors.mediawiki index 58a6165931..21ad0ab6cb 100644 --- a/bip-bod-descriptors.mediawiki +++ b/bip-bod-descriptors.mediawiki @@ -42,7 +42,7 @@ software). '''Why not use the older Blockchain Commons BCR-2020-010 format?''' The format was [[https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2020-010-output-desc.md#deprecated-superseded-by-version-3-output-descriptors|recently deprecated]], due to its use of reserved CBOR tags. -'''Why not use the new Blockchain Commons BCR-2023-010 format?'''. +'''Why not use the new Blockchain Commons BCR-2023-010 format?''' The [[https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2023-010-output-descriptor.md|BCR-2023-010 format]] is roughly equivalent to this proposal, but was designed and released outside of the usual BIP proposal process. @@ -54,14 +54,14 @@ more complex than the PSBT encoding. In contrast, the PSBT key-value maps are supported by all wallet software that interact with PSBT files, are easier to review, and they fit in embedded devices. -Released in late 2023, BCR-2023-010 doesn't even have the advantage of widespread +Released in late 2023, BCR-2023-010 doesn't have the advantage of widespread support at the time of this writing.. ==Specification== The Binary Output Descriptor (BOD) format consists of a fixed header, a -key-value map describing the output descriptor and its metadata followed by a -map for each key referenced by the descriptor. +key-value map describing the output descriptor, and a key-value map for +each key. := * := 0x62 0x6F 0x64 0xFF @@ -86,13 +86,13 @@ The defined global types are as follows: | The earliest block height that may contain transactions for the descriptor, optionally followed by the UTF-8 encoded name of the descriptor. | | The output descriptor in BIP 380 format without inline keys''Why not encode the descriptor in binary?''' -The BIP 380 descriptor language is complicated and expanding, and we believe -the effort of designing a parallel binary encoding does not outweigh the -space savings. +Designing and maintaining a compact binary representation of the complex and evolving BIP 380 descriptor +language entail significant effort and introduce a time lag from the introduction of new BIP 380 features +to the spcification of the binary counterpart. We don't believe the space savings outweigh the disadvantages. The major source of bloat, base58-encoded keys, are binary encoded for compactness and can be re-used multiple times in a single descriptor. -. +. |- | Proprietary Use Type | BOD_GLOBAL_PROPRIETARY = 0xFC