Specify opcodes for MATT

[bigspider]

Edit: Obsolete, see below for the latest attempt.

---

Hello, I hope it's fine if I slightly abuse this space to get feedback on my specific proposal - please let me know if this is not appreciated and I'll find a different venue.

I'm trying to formalize the exact opcodes for MATT for the purpose of implementing them for bitcoin-inquisition.

One thing that was left unspecified in the initial sketch (h/t @darosior for pointing that out) is that a way is needed to make "arguments" passed to the covenant non-malleable, if they are passed via the witness.
Building on top of Antoine Riard's PR on the annex format, I think using the annex for this purpose is the best course of action.

I would love if someone more experienced with bitcoin-core would like to help with putting together a good PR.
MATT needs your help! :)

This is what I'm planning right now for an (almost) complete set of opcodes - please let me know if you have any comments or suggestions.

MATT tentative specs

Covenant opcodes: verify or commit to data in the taproot internal key of an input/output

• OP_CHECKINPUTCOVENANTVERIFY: let x, d be the two top elements of the stack; fail if any of x and d is not exactly 32 bytes; otherwise, check that the x is a valid x-only pubkey, and the internal pubkey P of the current input is indeed obtained by tweaking lift_x(x) with d.
• OP_CHECKOUTPUTCOVENANTVERIFY: given a number out_i and three 32-byte hash elements x, d and taptree on top of the stack, verifies that the out_i-th output is a P2TR output with the taproot internal key computed as above, and tweaked with taptree. Fail if if any of x, d or taptree is not exactly 32 bytes.

Data manipulation:

• OP_CAT: pop the top two stack elements, push their concatenation; fail if the concatenation is longer than a predefined limit (256 bytes?).

Additional introspection (compulsory):

• OP_PUSH_ANNEX_RECORD: details TBD; push an annex record onto the stack. To build on top of Always Look On The Bright Side of the Annex bitcoin-inquisition/bitcoin#9.

Additional introspection (optional): one can probably do some interesting things without, but these would allow concepts like "pay to a contract" or "withdraw from a contract", where the contract is a covenant-encumbered UTXO.

• OP_INSPECTNUMINPUTS and OP_INSPECTNUMOUTPUTS: push to the stack the number of inputs/outputs.
• OP_INSPECTINPUTVALUE and OP_INSPECTOUTPUTVALUE: given an index i popped from the top of the stack, push to the stack the amount in sats of a certain input/output.

Still unresolved: as pointed out by Anthony Towns, amounts do not fit in 32-bytes; therefore, some way of enabling 64-bit maths would be desirable. It's a technicality, so it might safely be left for later.

Comments

• OP_CHECKINPUTCOVENANTVERIFY and OP_CHECKOUTPUTCOVENANTVERIFY could be replaced with more general opcodes to inspect input/output scriptPubKey, and do some elliptic curve maths. Those are already available in Liquid, see here.

[ariard]

Hello, I hope it's fine if I slightly abuse this space to get feedback on my specific proposal - please let me know if this is not appreciated and I'll find a different venue.

Proposals are very welcome! We're still experimenting with the format, please feel free to provide any feedback :)

I'm trying to formalize the exact opcodes for MATT for the purpose of implementing them for bitcoin-inquisition.

One thing that was left unspecified in the initial sketch (h/t @darosior for pointing that out) is that a way is needed to make "arguments" passed to the covenant non-malleable, if they are passed via the witness.
Building on top of Antoine Riard's PR on the annex format, I think using the annex for this purpose is the best course of action.

The annex has always been devised as a generic mechanism to extend the transaction data fields. So yes passing "arguments"
here sounds a good fit, is there any draft specification to explain the data format of "arguments" and check if they bind
well with current annex TLV records and sanitization rules, or if we need to relax them ?

Covenant opcodes: verify or commit to data in the taproot internal key of an input/output

OP_CHECKINPUTCOVENANTVERIFY: let x, d be the two top elements of the stack; fail if any of x and d is not exactly 32 bytes; otherwise, check that the x is a valid x-only pubkey, and the internal pubkey P of the current input is indeed obtained by tweaking lift_x(x) with d.
OP_CHECKOUTPUTCOVENANTVERIFY: given a number out_i and three 32-byte hash elements x, d and taptree on top of the stack, verifies that the out_i-th output is a P2TR output with the taproot internal key computed as above, and tweaked with taptree. Fail if if any of x, d or taptree is not exactly 32 bytes.

About OP_CHECKINPUTCOVENANTVERIFY (and I think it holds for output commitment verification), I think in the past there were
discussions about a nested inspection, where you could verify multiple level of tweaks. There is also the old generalized
taproot proposal in this direction I think,
maybe it could be combined I don't know ? Though maybe better to start with a simple version and see how to enrich after.

Additional introspection (compulsory):

OP_PUSH_ANNEX_RECORD: details TBD; push an annex record onto the stack. To build on top of Always Look On The Bright Side of the Annex bitcoin-inquisition/bitcoin#9.

Yeah, on me here to clean and finalize this branch quickly!

Additional introspection (optional): one can probably do some interesting things without, but these would allow concepts like "pay to a contract" or "withdraw from a contract", where the contract is a covenant-encumbered UTXO.

OP_INSPECTNUMINPUTS and OP_INSPECTNUMOUTPUTS: push to the stack the number of inputs/outputs.
OP_INSPECTINPUTVALUE and OP_INSPECTOUTPUTVALUE: given an index i popped from the top of the stack, push to the stack the amount in sats of a certain input/output.

I think there is the idea of OP_PUSH_IN_OUT_AMOUNTS (sketched out in the TLUV OG thread iirc) which could be understood as some kind of super-set of both proposal.
Though lower-level of amount granularity likely open more use-cases.

Staying available to answer Script coding questions on #bitcoin-contracting-primitives-wg :)

[bigspider]

The annex has always been devised as a generic mechanism to extend the transaction data fields. So yes passing "arguments" here sounds a good fit, is there any draft specification to explain the data format of "arguments" and check if they bind well with current annex TLV records and sanitization rules, or if we need to relax them ?

The purpose of those "arguments" depends on the specific contract; basically anything that can be pushed onto the stack would do.
With the usual tricks (hashes and Merkle trees), one could always reduce arbitrary data to a single hash, but I guess it might be convenient to be able to pass smaller data directly.

About OP_CHECKINPUTCOVENANTVERIFY (and I think it holds for output commitment verification), I think in the past there were discussions about a nested inspection, where you could verify multiple level of tweaks. There is also the old generalized taproot proposal in this direction I think, maybe it could be combined I don't know ? Though maybe better to start with a simple version and see how to enrich after.

As the design space is rather large (and the ability to do MATT covenants isn't really affected by small changes in the specs), I tried to keep the opcodes minimal, but not unnaturally minimal (hence CAT instead of SHA256CAT).
I'm not sure there is an advantage of generalizing to multiple tweaks for the specific constructions I have in mind, but it seems likely that there might be some interesting ones.

I think there is the idea of OP_PUSH_IN_OUT_AMOUNTS (sketched out in the TLUV OG thread iirc) which could be understood as some kind of super-set of both proposal. Though lower-level of amount granularity likely open more use-cases.

The reason I went for these 4 is that I could copy them directly from Liquid opcodes; I also tend to prefer more atomic operations, but I'm ok with other options, too.

[bigspider]

Hello, I did some work to formalize the core opcodes, based on the progress in the last few months. The changes are as follows:

• Generalize CICV and COCV above to a single OP_CHECKCONTRACTVERIFY, that is a superset of both, and can work on any input or output.
• Define special values to refer to the current input index, or the current taptree; Rationale: allows constructions that modify some data but keep the same taptree.
• Define a special value for a NUMS pubkey.
• Make both the tweaks optional, by allowing the empty string.
• Define an optional semantics for output amounts preservation. Rationale: for many contracts a logic of preserving the input amounts to the outputs is sufficient (vaults, payment channels, state channels, etc.); but this seems to inherently fall short for constructions like coinpools (see for example [here].(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-May/021719.html)), where more explicit amount introspection seems necessary.

Opcodes to access the annex do not seem to be necessary for most use cases, unlike what I thought in the initial post above.

I opted to not add a tag to the data tweak (unlike the taptweak that is tagged as per taproot rules); see here for the reasoning, which seems to apply here as well.

OP_CHECKCONTRACTVERIFY

Intro

OP_CHECKCONTRACTVERIFY is only active for scripts spending a Segwitv1 input. Get data, index, pk, taptree, flags from the stack (bottom-to-top).

OP_CHECKCONTRACTVERIFY verifies that the scriptPubKey of the input/output with the given index is a P2TR script with a pubkey obtained by the x-only pubkey pk, optionally tweaked with data, optionally taptweaked with taptree. The CIOCV_FLAG_CHECK_INPUT determines if the index refers to an input or an output. Special values for the parameters, are listed below.

The flags parameter alters the behaviour of the opcode. If negative, the opcode checks the scriptPubkey of an input; otherwise, it checks the one of the output. The following values for the flags are currently defined for checking an input:

• CCV_FLAG_CHECK_INPUT = -1: makes the opcode check an input.

Non-negative values make the opcode check an output, and different values have different behaviour in the way the output's amount (nValue) is checked. The following values for the flags are currently defined for checking an output:

• 0: default behavior, the (possibly residual) amount of this input must be present in the output. This amount
• CCV_FLAG_IGNORE_OUTPUT_AMOUNT = 1: For outputs, disables the default deferred checks on amounts defined below. Undefined when CCV_FLAG_CHECK_INPUT is present.
• CCV_FLAG_DEDUCT_OUTPUT_AMOUNT = 2: Fail if the amount of the output is larger than the amount of the input; otherwise, subtracts the value of the output from the value of the current input in future calls top OP_CHECKCONTRACTVERIFY.

The following values of the parameters are special values:

• If pk is empty, it is replaced with the NUMS x-only pubkey 0x50929b74c1a04954b78b4b6035e97a5e078a5a0f28ec96d547bfee9ace803ac0 defined in BIP-0340.
• If pk is -1, it is replaced with the current input's internal key.
• If index is -1, it is replaced with the current input index.
• If data is empty, the data tweak is skipped.
• If taptree is empty, the taptweak is skipped.
• If taptree is -1, the taptree of the current input is used for the taptweak.

The following additional deferred checks are performed after the validation of all inputs is completed:

• The amount of each output must be at least equal to the sum of the amount of all the inputs that have a CCV for that output with the default flag (equal to 0).
• No output that is a target of CCV_FLAG_DEDUCT_OUTPUT_AMOUNT can also be the target of another OP_CHECKCONTRACTVERIFY, unless it's with the CCV_FLAG_IGNORE_OUTPUT_AMOUNT.

Specs

Semantics (initialization before input evaluation):

  for out_index in range(n_outputs)
    out_min_amount[out_index] = 0

Semantics (per input):

if flags < CCV_FLAG_CHECK_INPUT or flags > CCV_FLAG_DEDUCT_OUTPUT_AMOUNT:
  return success()  # undefined flags are OP_SUCCESS

if index == -1:
  index = current_input_index

if flags == CCV_FLAG_CHECK_INPUT:
  if index < 0 or index >= n_inputs:
    return fail()

  target = inputs[index].scriptPubKey
else:
  if index < 0 or index >= n_outputs:
    return fail()

  target = outputs[index].scriptPubKey

if taptree == <-1>:
  taptree = current_input_taptree

if pk == <0>:
  result = BIP340_NUMS_KEY
elif pk == <-1>:
  result = current_input_internal_key 
elif len(pk) == 32:
  result = pk
else:
  return fail()

if data != <0>:
  if len(data) != 32:
    return fail()

  result = tweak(result, data)

if len(taptree) != 0:
  if len(taptree) != 32:
    return fail()

  result = taptweak(result, taptree)

if target != P2TR(result)
  return fail()

if flags == 0:
  out_min_amount[index] += inputs[current_input_index].amount
elif flags == CCV_FLAG_DEDUCT_OUTPUT_AMOUNT:
  if inputs[current_input_index].amount > outputs[index].amount:
    return fail()
  inputs[current_input_index].amount -= outputs[index].amount

stack.pop(5)  # drop all 5 stack elements

Semantics (deferred, checks after all inputs are validated successfully):

  for out_index in range(n_outputs):
    if outputs[out_index].amount < out_min_amount[out_index]:
      return fail()

  # TODO: check that no output that is used with CCV_FLAG_DEDUCT_OUTPUT_AMOUNT
  # is also used with another OP_CHECKCONTRACTVERIFY, unless it's with CCV_FLAG_IGNORE_OUTPUT_AMOUNT

Use cases:

The following are some common combination of parameters.

Check that some data is embedded in the current input

This is used to check data that was typically committed to in an output from a covenant-encumbered spend that produced the current input.

<data=data> <index=-1> <pk=naked_pk> <taptree=-1> <flags=CCV_FLAG_CHECK_INPUT> CCV

Check that some other input with index in_i is a specific contract with embedded data:

This allows to "read" the data of another input.

<data=input_data> <index=in_i> <pk=input_i_naked_pk> <taptree=input_taptree><flags=CCV_FLAG_CHECK_INPUT> CCV

Check that a certain output with index out_i is a certain contract with specified data, preserving input amount

This might be used for a 1-to-1 or many-to-1 covenant-encumbered spend: one or several inputs are spent to an output with certain code and data.

<data=data> <index=out_i> <pk=output_naked_pk> <taptree=output_taptree> <flags=0> CCV 

Check that a certain output with index out_i is a a P2TR with a pubkey output_pk, preserving amount:

A simpler case where we just want the output to be a certain P2TR output, without any embedded data.

<data=<>> <index=out_i> <pk=output_pk> <taptree=<>> <flags=0> CCV

Check that a certain output with index out_i is a certain contract with specified data; don't check amount

This could be used to check data and program of an output, but not its amount (which might be either irrelevant, or is checked via a different introspection opcode.

<data=data> <index=out_i> <pk=output_naked_pk> <taptree=output_taptree> <flags=CCV_FLAG_IGNORE_OUTPUT_AMOUNT> CCV 

Check that the input is sent exactly to the same scriptPubKey

This requires that the output with the same index as the current input is exactly the same script, and with the same amount.

<data=<>> <index=-1> <pk=-1> <taptree=-1> <flags=0> CCV

Extensions

For the full generality of the MATT construction, the following opcode is required:

• OP_CAT (or some more restricted versions if preferred): allows commitment to multiple pieces of data, and checking Merkle proofs

Other opcodes that would simplify some constructions and/or make them more efficient (but optional for MATT):

• OP_CHECKTEMPLATEVERIFY: it makes some constructions a lot more efficients (for example vaults).
• OP_CHECKSIGFROMSTACK: it might be the easiest way to implement eltoo/ln-symmetry.
• Explicit input/output amount introspection, and 64-bit arithmetics.

[bigspider]

I edited the order of parameters above so that the data parameter is at the bottom of the arguments in the stack; the rationale is that all the other parameters are typically constant, while data is dynamically computed. Therefore, this should minimize the amount of data rearrangements needed.

[bigspider]

Not sure if anyone is reading here, but here's a complete implementation following the semantics above:

bitcoin-inquisition/bitcoin@24.0...bigspider:bitcoin-inquisition:checkcontractverify

I adopted @jamesob's framework for deferred checks from the OP_VAULT PR for a similar semantics on output amounts.

Tests are still quite basic, but it should be good enough to play with a relatively stable semantics.

[bigspider]

Changes in the branch (and in the specs drafted above):

• allow pk=-1 (a single byte 0x81) with the meaning of "use the current input's internal key.
• use taptree = -1 with the meaning of "use the current input's taptree"; previously it was 1. Rationale: using -1 harmonizes the behavior with the semantic meaning of -1 for index and pk, and it can still be pushed onto the stack with a single byte with OP_1NEGATE.
• allow index = -1 also for outputs (that is, when no CCV_FLAG_CHECK_INPUT flag is present), still with the meaning of it being replaced with the current input's index. Rationale: it simplifies the code, and it could be useful in situations where you want a UTXO to be entirely spent onto a specific output UTXO, but without pre-determining the output index. For example, this could allow batching multiple spends of such UTXOs in a single transaction (impossible if the output index 0 is hardcoded in the script).

Also did some general cleanup of the code, and added one more test. Tests are still insufficient.

[bigspider]

In case anyone still lands here, there are some further updates.

The semantics of the flags was updated as follows:

• CCV_FLAG_CHECK_INPUT is now -1; negative values refer to inputs (no value other than -1 is currently defined)
• CCV_FLAG_IGNORE_OUTPUT_AMOUNT has now value 1
• A new flag CCV_FLAG_DEDUCT_OUTPUT_AMOUNT with value 2 was introduced, in order to allow the value of the output to be "subtracted" from the current value of the input. Future calls (within the evaulation of the Script of the same input) to CCV will use the residual value for the current input.

CCV_FLAG_DEDUCT_OUTPUT_AMOUNT allows OP_CCV to be a drop-in replacement for OP_VAULT, as it allows to send some of an input amount back to the a certain script, or possibly the same as the input (with an OP_CCV with CCV_FLAG_DEDUCT_OUTPUT_AMOUNT), and then use OP_CCV with the default flag 0 to proceed with unvaulting the rest of the amount.

Together with a possible OP_IN_OUT_AMOUNT or any opcode that allows equality checks on the output amounts, CCV_FLAG_DEDUCT_OUTPUT_AMOUNT should also allow unilateral exits of 1 or more parties a CoinPool, while avoiding explicit 64-bit amount arithmetic inside the Script.

[reardencode]

Thanks for the responses on X today.

Might it be worth combining flags and index into a single data push of 1-3 bytes? I'd suggest making the top 2 bits represent the mode with mapping 0-default, 1-ignore_output_amount, 2-deduct_output_amount, 3-check_input; the next bit selecting between absolute and relative index, and the remaining 5-21 bits representing an unsigned integer for absolute and a signed integer for relative index.

In typical cases this would save 2 bytes relative to the current design, and it would allow an additional subcategory of contracts involving groups of outputs (e.g. 2-in, 2-out vault spends with partial revault) that could still be batched.

What this gives up is the upgradeability of flags, but we can leave behind an upgrade hook for top arguments longer than 3 bytes (and as I mentioned on X probably better to make this a CTV/CHECKSIG-style upgrade where the operation succeeds but does not short circuit execution with longer arguments). The longer argument would be 5 total witness bytes for the upgraded behavior, 1 more than used with a 1-byte index and 1-byte flags in the current design.

Now that I've taken the time to wrap my head fully(?) around CCV, I'm enthusiastic about it as an alternative to OP_VAULT that is more general. cc @jamesob

The one major hurdle I see in bringing CCV to bitcoin is gaining consensus on the idea that hashrate escrow type contracts are either an inevitability or an acceptable (or even desirable) function for bitcoin.

I hope to see this work formalized into a BIP or BIN soon :)

[bigspider]

@reardencode
Thanks for the comments and for checking CCV out!
The suggestions on the different approaches to save a few bytes are great, but for now I'm mostly focusing on simplicity and generality, so that demos can be built on top.

The things I'm building in pymatt are (mostly) agnostic to the exact implementation details, so having more code examples might help the future inevitable bike-shedding (or maybe byte-shedding...).

For the same reason, in the short term I prefer to focus on writing more code, rather than championing a BIP/BIN, which is on its own a quite substantial amount of work!

The one major hurdle I see in bringing CCV to bitcoin is gaining consensus on the idea that hashrate escrow type contracts are either an inevitability or an acceptable (or even desirable) function for bitcoin.

I mostly try to stay out of this discussion at this time (especially on Twitter...). But I'll briefly summarize my opinions, which I hope to find time to write in long form at some point.

My take is that (some) MEV is an inevitability, and drivechain-y things are likely enabled by the other covenant proposals as well if one thinks hard enough (how drivechainy is drivechainy enough is, of course, arguable ad infinitum).
Moreover, some equally bad things™ are possible today without covenants.

I think people conflate covenants/introspection opcodes with "adding new features"; but covenants don't add anything fundamentally new to bitcoin - they just make some things trustless when today they can only be built with not fully trustless solutions.
The conjecture that being "trustless" (versus other forms of "trust-minimized") makes any significant difference in terms of mining incentives is, in my opinion, flawed and not based in reality.

Finally, general covenants enable substantially better L2s (and more generally, better 2-way pegs) than drivechains. Therefore, they might even end up decreasing the chance of drivechains adoption, since their only selling point is quite literally the "missing features" of bitcoin.

[halseth]

I plan on prototyping CCV together with 64-bit math as well, it would be worth thinking about the amounts of the inputs/outputs as part of the arguments also.

[bigspider]

I plan on prototyping CCV together with 64-bit math as well, it would be worth thinking about the amounts of the inputs/outputs as part of the arguments also.

My hunch is that adding this would make CCV slightly more expensive for common use cases that are already covered by the current opcode, while the use cases where you need explicit amount checks tend to be uncooperative branches in smart contracts, for example your work on HTLC output aggregation, or situations of unilateral exit in coinpool-like constructions.

As they are uncooperative cases, I'm not sure the byte saving is significant enough to justify more complex/expensive opcodes for CCV, and my plan there would be to use the CCV_FLAG_IGNORE_OUTPUT_AMOUNT flag together with separate opcodes for explicit output amount introspection (and 64-bit arithmetic if needed, although I suspect equality checks already go a long way).

Anyway, looking forward to review your proposed specs!

[reardencode]

The conjecture that being "trustless" (versus other forms of "trust-minimized") makes any significant difference in terms of mining incentives is, in my opinion, flawed and not based in reality.

Yeah, this is exactly the question. I am in the camp that making those things trustless does change the mining incentives simply by making them more likely to be adopted, but I am not confident in that. The target audience for drivechainy things are seeming way more comfortable with depending on trust than I expect.

Thanks for your thoughts. I'm hoping to have a good long talk with some of the loudest folks concerned about this (cough Shinobi) at some point soon.

[reardencode]

I spent some time this morning trying to design LN-Symmetry contracts for CCV+CSFS, but found myself needing CAT+CTV as well to make it work out nicely.

One potentially interesting output of that work was that if CCV was instead CC which left on the stack 0 or <contract_hash> (e.g.hash_contract(||||||)`) it would be quite useful in combination with CSFS.

[bigspider]

I spent some time this morning trying to design LN-Symmetry contracts for CCV+CSFS, but found myself needing CAT+CTV as well to make it work out nicely.

I always assume CAT in the current experiments. CTV is always replaceable with CCV, although of course this comes with a byte cost for usecases where CTV is enough.

As this repo is mostly dormant at this point, I created an issue in pymatt repo: Merkleize/pymatt#9. Implementing lightning would be great indeed!

Also feel free to create other issues if you have any ideas, questions, etc.

One potentially interesting output of that work was that if CCV was instead CC which left on the stack 0 or <contract_hash> (e.g.hash_contract(||||||)`) it would be quite useful in combination with CSFS.

Interesting, I didn't consider that semantic, but it seems intuitively useful (also, a number of contracts I implemented do indeed need a final OP_TRUE that would be avoided).

What is the use case you have in mind?