CIP-0172? | Self-Usable Transaction Outputs

[carlostome]

Allow transactions to use scripts and data from their own outputs during validation.

(rendered proposal)

Notes:

• I've left a TODO in the Implementation plan, since there seem to be two viable alternatives: either implement it in Conway (via a hardfork) or make it part of the upcoming Dijkstra era.

[rphair]

@carlostome I'm marking this Triage: to introduce at the CIP meeting in about 12 hours' time (https://hackmd.io/@cip-editors/126) if we can... though I would feel strange rushing to review this without a Ledger reviewer (cc @lehins @WhatisRT) first taking a quick look to confirm this is possible: maybe also confirming that it's consistent with other Ledger design goals.

Also @colll78 @polinavino @fallen-icarus especially from your interaction with the Nested Transactions CIP I think we would benefit from your insights about this proposal: including views on use cases, efficiency, safety, and other factors that have come up before. Personally I can't yet understand how this proposal and CIP-0118 are intertwined & I hope to get a better view of that through upcoming review.

[fallen-icarus]

@carlostome Do you know of any concrete use cases for this? As a dApp developer, I've never found myself wishing for this ability.

[carlostome]

@carlostome Do you know of any concrete use cases for this? As a dApp developer, I've never found myself wishing for this ability.

Hi @fallen-icarus, in the CIP, I have outlined the use case I have in mind. That is, making sharing scripts between top- and sub- level transactions (in the context of Nested transactions) independent on the order in which they are processed within a batch. I'm happy to add more explanation and/or clarifications if you find it is not clear enough.

I do not know of other use cases at the moment.

[fallen-icarus]

... making sharing scripts between top- and sub- level transactions (in the context of Nested transactions) independent on the order in which they are processed within a batch.

This is an abstract use case. What real world product needs this ability? Do batchers need this for aggregating sub-txs? Why wouldn't they just have the sub-txs omit the required script/data from the sub-tx so they can just add it to the top-level?

[rphair]

summary notes from today's CIP meeting:

➤ Use cases: as noted above, this question also came up at the meeting. My understanding is that this demand has already been met: when we combine @fallen-icarus's enquiry about use cases with @carlostome's response that "Nested Transactions" is itself the use case, it appears that "use cases of Nested Transactions" are also generally use cases of this proposal.

• @carlostome therefore, since that statement is already in your CIP document at this time, if possible please further qualify that set of use cases by confirming which particular patterns of nested transactions would benefit (most) from this proposal.

➤ Thanks mainly to @colll78, we highlighted a substantial benefit for this proposal (by avoiding an "ordering dependency") in that protocol creators wouldn't have to wait for the effects of something like a token minting to be processed by the chain in a previous transaction (whose Tx would be far too big to include as another Tx input) before submitting further transactions to define that protocol.

• @carlostome - likewise, please consider adding this scenario (waiting for large transactions to complete first, if I've understood correctly) to your Motivation if you think it's relevant.

My own list of review questions that came up are:

• Can making such a change — with any implied requirements on whether grouped transactions are "ordered" or "atomic" — break anything currently under development that would take the existing Ledger rules for granted?
• Would implementing this have any performance implications for the Ledger and/or the efficiency of regular or Nested transactions?
• @lehins @WhatisRT what about the "complexity cost" of adding this? Even if feasible to add (as asked already in this preliminary review, and as @colll78 also enquired, would it be worth the technical burden to accommodate this proposal?

Mainly pending Ledger response about feasibility we chose to leave this Unconfirmed until at least the feasibility & acceptable cost of a Ledger implementation can be established.

[carlostome]

What real world product needs this ability?

I don't know.

Do batchers need this for aggregating sub-txs?

This allows batchers to reduce duplication (and thus fees) in the case of cyclic dependencies between data and/or scripts in the top- and sub-level transactions.

Why wouldn't they just have the sub-txs omit the required script/data from the sub-tx so they can just add it to the top-level?

Because batchers cannot modify sub-txs since they are signed. Scripts are not signed but data is (via scriptIntegrityHash; see the Alonzo spec).

[carlostome]

@carlostome therefore, since that statement is already in your CIP document at this time, if possible you should further qualify that set of use cases by confirming which particular patterns of nested transactions would benefit (most) from this proposal.

I've changed the CIP to reflect the particular nature of the use case within nested transactions.

My own list of review questions that came up are:

Can making such a change — with any implied requirements on whether grouped transactions are "ordered" or "atomic" — break anything currently under development that would take the existing Ledger rules for granted?

I'm not sure what the scope of anything currently under development is, but a priori yes, in the sense that this change modifies the behaviour of some part of the Ledger rules.

Would implementing this have any performance implications for the Ledger and/or the efficiency of regular or Nested transactions?

I can't foresee any implications in terms of performance and/or efficiency. However, this change will definitely simplify the implementation of data and script sharing in Nested transactions.

[carlostome]

➤ Thanks mainly to @colll78, we highlighted a substantial benefit for this proposal (by avoiding an "ordering dependency") in that protocol creators wouldn't have to wait for the effects of something like a token minting to be processed by the chain in a previous transaction (whose Tx would be far too big to include as another Tx input) before submitting further transactions to define that protocol.

@carlostome - likewise, please consider adding this scenario (waiting for large transactions to complete first, if I've understood correctly) to your Motivation if you think it's relevant.

I don't fully understand this scenario and how it could benefit from script sharing. In particular I don't understand the part about submitting further transactions to define the protocol after minting the token.

[WhatisRT]

what about the "complexity cost" of adding this? Even if feasible to add (as asked already in #1135 (comment), and as @colll78 also enquired, would it be worth the technical burden to accommodate this proposal?

This feature was originally motivated by a simplification of the logic of the specification. So any implementation that implements the logic in a similar manner should also see a simplification in their code. Roughly speaking, checking that everything is ordered correctly is extra work, and this CIP recognises that this extra work is unnecessary and not doing it gives some extra flexibility to DApp authors.

[rphair]

Can making such a change... break anything currently under development that would take the existing Ledger rules for granted?

I'm not sure what the scope of anything currently under development is, but a priori yes, in the sense that this change modifies the behaviour of some part of the Ledger rules.

apologies @carlostome: I didn't mean to introduce a potentially infinite scope. 😅 What I meant more practically was (if you might know of some answers): What common design patterns in Cardano development would depend upon the unmodified Ledger rules and therefore be affected by this CIP?

The editors (among others) would just be trying to estimate the trade impact of making this change, and so anything you (or other reviewers) can provide here would be helpful: especially if you add these scenarios to the CIP itself.

I don't fully understand this scenario and how it could benefit from script sharing. In particular I don't understand the part about submitting further transactions to define the protocol after minting the token.

@colll78 can you please document here this example as you described it at yesterday's CIP meeting — a typical project launch, beginning with a token mint, with this CIP avoiding waiting periods for prior transactions to be verified before continuing to each new step — which was considered an advantage and use case for this CIP? I think we need your precision here so the author knows what to do with that suggestion & perhaps include it as a use case.

---

@carlostome please note that my writing above is mainly feedback from a CIP meeting yesterday (held biweekly) in which I put your proposal on the agenda for introduction. We can't rely on those present at the meeting to record their meeting statements back on GitHub... so during or after the meetings one of the CIP editors will generally try to document it back here. We do our best to record these statements but often can't perfectly reproduce the same point of view or technical understanding. You'd be very welcome at the meeting yourself & please find the schedule here: https://luma.com/cardano-cip

[lehins]

I believe it should have been implemented this way right from the start when reference scripts where introduced as a feature, because I can't see a single reason why it shouldn't have been done that way, especially since implementation for this is very simple. Luckily we should be able to retroactively enable this feature for PlutusV1-V3 scripts as well, without much effort, since witnesses are not supplied in the script context, so scripts cannot know for sure whether script was provided as a witness or as a reference script.

In my opinion, this CIP is simply an optimization. All it does is allows a user to use a script in the same transaction that adds it to the UTxO for sub-subsequent transaction to reuse as reference script. In a common case, it does not really make something that was previously impossible possible, except when a script that a person is trying to use is pretty big in size. It only makes some certain use cases cheaper, simpler for users and reduce amount of data placed on chain. This point is quite simple to prove: any transaction that could benefit out of using a script that was supplied in one of the outputs that it is creating, could always also add it to the witness set. So, all this CIP will allow is to do is accommodate those cases without including such script in a transaction twice, which is also why scripts that are so large in size that two copies of it would not fit in a transaction, could really benefit from this CIP.

Some benefits from this CIP that come to mind:

• From @colll78 the example of minting a token, at least as far as I imagine it, maybe @colll78 has a different use case in mind. Today in order to be able mint and burn a token, while using a reference script you require at least 3 transactions: first transaction adds a script to UTxO, while subsequent two that mint and burn use that script. With this CIP the same transaction that does the minting also could add the script to UTxO for the transaction that does the burning to use. So less fees and less contention for ephemeral tokens.
• A similar use case as for token minting will be applicable for CIP-160 - Protected Adddresses. Because then the script that needs to be executed for output creation can contain that said script, so later when it is being spent there will no longer going to be a need to even specify where the script is coming from
• Simplifies usage of reference scripts in Nested Transactions. Also by its nature Nested Transactions will have many transactions in it with a size limit of a single transaction, so anything we can do to reduce binary overhead of a transaction will be beneficial for Nested Transaction especially
• Unlike scripts that are supplied through reference inputs these scripts provided by the output do not carry the same dangers of not being priced by the regular fee calculatoin. Therefore, those scripts will not need to participate in the same costing function as reference scripts, since they will already be priced through the size of transaction, similarly to how scripts that are used through a witnesses are priced.

Best thing I like about this CIP is that implementation of this feature is very simple and, as far as I can see it, there are only benefits and no drawbacks.

Big 👍 from me.

[lehins]

This is an abstract use case. What real world product needs this ability? Do batchers need this for aggregating sub-txs? Why wouldn't they just have the sub-txs omit the required script/data from the sub-tx so they can just add it to the top-level?

I can try and make it concrete. Here is an example without this CIP:

• Some sub-transaction creators start requiring the same script in topLevelRequiredGuards, while supplying said script in the witness set, or not supplying it at all. (It can be any script type, it doesn't have to be a guard script, I've used guards for the sake of example)
• Most likely aggregator (batch creator) has two priorities in mind: quickest transaction acceptance on chain and lowering the cost. Without this CIP however, aggregator can only optimize for one of those two:
  1. include that script in a witness set to be able to use it right away and submit another transaction later (or if the script fits also create an output int the same transaction) adding the script for further reuse.
  2. create another transaction, wait for it to be accepted on chain and then start using that script.

With this CIP, top-level transaction builder will have this ability to add a script to UTxO that any of the sub-transactions might need for immediate and sub-subsequent use. So, IMHO, this CIP is especially useful for the top-level transaction builder.

[rphair]

@carlostome @lehins the elaboration of Ledger impact in #1135 (comment) abundantly addresses the only blocker we had at our last CIP meeting — uncertainty over whether this could be implemented easily enough in the Ledger — and so I'd like to promptly confirm this as a CIP candidate: as we would have if we'd been sure of this on Tuesday's meeting.

I've added this back on Review for the next CIP meeting (https://hackmd.io/@cip-editors/127), where we could do that: but if both @Ryun1 and @perturbing agree here in the meantime we'd have unanimity & can re-tag this as Confirmed, assign a CIP number, and start tagging other devs for a deep (technical) and broad (ecosystem) review.

[Ryun1]

@rphair
happy to have a CIP number applied 💪

[colll78]

➤ Thanks mainly to @colll78, we highlighted a substantial benefit for this proposal (by avoiding an "ordering dependency") in that protocol creators wouldn't have to wait for the effects of something like a token minting to be processed by the chain in a previous transaction (whose Tx would be far too big to include as another Tx input) before submitting further transactions to define that protocol.
@carlostome - likewise, please consider adding this scenario (waiting for large transactions to complete first, if I've understood correctly) to your Motivation if you think it's relevant.

I don't fully understand this scenario and how it could benefit from script sharing. In particular I don't understand the part about submitting further transactions to define the protocol after minting the token.

Currently to deploy just about any DeFi protocol or dApp on Cardano requires multiple transactions, even for an extremely simple dApp that only has a single script.

The minimal flow for a stateful dApp at the moment is roughly:
Transaction 1: Deploys reference script UTxOs
Transaction 2: Include the reference script UTxOs that were produced in the previous transaction as reference inputs, and use them to mint the relevant state tokens / dynamic parameters UTxO NFT.

With this proposal you could deploy such applications in a single transaction that produces the reference script UTxOs (ie. that contain the minting policy of the state tokens) and mint the state tokens in the same transaction (ie. create a DEX pool in the same tx where you publish the DEX pool script).

[rphair]

Since all editors who left this Unconfirmed at last week's CIP meeting have now supported making this a candidate, we're promoting it without waiting for the next meeting. @carlostome please rename the containing directory to CIP-0172 and updated the "rendered" link in your original comment. 🎉

Also please consider adding those couple of things to the Motivation and Rationale to resolve the comments in #1135 (comment) & #1135 (comment) based on @Ryun1's & other editorial feedback.

Though this proposal is of general developer interest and we would welcome comments from anyone, I'm tagging some further reviewers with a good eye for overall design patterns — especially contributors to the Nested Transactions discussion & roundtable — to review the CIP and maybe help us plan how to introduce this to the dev community:

@perturbing @Quantumplation @polinavino @ch1bo @sandtreader @AndrewWestberg @michele-nuzzi @disassembler @MicroProofs

[rphair]

@carlostome thanks for all your updates in response to previous reviews which I can see have addressed all the commentary from editor's review and our last meeting discussion. Thanks especially for your force-push being done in a way that preserves the commit history from different sources. 🙏

Editors will keep an eye out for further technical review as hopefully launched with those tagged in #1135 (review), and if no objection or further evolution will keep moving it forward in the CIP review cycle towards merge.

Once the suggestions below settle, plus hopefully some confirming review, I'd be ready to approve this CIP.

[rphair]

Suggested change

	### Acceptance Criteria
	- Fully implemented in Cardano node
	### Implementation Plan
	- TODO
	### Acceptance Criteria
	- [ ] Fully implemented in Cardano node
	### Implementation Plan
	- [ ] Update the formal Ledger specification with "changes to the Ledger logic" above
	- [ ] Implement the outlined changes in the Cardano node
	- [ ] Complete a hard fork enabling support for the changes outlined here

I used https://github.com/carlostome/CIPs/tree/master/CIP-0118#implementation-plan as a template:

1. @Ryun1 @lehins can please double check for consistency with similar Ledger proposals?
2. @carlostome & others please add anything in developer relations that could be seen as a dependency (a common issue with Ledger changes so perhaps implicit & therefore no need to publicise it?)

[rphair]

p.s. I can also see (somehow missed before) in the original comment the current uncertainty about implementation in Conway vs. Dijkstra... understandably this would still be TODO while that decision is made.

[ch1bo]

Gave a look because I got notified through Github and I could not really follow the motivation for this.

[ch1bo]

Why is a reference script at all needed in this use case?

This exact annoyance could guide a dApp developer to not use reference scripts, but just include the script in the witness set of the transaction that mints the tokens. Thus, only incurring a transaction fee and not require the additional deposit for the reference script in the output (which grows the utxo state, that must be paid).

[lehins]

Comparing to just including the witness set, adding a script to the UTxO will require the user to lock up the funds in the form of minUtxO, so that is the motivating factor against it. Which means the only reason why a user would want to do this instead of just using a script in witnesses is for further reuse.

So, this statement is incorrect: "not require the additional deposit for the reference script" There is still deposit for including a script in the output.

[colll78]

The point is that the script that is used to deploy the initial state tokens will of-course be reused. IE. A DEX pool script (minting policy and spending validator multi-validator). Right now you need to deploy reference scripts, and create the pool in two separate txs.

You could just mint the tokens providing pool script as witness and then deploy reference scripts after but that's still two txs.

[ch1bo]

Then, it was just not clear to me as a reader that this example is not an isolated usage, but the optimization of combined publishing scripts as reference + the first usage in the same transaction. The example was helpful and I suggest it should be added to the CIP / existing examples be made more concrete.

[rphair]

@carlostome I think adding such an example would help make the proposal more "user friendly" - which has been important for other CIPs to keep people aware of best-practice design patterns.

[ch1bo]

I don't follow the argument of re-using inlined data.

If a transaction wants to spend an output that carries inlined data, that data would already be in the utxo set and thus the creating tx paid for it. For "normal" datum use, there would only be a datumhash on the utxo and the spending transaction provides the preimage in the witness set. This data resolves across any spent inputs that are parameterized by the same data (hash). I don't know how this evolves for nested transactions but I would expect it to allow re-use of hashed data. For other validation purposes, there is no datum in the context.

So, what exactly motivates the sketched change of getDatum in the agda spec?

[lehins]

I do agree with this point, since datums are not reusable from reference inputs either. This change alone would lead to strange semantic where datum from outputs created can be reused throughout a transaction, while ones specified in inputs and reference inputs cannot. If we are to make such a change it would require its own CIP. However, I vaguely remember discussing this point long time ago with some folks that actually write DApps and their argument against was that it is very rare that a script requires the same data and it is usually fairly small anyways.
I think it would be worthwhile analyzing the chain and see whether there could actually be some space savings if we were to add such feature of inline datums being shared throughout a transaction.