Draft NEP for pending transaction queue.

[robin-near]

In the near future of the Near blockchain, we foresee that via the SPICE project, transaction and receipt execution will become decoupled from the blockchain itself; they will no longer run in lockstep. Instead, transactions will be included in the blocks first, and then execution will follow later.

This inherently introduces a problem that we must accept transactions before we know whether they are valid. Today, when a chunk producer produces a chunk containing a transaction, it can verify using the current shard state that the transaction has a valid signature, has enough balance, and a valid nonce. But as execution becomes asynchronous, we no longer have the current shard state to verify the
transactions against.

This NEP proposes a mechanism called the Pending Transaction Queue to solve this problem.

[birchmd]

Thanks for bringing this up! It is good to think about how to prevent possible DoS attacks under SPICE.

[birchmd]

Is this enough to prevent the attack described above? Instead of having one account they send many transactions to, it could be many accounts they send only a few transactions to each.

[robin-near]

The idea is that the amount of block space the attacker consumes vs amount of gas the attacker has to pay, i.e. the "waste amplification factor", is unbounded in the attack mentioned in the NEP, but only O(1) with a maximum parallelism. It's then still possible to attack, but the attack would cost a proportional amount of gas - just cheaper than it is today - which isn't a very great incentive to mount the attack (given that the attack is only a DoS).

[birchmd]

To check the constraints hold, it sounds like this means chunk producers need to include a state witness to validators along with the proposed chunk, just like they do today. Does this work against the idea of SPICE separating the consensus and execution? I suppose maybe if this state witness is very small relative to today then SPICE could still be effective.

[robin-near]

Ah, good point. We are only addressing the attack surface from (unauthenticated) users, not attacks from chunk producers. Chunk producers today (and in SPICE) are high-stake entities because they are also block producers, and mounting this attack as a chunk producer is not even very effective because you can only affect the chunks you produce. It is also already possible today to simply produce a chunk that is empty.

[gagdiez]

Hi @robin-near (or anyone who is interested in championing this NEP forward) – we are cleaning the NEP backlog and noticed that this PR still has some incomplete sections: particularly, it has a "TODO: Other things". Therefore, we are labeling this PR as "Needs author revision."

Please ping the @near/nep-moderators once you are ready for us to review it. We typically close NEPs that are inactive for more than two months, so please let us know if you need more time.

[garikbesson]

Because there has been no activity from the author for more than 3 months, I'm marking the proposal as RETRACTED. Please feel free to open it again if you'd like to continue working on it.

[nagisa]

This is mildly problematic for scaling/perf reasons. We're spending a humongous amount of time accessing the account + accesskey during transaction validation in tx runtime in some workloads already. This is adding another walk through a trie.

OTOH if we make nonces array be inline the GasKey struct, then every time we load/deserialize the GasKey we have to read out and deserialize all them nonces.

---

I see that this multiple-nonce mechanism is meant to enable something outside of this NEP's primary motivation. This is okay, but we should introduce proper motivation for adding this feature in the NEP text.

I also wouldn't lock in a specific implementation here for now. I suspect that with MAX_NONCES scaled down to a smaller number (e.g. 16) it might become feasible to keep GasKey as a monolithic type.

Another reason to keep MAX_NONCES low initially is that it is trivial to increase the limit in the future, but decreasing it would be effectively impossible.

[darioush]

Using gas keys would add one additional trie access (to load the nonce from a separate key).

However I am not sure it is a major performance concern as gas keys are only needed by accounts with contracts that wish to have more than 4 in-flight transactions (or by users that issue parallel transactions using multiple nonces).

There is a proof size tradeoff as well: if vector approach is used, value of all nonces will be included in the proof vs. just the accessed nonce.

I agree about keeping MAX_NONCES low initially.

[darioush]

@robin-near can you please weigh in on the choice about making each nonce its own trie key?

[darioush]

Actually it seems with some of the recent caching efforts we can cache fetching the "main part" of the GasKey (with balance, permissions etc). so an additional trie access only happens only for the first access of each GasKey (then accessing other nonce indexes of the same gas key become cheaper for the same chunk).

[nagisa]

then accessing other nonce indexes of the same gas key become cheaper for the same chunk

I'm not sure that holds true with memtries at least. Accesses to memtrie don't really retain any memory of the prior accesses and so any new access will start walking from the trie root until it gets to the value.

Conceptually, though, this could be possible to make optimal by changing the memtrie. It would just need a much more involved API that allowed you to pass in a node from previous lookup to start walking from and only use the tail of the key, rather than the entire key.

[jakmeier]

Note that #522 is also open and might have a different solution to parallelization. But I'm not sure if it is compatible with the exact limit we want to enforce here.

[darioush]

This is an interesting proposal, and indeed a valid approach to dealing with duplicate transactions.
I don't agree with the part about writing state into the trie (this seems very expensive, to account for transactions in recent blocks)

This means the nodes would have to re-compute the new "recently accepted transactions" data structure. In spice we are trying to avoid this type of dependency on past chunks / blocks.

[nagisa]

I think there is a possible middle ground that does not make gas keys be this risky.

Indeed, if this were to transfer the balance out of the gas key, we'd have to prevent the issue described in the motivation section (with send_near replaced with gas key deletion.) Even in that context a middle ground is still achievable by e.g. treating this action specially and:

1. Whenever we put a transaction containing this action into the pending transaction queue, reserving all the remaining balance inside of it;
2. Not accepting any further transactions (cause the chunk producers see 0/negative remaining balance due to the reservation above.)
3. (It might be necessary to make this action only valid as the only action of the transaction, i.e. not possible to construct via promises.)

[darioush]

There is a compounding effect here with deleting the account (which can be done programmatically).

If we do a refund on delete for GasKey then we need to do something like not allowing accounts with GasKey to be deleted.

[nagisa]

Hm, fair enough. It could be quite nasty if an account deletion with a boatload of gas keys required a bunch of book-keeping on the chunk producer side as well.

I think it would be a fair requirement to impose that if you want to delete an account you have to get rid of all gas keys first and use the access key to sign the action. We already have a limit on the amount of storage for the contract already, this wouldn't be an unusual requirement.

[darioush]

I also agree with the alternative of forbidding a deletion of accounts with gas keys.

This is because if we try to add some protection from deleting high balances, then we have to check an unspecified number of gas keys to do 1 account delete, which makes charging for it difficult.

This makes it such that the user has to first delete each of the gas keys and pay for the nonce deletion accordingly, in a separate action.

[matejpavlovic]

As I understand it, the requirement for burning the remaining balance stems from the fact that a gas key (or the whole account) can be deleted through an access key, i.e. not involving a transaction signed by the gas key. Is this the case? Even if we disallow deleting accounts with gas keys?

If yes, would it make sense (or even be possible) to enforce the same restrictions for gas key deletions as we do for transfers? Putting it differently, can we treat a gas key deletion the same way as we treat a withdrawal, with the exception that the gas key would be deleted afterwards? The NEP later says that " contract execution cannot create receipts that withdraw from gas keys; only gas key transactions can". Is there a fundamental reason to restrict this to withdrawals?

[darioush]

i.e. not involving a transaction signed by the gas key. Is this the case? Even if we disallow deleting accounts with gas keys?

This is correct. Notably, account deletion can also occur programmatically (as a result of contract execution).

If yes, would it make sense (or even be possible) to enforce the same restrictions for gas key deletions as we do for transfers?

We could force the same restriction, i.e., disallow programmatic deletion of gas keys. However, contracts may benefit from the flexibility of programmatically adding/deleting gas keys.

[nagisa]

If we decide to have separate trie keys for each nonce, then each of those trie keys should be charged for as its own write operation, rather than only for the byte-length of the conceptual vector of nonces.

[darioush]

I intended to charge each trie key as its own operation, by charging per nonce: storage_write_base + appropriate key and value size multiplied by storage_write_key_byte and storage_write_value_byte respectively.

For the value of each nonce, len(Nonce) seems the correct amount to charge (Nonce = u64 here).
For the key length it seems more appropriate to charge for the common prefix (public_key) part only once, which is accounted for already in the fee that is similar to adding an access key.

Are you suggesting to charge the common key prefix also num_nonces times?

[nagisa]

Yes, I believe the key fees should be charged for the common key prefix as accessing/writing to each key is going to (currently) walk the memtrie from the root each time. Doing anything more efficient would require a fancy API to allow looking up nodes rather than values and doing lookups starting at arbitrary provided nodes.

This is a good API to add regardless for many reasons (e.g. isolation of contract data accesses,) but right now it does not exist AFAIK.

[darioush]

Updating this to account for key len corresponding to the trie key len.

[nagisa]

More specifically the current contract-host interface now has a hard assumption that refunds can (only) be specified to go to an AccountId, which strongly twists our hand towards only ever refunding to AccountId. There are some issues that stem largely from the fact that its the caller's decision whether to use an AccessKey or a GasKey and contracts (especially contracts that are "locked") can't upgrade to code that's GasKey aware.

So the solution space starts looking like:

1. If we force the refund provenance (i.e. refunds are only possible to the same type of balance from which it was charged from) and an incompatible contract calls refund_to, we panic for calls that use GasKey?
2. If we do not force the refund provenance (i.e. refunds can go to either account or gas key) then we still force the "inconvenient" behaviour of having to recharge the gas key more frequently than otherwise required;
3. If we force refunds to go to the account ID, the contract-host interface does not change in any way and gas keys are transparent and a strictly additive feature from the contract's standpoint. I think both of these properties are quite high priority if possible.

---

I'd also say there is a very good reason to force frequent gas key recharges anyway. Similar to how forcing short-lived certificates force infra engineers to stay on their toes and "simply" set up infrastructure to automatically renew certificates, gas keys refunding to an account unconditionally would force infra engineers for the contract to do the same. That way they won't run out of balance in 10 years unexpectedly breaking their production and long after everybody involved have left the company :)

[darioush]

I am tending to agree with this because of the above complications and the user will need some mechanism to top up their balance periodically regardless.
Also it will discourage users from attaching too much gas to transactions.

[jakmeier]

Throwing in another idea: Why not make gas keys just another type of access key and simply refer to them by public key?

Today, gas refunds of transactions created with a function access keys already restore the balance allowance on the corresponding key. (Unless it has been deleted while the transaction's receipts DAG was executing.) This works by recognizing a refund receipt by its sender being "system" plus checking that the transaction signer (always preserved across all receipts of a transaction) is a function access key.

See here:
https://github.com/near/nearcore/blob/be8d8cc7acf716c3c084d1a0fe323109aa32d478/runtime/runtime/src/lib.rs#L2481-L2482

It should be possible to use the same mechanism to detect gas refunds from a transaction that was initiated by a gas key. But only if access keys and gas keys share the same namespace. I think they should.

[darioush]

Note: this approach may help avoid require separate Add/Remove actions

[walnut-the-cat]

Thank you @darioush for re-submitting this NEP.

As a moderator, I reviewed this NEP and it contains enough information to initiate SME review process based on the proposed template guidelines, therefore I am moving this NEP to the REVIEW stage.

I would like to nominate @nagisa and @matejpavlovic as SMEs for this NEP to complete a technical review (see expectations below).

Just for clarity, Technical Reviewers play a crucial role in scaling NEAR ecosystem as they provide their in-depth expertise in the niche topic while work group members can stay on guard of the NEAR ecosystem. The discussions may get too deep and it would be inefficient for each WG member to dive into every single comment, so NEAR Developer Governance designed this process that includes subject matter experts helping us to scale by writing a summary with the raised concerns and how they were addressed.

Technical Review Guidelines

• First, review the proposal within one week. If you have any suggestions that could be fixed, leave them as comments to the author. It may take a couple of iterations to resolve any open comments.

• Second, once all the suggestions are addressed, produce a Technical Summary, which helps the working group members make a weighted decision faster. Without the summary, the working group will have to read the whole discussion and potentially miss some details.

Technical Summary guidelines:

• A recommendation for the working group if the NEP is ready for voting (it could be approving or rejecting recommendation). Please note that this is the reviewer's personal recommendation.

• A summary of benefits that surfaced in previous discussions. This should include a concise list of all the benefits that others raised, not just the ones that the reviewer personally agrees with.

• A summary of concerns or blockers, along with their current status and resolution. Again, this should reflect the collective view of all commenters, not just the reviewer's perspective.

Here is a nice example and a template for your convenience:

### Recommendation

Add recommendation

### Benefits

* Benefit

* Benefit

### Concerns

| # | Concern | Resolution | Status |

| - | - | - | - |

| 1 | Concern | Resolution | Status |

| 2 | Concern | Resolution | Status |

Please tag the @near/nep-moderators once you are done, so we can move this NEP to the voting stage. Thanks again.

[matejpavlovic]

Well written. I added some comments inline. Regarding performance issues and the section on nonces, I'm not able to make properly informed opinion - my knowledge of the implementation is unfortunately not deep enough to meaningfully contribute to the discussion.

[matejpavlovic]

I assume that by "costs" of transactions we mean gas limits, right? Since the tXs are not yet executed, we don't know their actual costs.

[darioush]

Yes good point. Here the costs refer to the sum of near and gas attached (ie, maximum cost) of the transaction.

It is possible that not all the gas would be used in execution, and that the user would receive a refund later.

Adding some clarification for this.

[matejpavlovic]

"The notion of pending transactions is anchored at a specific chunk that is being produced." Agree.

The tricky part is that the chunk producer is required to know about all the transactions included in the previous chunks all the way up to the chunk that it is producing. I.e., producing a chunk for height h requires the knowledge of all transactions associated with heights until h-1. (We're considering a single shard.)

In SPICE we count on decoupling (from consensus) not just the execution, but data availability (even though we are not implementing it in SPICE v1). That is, making transaction data available will also be asynchronous. A chunk producer with the right of producing a chunk associated with height h might actually have a window of a few blocks after h to make sure the data is available, during which the data owners will certify (on chain) having received their respective data parts. This means that the availability certificate for a chunk associated with height h might only occur in a block at height h+a (a being the number of blocks it takes to ensure the availability). The next chunk producer (producing a chunk for h+1), however, might not yet know the contents of the chunk associated with h, and thus not have a precise view of the pending transaction set. This is a more general problem that also has other implications (such the problem with transaction duplication), but an incomplete view of the pending transaction set is also an issue.

I could imagine a solution that requires the chunk producer for h to post at least a commitment to the chunk on chain as soon as possible, and the pending transaction set at h+1 would be defined through these commitments. The chunk producer for h+1 would then need to obtain the transaction data associated with h before proposing its own chunk. Ideally the chunk producer at h would directly (with high priority) send its chunk to the producer at h+1. If the former does not send the chunk, the latter would need to retrieve the chunk from the data owners, slowing down the chunk production.

[matejpavlovic]

As I understand it, the requirement for burning the remaining balance stems from the fact that a gas key (or the whole account) can be deleted through an access key, i.e. not involving a transaction signed by the gas key. Is this the case? Even if we disallow deleting accounts with gas keys?

If yes, would it make sense (or even be possible) to enforce the same restrictions for gas key deletions as we do for transfers? Putting it differently, can we treat a gas key deletion the same way as we treat a withdrawal, with the exception that the gas key would be deleted afterwards? The NEP later says that " contract execution cannot create receipts that withdraw from gas keys; only gas key transactions can". Is there a fundamental reason to restrict this to withdrawals?

[matejpavlovic]

Would it make sense to consider an additional "force" flag in the DeleteGasKey action to give the user the possibility to override this check?

[darioush]

Adding a note for this as an implementation consideration.

[jakmeier]

This is a high quality proposal, thanks to everyone who brought it so far!

I left an idea to potentially simplify the refund story in the comments.

[jakmeier]

To be clear, I assume the same would be enforced for "deploy-like" actions, right?

UseGlobalContract, DeterministicStateInit, and Delegate with inner deploy-like actions all should have the same restrictions, I think.

[darioush]

I agree, how can we find the full list of these actions? (are there any others?)

[jakmeier]

As of now, that should be the full list. Any potential future action that can also set the contract field of an account should be included, though.

[jakmeier]

Throwing in another idea: Why not make gas keys just another type of access key and simply refer to them by public key?

Today, gas refunds of transactions created with a function access keys already restore the balance allowance on the corresponding key. (Unless it has been deleted while the transaction's receipts DAG was executing.) This works by recognizing a refund receipt by its sender being "system" plus checking that the transaction signer (always preserved across all receipts of a transaction) is a function access key.

See here:
https://github.com/near/nearcore/blob/be8d8cc7acf716c3c084d1a0fe323109aa32d478/runtime/runtime/src/lib.rs#L2481-L2482

It should be possible to use the same mechanism to detect gas refunds from a transaction that was initiated by a gas key. But only if access keys and gas keys share the same namespace. I think they should.

[jakmeier]

As I wrote in another comment to make refunds easier, can we consider another representation?

Specifically, if we can store gas keys the same way we store the two existing access key types, then this would let them share the same namespace. Which makes it possible to detect refunds that should go to a gas key by using the public key.

So I would suggest something more like this:

pub enum AccessKeyPermission {
    FunctionCall(FunctionCallPermission),

    /// Grants full access to the account.
    /// NOTE: It's used to replace account-level public keys.
    FullAccess,

+   // Same access as FullAccess but with separate gas accounting
+   ConceptualGasKey,
}

struct ConceptualGasKey {
-   public_key: PublicKey,
    nonces: Vec<Nonce>,
    balance: Balance,
-   permission: AccessKeyPermission,
}

This would be slightly less flexible, as it doesn't allow to limit methods callable by the gas key. We could add a FunctionCallGasKey as another variant if that's needed.

Of course, this has pretty large ramifications for technical code details. We might not even need AddGasKey and DeleteGasKey actions and instead just reuse AddAccessKey and RemoveAccessKey.

Maybe this is going too far off the intended design. But from my point of view, this seems to be an alternative that simplifies a bunch of things by staying closer to the existing concepts of access key types.

[jakmeier]

Importantly, this cost is knowable by just inspecting the actions contained in the transaction and does not depend on state or execution.

It does depend on the gas price though, doesn't it? The gas price is part of the block, so not state. I'm not very familiar with SPICE. Is the block header used for execution already known when we do these checks?

[darioush]

Yes, that's right, it depends on gas price. I will clarify that it is needed as well.

For Spice, we have not yet figured out the details of propagating the gas price. As execution would be a few chunks behind, we could use the latest executed chunk as a basis for the gas price of the next chunk. There is likely some nuanced tradeoffs which we have to consider.

Notably, this proposal doesn't require spice. For this NEP, we can assume the chunk producer would have a way to know the appropriate gas price, like it does currently.