Draft NEP for pending transaction queue.

neps/nep-0611.md

@@ -0,0 +1,233 @@
+---
+NEP: 611
+Title: Pending Transaction Queue
+Authors: Robin Cheng <[email protected]>
+Status: Draft
+DiscussionsTo: https://github.com/near/NEPs/pull/611
+Type: Protocol
+Version: 1.0.0
+Created: 2025-05-28
+LastUpdated: 2025-05-28
+---
+
+## Summary
+In the near future of the Near blockchain, we foresee that via the SPICE project, transaction and receipt
+execution will become become decoupled from the blockchain itself; they will no longer run in lockstep.
+Instead, transactions will be included in the blocks first, and then execution will follow later.
+
+This inherently introduces a problem that we must accept transactions before we know whether they are
+valid. Today, when a chunk producer produces a chunk containing a transaction, it can verify using the
+current shard state that the transaction has a valid signature, has enough balance, and a valid nonce.
+But as execution becomes asynchronous, we no longer have the current shard state to verify the
+transactions against.
+
+This NEP proposes a mechanism called the Pending Transaction Queue to solve this problem.
+
+## Motivation
+
+### Why is this worth solving?
+
+A potential for DoS attacks exists whenever the blockchain allows anyone to submit work without paying.
+Invalid transactions present such a vulnerability: if a transaction is included in a block (or more
+precisely, a chunk of the block) but ends up being invalid because the sender does not have enough
+balance, this transaction takes block space but cannot be charged against anyone.
+
+A very easy-to-perform attack exists if we do nothing to mitigate the problem:
+
+* The attacker creates two accounts, $A$ and $B$, with sufficiently many access keys each, and deploying
+  a specific contract for both accounts that provides a `send_near` function.
+* The attacker deposits 10 NEAR in $A$.
+* The attacker then performs the following, repeatedly:
+  * Submit a transaction to call A's `send_near` function, instructing it to send the account's remaining
+    balance to $B$.
+  * Right after, it floods the blockchain by signing and submitting many (arbitrary) transactions as $A$.
+    Because execution is asynchronous, the chunk producers think that there is still enough balance in
+    $A$'s account, so these transactions are accepted into chunks.
+  * Some blocks later, the execution catches up, and the `send_near` function drains $A$'s account.
+  * Subsequent executions of the following transactions all fail because $A$ has insufficient balance.
+  * After this is done, the attacker repeats but with $A$ and $B$ swapped.
+
+This attack can be carried out with a very simple script and requires no cost other than a single contract
+call every few blocks, but this ends up filling up the blockchain, denying legitimate transactions from
+being included. This is also very hard to defend against, because the attacker can simply create more
+accounts. Note that this attack pattern is not the only problematic one; instead of sending away the
+balance, the attacker can also delete access keys or delete the whole account.
+
+## Specification
+To solve this problem, we present two critical components which work together to ensure that all accepted
+transactions are valid. At a high level, they are:
+* **Access Key vs Gas Key**: In addition to Access Keys, we introduce a second kind of signing key called the Gas Key. A gas key can be funded with NEAR, and when issuing a transaction using a gas key, the transaction only consumes gas from the gas key, not from the account.
+* **Pending Transaction Queue**: Chunk producers keep track of pending (accepted into a block, but not 
+  executed) transactions, and ensures that transactions sent with access keys are limited in parallelism,
+  whereas transactions sent with gas keys are limited to the available gas in the gas key.
+
+We will now specify how exactly they work.
+
+### Definition of "Pending Transactions"
+In a model where execution follows but lags behind consensus, there are transactions which are accepted
+into consensus and thus committed to be executed in the near future, but are not yet executed. This set
+of transactions is called the *pending transactions*. We always discuss this in the context of one shard.
+
+Note that there are two slightly different ways to treat this definition, depending on how exactly the
+"execution head" (how far the execution has caught up) is defined:
+* It can be defined locally as the progress of execution at a node, but this will be different between
+  nodes.
+* It can be defined deterministically as the last block whose execution result is certified by consensus.
+
+For the purpose of this NEP, we use the latter definition, so that the notion of pending transactions is
+consistent across all nodes and the determinination of what transactions are eligible to be included by
+a chunk producer can be verified -- even though we do not plan to implement this verification right now.
+
+Another note is that the notion of pending transactions is anchored at a specific chunk that is being produced. In case of forks, we use the block that the chunk is being produced on top of to compute the
+set of pending transactions.
+
+Finally, this NEP does *not* depend on the implementation of SPICE. In the context where SPICE is not
+yet in effect, we consider the pending transactions queue to always be empty (despite technically being
+one chunk worth of transactions due to execution lagging one block behind in the current implementation),
+because we can always verify the validity of all transactions at the moment of inclusion.
+
+### Access Key vs Gas Key
+We introduce a new transaction version, `TransactionV2`, which is able to either specify an access key
+or a gas key. Any older transaction version is equivalent to a `TransactionV2` that specifies an access
+key.
+
+#### Access Key Parallelism Restriction
+We now restrict the ability to send multiple parallel pending transactions with Access Keys.
+
+Specifically, for any given account $A$ with any number of access keys, the total number of access key transactions in the pending transaction queue whose sender is $A$ cannot exceed $P_{\mathrm {max}}$, a
+constant determined by the epoch; we propose $P_{\mathrm {max}} = 4$.
+
+In other words, with traditional access keys, one cannot send more than 4 transactions with the same
+account before they are executed. If one wishes the send more transactions in parallel, they would need
+to create a gas key.
+
+From a UX perspective, we pick the $P_{\mathrm {max}}$ constant so that it is very unlikely that anyone
+exceeding this parallelism is an end user with a wallet app. Those who need more parallelism than this
+would be using a script to send transactions, and for those use cases we require them to use gas keys.
+
+#### Gas Keys
+This NEP adds Gas Keys, **conceptually** defined as the following:
+```rust
+struct ConceptualGasKey {
+    public_key: PublicKey,
+    nonces: Vec<Nonce>,
+    balance: Balance,
+    permission: AccessKeyPermission,
+}
+```
+
+We add these operations to manipulate gas keys:
+* `AddGasKey(PublicKey, AccessKeyPermission, usize)`: Creates a gas key with the given public key, 
+  permission, and number of nonces.
+* `DeleteGasKey(PublicKey)`: Deletes a gas key.
+* `FundGasKey(PublicKey, Balance)`: deducts balance from the account and gives it to the gas key.
+
+#### Semantics of Gas Key Actions
+`AddGasKey` is verified and executed as follows:
+* Check that the key does not already exist.
+* Check that if the permission is a `FunctionCallPermission`, the allowance is `None` (unlimited
+  allowance).
+* A new `GasKey` entry is added to the trie, keyed by (gas key prefix, account ID, public key). The
+  gas key prefix is a new trie prefix.
+* For each nonce ID (from 0 to the number of nonces minus 1), store the default nonce at the trie
+  prefix (gas key prefix, account ID, public key, nonce ID).
+  * The default nonce is block height * 1e6, the same as the default nonce calculation for access keys.
+* The operation is charged according to the amount of trie modifications.
+
+`FundGasKey` is verified and executed as follows:
+* The gas key must exist under the account.
+* The cost of this action is the amount to fund; it is then verified the same way as a `Transfer` action.
+  * See below for "gas key transactions": it is also possible to fund a gas key using a gas key; so this
+    amount is not necessarily deducted straight from the account.
+* To apply this action, the balance on the gas key is increased by the same amount.
+
+`DeleteGasKey` is verified as executed as follows:
+* The gas key must exist under the account.
+* To apply this action, all relevant trie nodes are deleted, and the cost accounted for from trie
+  modifications.
+* The remaining balance left in the key is **burned**. (This prevent the key deletion attack.)
+  * For user's benefit we may consider failing this action if the balance exceeds a certain threshold.
+  * Note that this does not mean that all balance in a gas key is eventually burned; one can withdraw from
+    a gas key with a gas key transaction with a simple `Transfer` action.
+
+#### Gas Key Transactions
+
+`TransactionV2` reuses the fields of `TransactionV1` except replacing the `public_key` and `nonce` fields
+with an enum:
+```rust
+enum TransactionKeyKind {
+    AccessKey {
+        key: PublicKey,
+        nonce: Nonce
+    },
+    GasKey {
+        key: PublicKey,
+        nonce_id: usize,
+        nonce: Nonce,
+    }
+}
+```
+
+The semantics for gas key transactions, at the moment of execution (conversion to receipts), are:
+* A gas key transaction is valid iff all of the following are true:
+  * The public key corresponds to a valid gas key;
+  * The gas key has enough balance to cover the total transaction cost (all costs included in config.rs
+    `tx_cost`; this includes amounts included in `Transfer` actions too);
+  * The nonce ID < total number of nonces for the gas key, and the nonce is a valid nonce for that nonce 
+    ID (per the same nonce checks as access key);
+* When converting the gas key transaction to a receipt, the same logic applies as for access key
+  transactions, except
+  * The transaction cost is deducted from the gas key instead of the account.
+  * The new nonce is written for the specific nonce ID under the gas key.
+
+#### Gas Key Pending Transaction Constraints
+Unlike access key transactions, gas key transactions are not limited in parallelism; they are only limited
+by the amount of gas these transactions consume. Specifically, for a gas key $G$, the total cost of 
+the transactions signed with $G$ in the pending transactions must not exceed the balance of $G$
+(according to the state that the pending transactions are calculated based on).
+
+This constraint should be good enough to cover cases of adding, funding, removing, or withdrawing from gas
+keys as well. For adding and funding, we do need the execution to catch up before the newly available
+balance can be used for pending transactions, which is suboptimal but correct. For withdrawing, the
+action of withdrawing itself goes into the cost of the transaction and therefore is accounted for (and
+note that contract execution cannot create receipts that withdraw from gas keys; only gas key
+transactions can). For deletion, balance in gas keys are not refunded, so although subsequent pending
+transactions may end up failing, the gas committed to those transactions are already burnt, eliminating
+the opportunity for the aforementioned attack.
+
+### Pending Transaction Queue
+
+We would now maintain a new data structure that stores the pending transactions, called the Pending
+Transaction Queue. Although it's conceptually a queue, it is stored as a collection indexed by
+(account ID, transaction type), and further indexed by the block hash. Furthermore, the pending
+transaction queue is stored per block per shard, not as a single data structure. The contents of the queue
+is exactly the pending transactions according to the definition above.
+
+The constraints are enforced as described above; we reiterate it here:
+* There cannot be more than 4 access key transactions under the same account in the queue;
+* There cannot be more gas key transactions under the same gas key in the queue whose total transaction
+  cost exceed the gas key's balance.
+
+The constraints are maintained at the time of chunk production: when producing a chunk, we only accept
+transactions that would maintain these constraints. For limiting gas key transactions, we always query the
+balance of the gas key from the executed state (as opposed to storing it in the queue).
+
+To compute the pending transaction queues (one for each tracked shard) for a new block,
+* Start from the queue from the previous block;
+* Subtract the transactions that are included in each newly certified block;
+* Add new transactions included in this block.
+
+### Impact to Existing Protocol without SPICE
+As mentioned above, this NEP applies with or without SPICE. The impact to the existing protocol is purely
+positive:
+* Access keys are not restricted, as the set of pending transactions is always empty.
+* Gas keys allow programmatic transaction senders to more easily manage multiple nonces. Rather than
+  requiring multiple access keys to be created, they just need to create a single gas key.
+
+The implementation of gas keys before SPICE also allows programmatic users to migrate to using gas keys,
+preparing them for when SPICE is launched.
+
+## Alternatives
+TODO
+
+## TODO: Other things