-
Notifications
You must be signed in to change notification settings - Fork 491
Add txpool namespace spec and test generators #758
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bomanaps
wants to merge
7
commits into
ethereum:main
Choose a base branch
from
bomanaps:add-txpool-namespace
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
0224433
Add txpool namespace spec and test generators
bomanaps 5c07f5a
Fix txpool schema: allow null for block fields
bomanaps cda8d92
Add txpool_contentFrom and txpool_inspect method specs
bomanaps a956a44
clarify queued definition and txpool_inspect format in specs
bomanaps db52750
remove txpool_inspect from txpool namespace
bomanaps 72b30d3
regenerate txpool content and status test fixtures
bomanaps 6ba587a
mark txpool test generators as spec-only
bomanaps File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| TxpoolStatus: | ||
| type: object | ||
| title: Transaction pool status | ||
| description: The number of pending and queued transactions in the pool. | ||
| required: | ||
| - pending | ||
| - queued | ||
| properties: | ||
| pending: | ||
| title: pending count | ||
| description: Number of transactions ready for inclusion in the next block(s) | ||
| $ref: '#/components/schemas/uint' | ||
| queued: | ||
| title: queued count | ||
| description: Number of transactions with nonce gaps awaiting preceding transactions before they can be executed | ||
| $ref: '#/components/schemas/uint' | ||
|
|
||
| PendingTransactionInfo: | ||
| type: object | ||
| title: Pending transaction information | ||
| description: Transaction in the pool, not yet included in a block. | ||
| allOf: | ||
| - title: Contextual information | ||
| required: | ||
| - from | ||
| - hash | ||
| properties: | ||
| blockHash: | ||
| title: block hash | ||
| description: Always null for pending transactions | ||
| type: "null" | ||
| blockNumber: | ||
| title: block number | ||
| description: Always null for pending transactions | ||
| type: "null" | ||
| blockTimestamp: | ||
| title: block timestamp | ||
| description: Always null for pending transactions | ||
| type: "null" | ||
| from: | ||
| title: from address | ||
| $ref: '#/components/schemas/address' | ||
| hash: | ||
| title: transaction hash | ||
| $ref: '#/components/schemas/hash32' | ||
| transactionIndex: | ||
| title: transaction index | ||
| description: Always null for pending transactions | ||
| type: "null" | ||
| - $ref: '#/components/schemas/TransactionSigned' | ||
|
|
||
| TxpoolContentByAddress: | ||
| type: object | ||
| title: Transactions by nonce | ||
| description: Map of nonce to transaction object | ||
| additionalProperties: | ||
| $ref: '#/components/schemas/PendingTransactionInfo' | ||
|
|
||
| TxpoolContentAddressMap: | ||
| type: object | ||
| title: Transactions by address | ||
| description: Map of address to transactions grouped by nonce | ||
| additionalProperties: | ||
| $ref: '#/components/schemas/TxpoolContentByAddress' | ||
|
|
||
| TxpoolContent: | ||
| type: object | ||
| title: Transaction pool content | ||
| description: All pending and queued transactions in the pool, grouped by address and nonce. | ||
| required: | ||
| - pending | ||
| - queued | ||
| properties: | ||
| pending: | ||
| title: pending transactions | ||
| description: Transactions ready for inclusion in the next block(s) | ||
| $ref: '#/components/schemas/TxpoolContentAddressMap' | ||
| queued: | ||
| title: queued transactions | ||
| description: Transactions with nonce gaps awaiting preceding transactions before they can be executed | ||
| $ref: '#/components/schemas/TxpoolContentAddressMap' | ||
|
|
||
| TxpoolContentFromResult: | ||
| type: object | ||
| title: Transaction pool content from address | ||
| description: Pending and queued transactions from a specific address, grouped by nonce. | ||
| required: | ||
| - pending | ||
| - queued | ||
| properties: | ||
| pending: | ||
| title: pending transactions | ||
| description: Transactions ready for inclusion in the next block(s) | ||
| $ref: '#/components/schemas/TxpoolContentByAddress' | ||
| queued: | ||
| title: queued transactions | ||
| description: Transactions with nonce gaps awaiting preceding transactions before they can be executed | ||
| $ref: '#/components/schemas/TxpoolContentByAddress' | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| - name: txpool_status | ||
| summary: Returns the number of pending and queued transactions in the pool. | ||
| description: | | ||
| Returns an object containing the count of transactions currently pending for | ||
| inclusion in the next block(s), as well as ones that are scheduled for future | ||
| execution (transactions with nonce gaps). | ||
| params: [] | ||
| result: | ||
| name: Transaction pool status | ||
| schema: | ||
| $ref: '#/components/schemas/TxpoolStatus' | ||
| examples: | ||
| - name: txpool_status example | ||
| params: [] | ||
| result: | ||
| name: Transaction pool status | ||
| value: | ||
| pending: '0xa' | ||
| queued: '0x7' | ||
|
|
||
| - name: txpool_content | ||
| summary: Returns the contents of the transaction pool. | ||
| description: | | ||
| Returns an object containing all pending and queued transactions in the pool, | ||
| grouped by origin address and sorted by nonce. Pending transactions are ready | ||
| for inclusion in the next block(s). Queued transactions have nonce gaps and | ||
| are scheduled for future execution. | ||
| params: [] | ||
| result: | ||
| name: Transaction pool content | ||
| schema: | ||
| $ref: '#/components/schemas/TxpoolContent' | ||
| examples: | ||
| - name: txpool_content example | ||
| params: [] | ||
| result: | ||
| name: Transaction pool content | ||
| value: | ||
| pending: | ||
| '0x0216d5032f356960cd3749c31ab34eeff21b3395': | ||
| '806': | ||
| blockHash: null | ||
| blockNumber: null | ||
| from: '0x0216d5032f356960cd3749c31ab34eeff21b3395' | ||
| gas: '0x5208' | ||
| gasPrice: '0xba43b7400' | ||
| hash: '0xaf953a2d01f55cfe080c0c94150a60105e8ac3d51153058a1f03dd239dd08586' | ||
| input: '0x' | ||
| nonce: '0x326' | ||
| to: '0x7f69a91a3cf4be60020fb58b893b7cbb65376db8' | ||
| transactionIndex: null | ||
| value: '0x19a99f0cf456000' | ||
| queued: | ||
| '0x0216d5032f356960cd3749c31ab34eeff21b3395': | ||
| '808': | ||
| blockHash: null | ||
| blockNumber: null | ||
| from: '0x0216d5032f356960cd3749c31ab34eeff21b3395' | ||
| gas: '0x5208' | ||
| gasPrice: '0xba43b7400' | ||
| hash: '0x593f723c6f7abc7878c4927d2f1a0e6c5a6c0b4c9f5a7b2e3d4c5f6a7b8c9d0e' | ||
| input: '0x' | ||
| nonce: '0x328' | ||
| to: '0x7f69a91a3cf4be60020fb58b893b7cbb65376db8' | ||
| transactionIndex: null | ||
| value: '0x19a99f0cf456000' | ||
|
|
||
| - name: txpool_contentFrom | ||
| summary: Returns the transactions in the pool from a specific address. | ||
| description: | | ||
| Returns an object containing pending and queued transactions from the specified | ||
| address, grouped by nonce. This is a filtered version of txpool_content. | ||
| params: | ||
| - name: address | ||
| required: true | ||
| schema: | ||
| $ref: '#/components/schemas/address' | ||
| result: | ||
| name: Transaction pool content from address | ||
| schema: | ||
| $ref: '#/components/schemas/TxpoolContentFromResult' | ||
| examples: | ||
| - name: txpool_contentFrom example | ||
| params: | ||
| - name: address | ||
| value: '0x0216d5032f356960cd3749c31ab34eeff21b3395' | ||
| result: | ||
| name: Transaction pool content from address | ||
| value: | ||
| pending: | ||
| '806': | ||
| blockHash: null | ||
| blockNumber: null | ||
| from: '0x0216d5032f356960cd3749c31ab34eeff21b3395' | ||
| gas: '0x5208' | ||
| gasPrice: '0xba43b7400' | ||
| hash: '0xaf953a2d01f55cfe080c0c94150a60105e8ac3d51153058a1f03dd239dd08586' | ||
| input: '0x' | ||
| nonce: '0x326' | ||
| to: '0x7f69a91a3cf4be60020fb58b893b7cbb65376db8' | ||
| transactionIndex: null | ||
| value: '0x19a99f0cf456000' | ||
| queued: {} | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| // retrieves the transaction pool content | ||
| // speconly: client response is only checked for schema validity. | ||
| >> {"jsonrpc":"2.0","id":1,"method":"txpool_content"} | ||
| << {"jsonrpc":"2.0","id":1,"result":{"pending":{"0x0c2c51a0990AeE1d73C1228de158688341557508":{"0":{"blockHash":null,"blockNumber":null,"blockTimestamp":null,"from":"0x0c2c51a0990aee1d73c1228de158688341557508","gas":"0x61a8","gasPrice":"0x5763d65","hash":"0x66734e85ef096167acb887cf445946a1ed57b90b66ffe38af87e11294febbfa9","input":"0x5544","nonce":"0x0","to":"0xaa00000000000000000000000000000000000000","transactionIndex":null,"value":"0xa","type":"0x0","chainId":"0xc72dd9d5e883e","v":"0x18e5bb3abd109f","r":"0xc8e3b4a0087357bd49d80a0ac24daf0c91191e71086c1e355fc62cfab2218873","s":"0x74f4636f740fa4d1697b6e736e5982b700be2c8b63031a24fa531ae4814b3af8"},"1":{"blockHash":null,"blockNumber":null,"blockTimestamp":null,"from":"0x0c2c51a0990aee1d73c1228de158688341557508","gas":"0xea60","gasPrice":"0x5763f58","maxFeePerGas":"0x5763f58","maxPriorityFeePerGas":"0x1f4","hash":"0xfa245384e9eb7d6a4a40f3bf4bf70f1f44929d8bfcdf75762ce1a015389449e3","input":"0x3d602d80600a3d3981f3363d3d373d3d3d363d734d11c446473105a02b5c1ab9ebe9b03f33902a295af43d82803e903d91602b57fd5bf3","nonce":"0x1","to":null,"transactionIndex":null,"value":"0x2a","type":"0x2","accessList":[],"chainId":"0xc72dd9d5e883e","v":"0x1","r":"0xfe6d380224a516b802717755d2f640163e81bae64a4ab5adbcf741267f20ad66","s":"0x15d9ceb9fecb47b342be00782b2485f42ab53715006d208897cc969d7c05ab67","yParity":"0x1"},"2":{"blockHash":null,"blockNumber":null,"blockTimestamp":null,"from":"0x0c2c51a0990aee1d73c1228de158688341557508","gas":"0x15f90","gasPrice":"0x5763f58","hash":"0xd07a55a00aeb93c7825d1ca42238abdc3bc225de097ee1b8b2a4a9240ae55f9c","input":"0x010203","nonce":"0x2","to":"0x7dcd17433742f4c0ca53122ab541d0ba67fc27df","transactionIndex":null,"value":"0x0","type":"0x1","accessList":[{"address":"0x7dcd17433742f4c0ca53122ab541d0ba67fc27df","storageKeys":["0x0000000000000000000000000000000000000000000000000000000000000000","0x0100000000000000000000000000000000000000000000000000000000000000"]}],"chainId":"0xc72dd9d5e883e","v":"0x0","r":"0xf9dc42e8bab0a70132fb8399cf03cf38e1c12cc47f736d19e6e7728356d97db3","s":"0x53daf342acd24da15073f5dac02bec0501a0716165984aab2df9694882b91fac","yParity":"0x0"},"3":{"blockHash":null,"blockNumber":null,"blockTimestamp":null,"from":"0x0c2c51a0990aee1d73c1228de158688341557508","gas":"0x13880","gasPrice":"0x5763f58","maxFeePerGas":"0x5763f58","maxPriorityFeePerGas":"0x1f4","hash":"0x0baf604666cbc4d04263bbc98c048000451b8c188d93ec87ca5a86b044fd956c","input":"0x01020304","nonce":"0x3","to":"0x7dcd17433742f4c0ca53122ab541d0ba67fc27df","transactionIndex":null,"value":"0x0","type":"0x2","accessList":[{"address":"0x7dcd17433742f4c0ca53122ab541d0ba67fc27df","storageKeys":["0x0000000000000000000000000000000000000000000000000000000000000000","0x0100000000000000000000000000000000000000000000000000000000000000"]}],"chainId":"0xc72dd9d5e883e","v":"0x0","r":"0xe56d869d8b32f767582fdcb03d1d9d3bcc47f3c7ae08984feafdcd57f2f205f5","s":"0x74134e4bf0fb11ff606b47259aff0d01bf7cb9ec68cb179b62576b9dd6631cf0","yParity":"0x0"}},"0x14e46043e63D0E3cdcf2530519f4cFAf35058Cb2":{"0":{"blockHash":null,"blockNumber":null,"blockTimestamp":null,"from":"0x14e46043e63d0e3cdcf2530519f4cfaf35058cb2","gas":"0x5208","gasPrice":"0x5763f58","maxFeePerGas":"0x5763f58","maxPriorityFeePerGas":"0x1f4","hash":"0x5acd356e2e5dcd345a43e8a714c0bba3138856c25ac0f9cec51f4ae6aad77c4d","input":"0x","nonce":"0x0","to":"0x7dcd17433742f4c0ca53122ab541d0ba67fc27df","transactionIndex":null,"value":"0x3e8","type":"0x2","accessList":[],"chainId":"0xc72dd9d5e883e","v":"0x0","r":"0x2ff0582cbfd9034c5fa5081d8e87689fca126ef89e764ed75b9377a5abc17174","s":"0x3f88569a957315fa1204dcc026fcc50cef44c6268642990b8f05b226d8f60a40","yParity":"0x0"}}},"queued":{}}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| // retrieves pending transactions from a specific address | ||
| // speconly: client response is only checked for schema validity. | ||
| >> {"jsonrpc":"2.0","id":1,"method":"txpool_contentFrom","params":["0x0000000000000000000000000000000000000000"]} | ||
| << {"jsonrpc":"2.0","id":1,"result":{"pending":{},"queued":{}}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| // retrieves the transaction pool status | ||
| // speconly: client response is only checked for schema validity. | ||
| >> {"jsonrpc":"2.0","id":1,"method":"txpool_status"} | ||
| << {"jsonrpc":"2.0","id":1,"result":{"pending":"0x6","queued":"0x0"}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -74,6 +74,7 @@ ssz | |
| statev | ||
| statusv | ||
| teku | ||
| txpool | ||
| txs | ||
| txt | ||
| uint | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand it is convenient to return all the txs with a single call, but that does not scale well with large pools, and could hit some of the RPC response size limits, so it would be nice to have some kind of agreed limit on how many are returned by this call.
The same applies also to other calls, only
txpool_statushas a bounded response size, so a general approach to filtering and/or pagination could be evaluated.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess the question is: Are we documenting the feature that already exists and how clients treat the method today, or are we trying to standardize something new and push the clients to change? Because this is how the method works, in basically all the clients today.
https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-txpool#txpool-content
https://reth.rs/jsonrpc/txpool/#txpool_content
https://docs.nethermind.io/interacting/json-rpc-ns/txpool/#txpool_content
https://docs.erigon.tech/interacting-with-erigon/interacting-with-erigon/txpool#txpool_content
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both things are valuable, and could be part of an iteration, where starting from the de facto standard, API are evolved to match current use cases.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure but I don't know if pagination is even an option, the mempool changes constantly, you wouldn't be able to split transactions into pages AND track the pages between calls.
In theory the mempool is already bound by a max size that is configurable on the nodes. You'r right in that it returns potentially a very large responses and is annoying to support at the RPC provider level, and many don't, they simply turn it off / block it.
txpool_contentis ripe for abuse from MEV bots and searchers against the pulbic mempool, but it can also be a useful debugging tool to pull all the txs out of the mempool.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed that scalability is a valid concern. This PR documents the existing behavior across clients. The proposed txpool_transactions method with filtering and limits will address these concerns through the EIP process that's where we can design a more scalable approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that perfect pagination is not worth, but if there is any use case for downloading everything, some kind of range could be passed, like responses are sorted by sender, so you can use the last sender as input for the next batch.
For example in Besu, the txpool is only bounded by the amount of memory you give to it, so if you configure 1GB there could be millions of pending txs in the pool.
Said that, I do not think there is any urgency in doing some sort of pagination now, I see filtering more useful to have as debugging tool.