diff --git a/README.md b/README.md index 6c9f21561..0291fcc88 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,7 @@ For more details on how to use rbuilder for backtesting, see https://github.com/ To run rbuilder you need: * Reth node for state. (`reth_datadir`) * Reth node must expose ipc interface for mempool tx subscription (`el_node_ipc_path`). +* Reth node must be configured to flush every block (--engine.persistence-threshold "0" --engine.memory-block-buffer-target "0") * CL node that triggers new payload events (it must be additionally configured to trigger payload event every single time). * Source of bundles that sends `eth_sendBundle`, `mev_sendBundle`, `eth_sendRawTransaction` as JSON rpc calls. (`jsonrpc_server_port`) (by default rbuilder will take raw txs from the reth node mempool) @@ -55,7 +56,22 @@ A sample configuration for running Lighthouse and triggering payload events woul --prepare-payload-lookahead 8000 \ --suggested-fee-recipient 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045 ``` - +and for reth: +``` +/usr/local/bin/reth node \ + --datadir /mnt/md0/rethdata \ + --authrpc.jwtsecret /secrets/jwt.hex \ + --authrpc.addr 127.0.0.1 \ + --authrpc.port 8551 \ + --http \ + --ws \ + --ws.addr 127.0.0.1 \ + --ws.port 8545 \ + --rpc-max-connections 429496729 \ + --http.api trace,web3,eth,debug \ + --ws.api trace,web3,eth,debug \ + --engine.persistence-threshold "0" --engine.memory-block-buffer-target "0" +``` Additionally, you can: * configure block processor API as a sink for submitted blocks (`blocks_processor_url`) * setup Prometheus / Grafana for metrics (served on `telemetry_port` + `/debug/metrics/prometheus`) diff --git a/docs/CONFIG.md b/docs/CONFIG.md index 427b834df..8e1740e42 100644 --- a/docs/CONFIG.md +++ b/docs/CONFIG.md @@ -5,51 +5,56 @@ Every field has a default if omitted. | Name | Type | Comments | Default | |------|------|-------------|---------| -|full_telemetry_server_port|string| |"0.0.0.0"| -|full_telemetry_server_ip|int || 6069| -|redacted_telemetry_server_port|string| |"0.0.0.0"| -|redacted_telemetry_server_ip|int || 6070| +|full_telemetry_server_port|int| |6069| +|full_telemetry_server_ip|string| |"0.0.0.0"| +|redacted_telemetry_server_port|int| |6070| +|redacted_telemetry_server_ip|string| |"0.0.0.0"| |log_json|bool|JSON vs Raw|false| |log_level|env/string| Defines the log level (EnvFilter) for each mod. See https://docs.rs/tracing-subscriber/latest/tracing_subscriber/index.html for more info on this.|"info" |log_color|bool||false| +|otlp_env_name|optional string|Name of the OTEL environment (e.g. production, staging).|None| |error_storage_path|optional string| Path to a sqlite file that will store info for some critical errors|None| |coinbase_secret_key|optional env/string|If no key is provided a random one is generated. Format is "0x121232432...."|None| |el_node_ipc_path|optional string| Path for Ipc communication with reth's mempool, Usually something like "/tmp/reth.ipc". If not set mempool will not be used as a source of txs|None| -|jsonrpc_server_port| int| |8645| +|jsonrpc_server_port|int| |8645| |jsonrpc_server_ip|string||"0.0.0.0"| -|jsonrpc_server_max_connections|int|The maximum number of connections|4096| +|jsonrpc_server_max_connections|optional int|Max connections for JSON-RPC server. If omitted, 4096 is used.|None (effective 4096)| |ignore_cancellable_orders|bool|If true any order with replacement id will be dropped|true| |ignore_blobs|bool|If true txs with blobs will be ignored|false| |chain|string| |"mainnet"| |reth_datadir|optional string|It will assume default child dirs "db"/"static_files".
**It's mandatory to set reth_datadir or reth_db_path+reth_static_files_path or ipc_provider**|"/mnt/data/reth"| |reth_db_path|optional string|**It's mandatory to set reth_datadir or reth_db_path+reth_static_files_path or ipc_provider**|None| |reth_static_files_path|optional string|**It's mandatory to set reth_datadir or reth_db_path+reth_static_files_path or ipc_provider**|None| -|ipc_provider|optional | If configured it will use IPC provider for EVM state (instead of accessing a local reth db). More doc pending.|None| +|ipc_provider|optional object| If set, use IPC for EVM state instead of local reth db. Fields: **ipc_path** (path), **request_timeout_ms** (u64, default 100), **mempool_server_url** (string).|None| |blocklist_file_path|optional string| Backwards compatibility. Downloads blocklist from a file. Same as setting a file name on blocklist.|None| |blocklist|optional string| Can contain an url or a file name.
If it's a url download blocklist from url and updates periodically.
If it's a filename just loads the file (no updates).|None| |blocklist_url_max_age_hours|optional int|If the downloaded file get older than this we abort.|None| |blocklist_url_max_age_secs|optional int|If the downloaded file get older than this we abort. Used for debugging only|None| -|require_non_empty_blocklist|bool|if true will not allow to start without a blocklist or with an empty blocklist.|false| +|require_non_empty_blocklist|optional bool| If true, will not allow start without a blocklist or with an empty blocklist.|false| |extra_data|string|Extra data for generated blocks|"extra_data_change_me"| |simulation_threads|int| Number of threads used for incoming order simulation|1| |simulation_use_random_coinbase|bool| |true| |root_hash_use_sparse_trie|bool| Uses cached sparse trie for root hash (much faster)|false| +|root_hash_sparse_trie_version|string| Sparse trie version: "v1" or "v2".|"v1"| |root_hash_compare_sparse_trie|bool| If using sparse trie also computes against reth's native version to check the hash is ok|false| -|root_hash_threads| int|Threads used when using reth's native root hash calculation. If 0 global rayon pool is used| 0 -| watchdog_timeout_sec| optional int| If now block building is started in this period rbuilder exits.|None| +|root_hash_threads|int|Threads used when using reth's native root hash calculation. If 0 global rayon pool is used|0| +|adjust_finalized_blocks|bool| Use pipelined finalization (blocks prefinalized first, payment tx inserted later for faster bidding).|false| +|watchdog_timeout_sec|optional int| If no block building is started in this period rbuilder exits.|None| |live_builders|vec[string]| List of `builders` to be used for live building.
Notice that you can define on **builders** some builders and select only a few here.|["mgp-ordering","mp-ordering"]| |evm_caching_enable|bool|If enabled per block EVM execution will be enabled|false| |faster_finalize|bool| If enabled improves block finalization by catching proofs|false| -|time_to_keep_mempool_txs_secs|u64| /// After this time a mempool tx is dropped.|1| +|time_to_keep_mempool_txs_secs|u64| After this time a mempool tx is dropped.|60| +|system_recipient_allowlist|vec[Address]| Senders from which incoming tx profit is not counted towards coinbase profit.|[]| |backtest_fetch_mempool_data_dir|env/string|Dir used to store mempool data used in backtesting|"/mnt/data/mempool"| |backtest_fetch_eth_rpc_url|string|url to EL node RPC used in backtesting|"http://127.0.0.1:8545"| -|backtest_fetch_eth_rpc_parallel| int|Number of parallel connections allowed on backtest_fetch_eth_rpc_url|1| +|backtest_fetch_eth_rpc_parallel|int|Number of parallel connections allowed on backtest_fetch_eth_rpc_url|1| |backtest_fetch_output_file|string | Path to a sqlite containing block information for backtesting. This file is filled with backtest-fetch|"/tmp/rbuilder-backtest.sqlite"| |backtest_results_store_path|string|Path to a sqlite containing backtest results|"/tmp/rbuilder-backtest-results.sqlite"| |backtest_builders|vec[string]| List of `builders` to be used for backtesting.
Notice that you can define on **builders** some builders and select only a few here.|[]| |backtest_protect_bundle_signers|vec[string]|Doc pending.|[]| -|orderflow_tracing_store_path|Optional string|We will store a file per block in this path.|None| -|orderflow_tracing_max_blocks|Optional int|Max number of blocks to keep in disk.
Remember to also set this if you enable orderflow_tracing_store_path.|None| +|orderflow_tracing_store_path|optional string|We will store a file per block in this path.|None| +|orderflow_tracing_max_blocks|int|Max number of blocks to keep on disk. Set > 0 if you enable orderflow_tracing_store_path.|0| +|max_order_execution_duration_warning_us|optional u64| If set, log a warning when an order execution exceeds this duration (microseconds).|None| ## L1 related fields | Name | Type | Comments | Default | @@ -68,20 +73,25 @@ Every field has a default if omitted. |RelayConfig.optimistic|optional bool||false| |RelayConfig.interval_between_submissions_ms|optional int| Caps the submission rate to the relay|None| |RelayConfig.max_bid_eth|optional string| Max bid we can submit to this relay. Any bid above this will be skipped.
None -> No limit.|None| +|RelayConfig.adjustment_fee_payer|optional string| Address that pays bid adjustment fees for this relay.|None| +|RelayConfig.submit_config.optimistic_v3|bool| Use optimistic V3 submissions for this relay.|false| +|RelayConfig.submit_config.optimistic_v3_bid_adjustment_required|bool| Whether bid adjustments are required for optimistic V3.|false| |RelayConfig.is_bloxroute|bool|Set to `true` for bloxroute relays to add extra headers.|false| +|RelayConfig.bloxroute_rproxy_regions|vec[string]| Bloxroute rproxy regions to try, in order of preference.|[]| +|RelayConfig.bloxroute_rproxy_only|bool| If true, only submit to bloxroute rproxy endpoints when available.|false| |RelayConfig.ask_for_filtering_validators|optional bool| Adds "filtering=true" as query to the call relay/v1/builder/validators to get all validators (including those filtering OFAC).
On 2025/06/24 only supported by ultrasound.|false| |RelayConfig.can_ignore_gas_limit|optional bool| If we submit a block with a different gas than the one the validator registered with in this relay the relay does not mind. Useful for gas limit conflicts. On 2025/08/20 only ultrasound confirmed that is ok with this. (we didn't asked the rest yet)|false| -|enabled_relays| vec["string"]| Extra hardcoded relays to add (see DEFAULT_RELAYS in [config.rs](../crates/rbuilder/src/live_builder/config.rs))|[]| -|relay_secret_key|optional env/string|Secret key that will be used to sign normal submissions to the relay.|None| -|optimistic_relay_secret_key|optional env/string|Secret key that will be used to sign optimistic submissions to the relay.|None| -|optimistic_enabled|bool|When enabled builder will make optimistic submissions to optimistic relays|false| -|optimistic_max_bid_value_eth|string| Bids above this value will always be submitted in non-optimistic mode.|"0.0"| -|cl_node_url|vec[env/stirng]| Array if urls to CL clients to get the new payload events|["http://127.0.0.1:3500"] +|enabled_relays|vec[string]| Extra hardcoded relays to add (see DEFAULT_RELAYS in [config.rs](../crates/rbuilder/src/live_builder/config.rs))|[]| +|relay_secret_key|optional env/string|Secret key that will be used to sign submissions to the relay.|None| +|cl_node_url|vec[env/string]| Array of urls to CL clients to get the new payload events.|["http://127.0.0.1:3500"] |genesis_fork_version|optional string|Genesis fork version for the chain. If not provided it will be fetched from the beacon client.|None| |relay_bid_scrapers||See [bid scraper publishers](../crates/bid-scraper/README.md) |Empty| -|max_order_execution_duration_warning_us|optional u64| If set, while building a block if an order execution duration is greater than this, we will log a warning with some info about the order.|None| +|optimistic_v3_server_ip|string| Optimistic V3 server bind IP.|"0.0.0.0"| +|optimistic_v3_server_port|int| Optimistic V3 server port.|6071| +|optimistic_v3_public_url|string| Public URL where relays can fetch blocks (for optimistic V3).|""| +|optimistic_v3_relay_pubkeys|set[string]| BLS public keys of relays that may use optimistic V3.|[]| ## Building algorithms -rbuilder can multiple building algorithms and each algorithm can be instantiated multiple times with it's own set of parameters each time. +rbuilder can use multiple building algorithms and each algorithm can be instantiated multiple times with it's own set of parameters each time. Each instantiated algorithm starts with: | Name | Type | Comments | Default | |------|------|-------------|---------| @@ -101,16 +111,17 @@ Each instantiated algorithm starts with: ### Fields for algo="parallel-builder" - | Name | Type | Comments | Default | |------|------|-------------|---------| -|discard_txs|mandatory bool| If a tx inside a bundle fails with TransactionErr (don't confuse this with reverting which is TransactionOk with !.receipt.success) and it's configured as allowed to revert (for bundles tx in reverting_tx_hashes or dropping_tx_hashes) we continue the execution of the bundle. The most typical value is true.|| -|num_threads| mandatory int| Number of threads to use for merging.|| +|discard_txs|mandatory bool| If a tx inside a bundle fails with TransactionErr (don't confuse this with reverting which is TransactionOk with !.receipt.success) and it's configured as allowed to revert (for bundles tx in reverting_tx_hashes or dropping_tx_hashes) we continue the execution of the bundle. The most typical value is true.|| +|num_threads|mandatory int| Number of threads to use for merging.|| +|safe_sorting_only|bool| Only use sort modes that don't risk breaking "best refund for user" (avoids putting worst kickback first).|true| ## Bidding fields | Name | Type | Comments | Default | |------|------|-------------|---------| -|slot_delta_to_start_bidding_ms| optional int| When the sample bidder (see TrueBlockValueBiddingService) will start bidding relative to the slot start.
Usually a negative number.|None| -|subsidy|optional string|Value added to the bids (see TrueBlockValueBiddingService).
The builder address must have enough balance for the subsidy.
Example:"1.23" for 1.23 ETH|None| +|slot_delta_to_start_bidding_ms|optional int| When the sample bidder (see TrueBlockValueBiddingService) will start bidding relative to the slot start.
Usually a negative number.|None| +|subsidy|optional string|Value added to the bids (see TrueBlockValueBiddingService).
The builder address must have enough balance for the subsidy.
Example: "1.23" for 1.23 ETH|None| +|subsidy_overrides|vec[{relay, value}]| Per-relay subsidy override. Example: `[[subsidy_overrides]] relay = "flashbots_test2" value = "0.05"`|[]|