This document will describes the minimum requirements to deploy and support EOS EVM Network for Centralized Cryptocurrency Exchanges.
This is the minimum setup to run a EOS EVM service. It does not contain the high availability setup. Exchanges can duplicate the real-time service part for high availability purpose if necessary.
Real-time service:
+--VM1 (spring node) ----------+ +--VM2 (EOS-EVM node)--------+
| EOS main node process with | <-- | eos-evm-node & eos-evm-rpc | <-- eth compatible read requests
| state_history_plugin enabled | +----------------------------+ (e.g. eth_getBlockByNumber, eth_call ...)
+------------------------------+ \
^ \ +--- VM2 (EOS-EVM node)----+
| <-- | proxy to separate read & |
| / | write requests (optional)|
| / +--------------------------+
| /
| +--VM2 (EOS-EVM node)-----+
\-- push EOS transactions --| eos-evm-miner (wrapper) | <-- eth_gasPrice
+-------------------------+ eth_sendRawTransaction
Periodic Backup service:
+--VM3 (Backup VM) ------------------------+ +-- VM3 (Backup VM) ---------+
| spring node running in irreversible mode | <----- | eos-evm-node & eos-evm-rpc |
| with state_history_plugin enabled | +----------------------------+
+------------------------------------------+
spring node stands for the EOS (Level 1) blockchain, and eos-evm-node stands for the EOS-EVM (Level 2) blockchain. eos-evm-rpc talk to eos-evm-node in the same VM and it is used for providing read-only ETH APIs (such as eth_getBlockByNumber, eth_call, eth_blockNumber, ... ) which is compatible with standard ETH API. For ETH write requests eth_sendRawTransaction, and eth_gasPrice, they will be served via eos-evm-miner instead of eos-evm-rpc.
- VM1: this VM will run EOS spring node with state_history_plugin enabled. A high end CPU with good single threaded performance is recommended. RAM: 128GB+, SSD 2TB+ (for storing block logs & state history from the EVM genesis time (2023-04-05T02:18:09 UTC) up to now)
- VM2: this VM will run eos-evm-node, eos-evm-rpc & eos-evm-miner. Recommend to use 8 vCPU, 32GB+ RAM, and 1TB+ SSD
- VM3: this VM will run spring (in irrversible mode), eos-evm-node & eos-evm-rpc and mainly for backup purpose. Recommend to use 8 vCPU, 128GB+ RAM, 3TB+ SSD (backup files can be large).
OS: Recommend to use ubuntu 22.04
-
EOS (spring) Node: the main process for EOS chain please refer to https://github.com/AntelopeIO/spring The latest pre-built stable version can be downloaded from https://github.com/AntelopeIO/spring/releases
-
eos-evm-node, eos-evm-rpc: the main process for EOS-EVM chain Please build the latest stable version in https://github.com/eosnetworkfoundation/eos-evm-node
for example, the latest stable version is v1.0.1 up to the time of writing this doc.
git clone https://github.com/eosnetworkfoundation/eos-evm-node.git
cd eos-evm-node
git checkout v1.0.1
git submodule update --init --recursive
mkdir build; cd build;
cmake .. && make -j8
-
Eos-evm-miner: please refer to https://github.com/eosnetworkfoundation/eos-evm-miner
-
Proxy to separate read requests & write requests: please refer to https://github.com/eosnetworkfoundation/eos-evm-node/tree/main/peripherals/proxy
- For the first time: You need a snapshot file whose timestamp is before the EVM genesis timestamp 2023-04-05T02:18:09 UTC.
- The block log and state history logs need to be replayed from the snapshot time and need to be saved together in the periodic backup.
- You need to keep the block logs, state-history logs starting from the snapshot point. This is because eos-evm-node may ask for old blocks for replaying the EVM chain.
- You can download the snapshot from any public EOS snapshot service providers (such as https://snapshots.eosnation.io/), or use your own snapshot.
- Supported version: spring 1.0 or newer versions
example data-dir/config.ini
chain-state-db-size-mb = 184320
access-control-allow-credentials = false
allowed-connection = any
p2p-listen-endpoint = 0.0.0.0:9876
p2p-max-nodes-per-host = 10
http-server-address = 0.0.0.0:8888
state-history-endpoint = 0.0.0.0:8999
trace-history = true
chain-state-history = false
http-max-response-time-ms = 1000
# add or remove peers if needed
p2p-peer-address=eos.p2p.eosusa.io:9882
p2p-peer-address=p2p.eos.cryptolions.io:9876
p2p-peer-address=p2p.eossweden.se:9876
p2p-peer-address=fullnode.eoslaomao.com:443
p2p-peer-address=mainnet.eosamsterdam.net:9876
# Plugin(s) to enable, may be specified multiple times
plugin = eosio::producer_plugin
plugin = eosio::chain_api_plugin
plugin = eosio::http_plugin
plugin = eosio::producer_api_plugin
plugin = eosio::state_history_plugin
plugin = eosio::net_plugin
plugin = eosio::net_api_plugin
plugin = eosio::db_size_api_plugin
example run command (VM1, head or speculative mode):
./nodeos --p2p-accept-transactions=0 --data-dir=./data-dir --config-dir=./data-dir --http-max-response-time-ms=200 --disable-replay-opts --max-body-size=10000000
example run command (VM3, irreversible mode):
./nodeos --read-mode=irreversible --p2p-accept-transactions=0 --data-dir=./data-dir --config-dir=./data-dir --http-max-response-time-ms=200 --disable-replay-opts --max-body-size=10000000
Notes:
- To boost performance, it is important to set "--p2p-accept-transactions=0" to disallow executing transactions (which are not yet included in a blocks) received from other peers.
- for the 1st time, run it also with
--snapshot=SNAPSHOT_FILEto begin with the snapshot state.
use ./cleos get info to check if the node the in-sync with the native network (via the head_block_time), for example:
./cleos get info
{
"server_version": "57465074",
"chain_id": "aca376f206b8fc25a6ed44dbdc66547c36c6c33e3a119ffbeaef943642f0e906",
"head_block_num": 399277762,
"last_irreversible_block_num": 399277760,
"last_irreversible_block_id": "17cc7ec09ef0f9d50c7a7da3751f370ae215912d1f4588b968f4e1d14a257858",
"head_block_id": "17cc7ec2498e4dae12046843502ec8c88bfdbee801b96aa1ba58ea6e3f2508d6",
"head_block_time": "2024-10-14T02:30:34.000",
"head_block_producer": "big.one",
"virtual_block_cpu_limit": 200000,
"virtual_block_net_limit": 1048576000,
"block_cpu_limit": 200000,
"block_net_limit": 1048576,
"server_version_string": "v1.0.1",
"fork_db_head_block_num": 399277762,
"fork_db_head_block_id": "17cc7ec2498e4dae12046843502ec8c88bfdbee801b96aa1ba58ea6e3f2508d6",
"server_full_version_string": "v1.0.1-574650744460373f635d48cac9aa6dee67dcbfdb",
"total_cpu_weight": "382730335491165",
"total_net_weight": "95742723072332",
"earliest_available_block_num": 399104422,
"last_irreversible_block_time": "2024-10-14T02:30:33.000"
}
use curl http://127.0.0.1:8888/v1/db_size/get, for example
curl http://127.0.0.1:8888/v1/db_size/get 2>/dev/null | jq
{
"free_bytes": "92051578720",
"used_bytes": "101221948576",
"reclaimable_bytes": 157837488,
"size": "193273527296",
- Copy the mainnet EOS-EVM genesis from https://github.com/eosnetworkfoundation/evm-public-docs/blob/main/mainnet-genesis.json
- run the eos-evm-node
mkdir ./chain-data
./eos-evm-node --ship-endpoint=<NODEOS_IP_ADDRESS>:8999 --ship-core-account eosio.evm --chain-data ./chain-data --plugin block_conversion_plugin --plugin blockchain_plugin --nocolor 1 --verbosity=4 --genesis-json=./genesis.json
- run the eos-evm-rpc (must be in the same VM as eos-evm-node)
./eos-evm-rpc --api-spec=eth,debug,net,trace --http-port=0.0.0.0:8881 --eos-evm-node=127.0.0.1:8080 --chaindata=./chain-data
- The EVM state, logs will be stored in ./chain-data directory
The eos-evm-rpc will talk to eos-evm-node and provide the eth compatible RPC services, for example, you can check the current block number of eos-evm-node via:
curl --location --request POST '127.0.0.1:8881/' --header 'Content-Type: application/json' --data-raw '{"method":"eth_blockNumber","params":["0x1",false],"id":0}'
example output:
{"id":0,"jsonrpc":"2.0","result":"0xa4e03"}
- if either spring or eos-evm-node can't start, follow the recovery process in the next session.
- It is quite important for node operator to backup all the state periodically (for example, once per day).
- backup must be done on the spring node running in irreversible mode. And because of such, all the blocks in eos-evm-node has been finialized and it will never has a fork.
- create the nodeos (spring) snapshot:
curl http://127.0.0.1:8888/v1/producer/create_snapshot - gracefull kill all processes:
pkill eos-evm-node
sleep 2.0
pkill eos-evm-rpc
sleep 2.0
pkill nodeos
- backup spring's data-dir folder and eos-evm-node's chain-data
- restart nodeos wait until the nodeos sync-up (use
./cleos get infoto verify) - restart eos-evm-node & eos-evm-rpc
Recover process:
- for spring recovery, please restore the data-dir folder of the last backup and use the spring's snapshot
- for eos-evm-node recovery, please restore the chain-data folder of the last backup.
The miner service will help to package the EVM transaction into EOS transaction and set to the EOS network. It will provide the following 2 eth API:
-
eth_gasPrice: retrieve the currect gas price from EOS Network
-
eth_sendRawTransaction: package the ETH transaction into EOS transaction and push into the EOS Network. clone the https://github.com/eosnetworkfoundation/eos-evm-miner repo
-
create your miner account (for example: a123) on EOS Network
-
open account balance on EVM side:
./cleos push action eosio.evm open '{"owner":"a123"}' -p a123 -
powerup the miner account with enough CPU & NET resource (for example: 1min CPU. 10 MB net per day). You can use some existing auto powerup service such as https://eospowerup.io/auto or push the powerup transaction (eosio::powerup) via cleos.
-
prepare the .env file with the correct information
PRIVATE_KEY=5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79zkvFD3
MINER_ACCOUNT=a123
RPC_ENDPOINTS=http://127.0.0.1:8888|http://192.168.1.1:8888
PORT=18888
LOCK_GAS_PRICE=false
MINER_PERMISSION=active
EXPIRE_SEC=300
- build and start the miner
npm install
yarn build
yarn start
- test if the miner is working
curl http://127.0.0.1:18888 -X POST -H "Accept: application/json" -H "Content-Type: application/json" --data '{"method":"eth_gasPrice","params":[],"id":1,"jsonrpc":"2.0"}'
{"jsonrpc":"2.0","id":1,"result":"0x22ecb25c00"}
[For centralized exchanges] Calculate the irreversible block number from EOS (L1) chain to EOS-EVM (L2) Chain
For centralized exchanges it is important to know up to which block number the chain is irreversible. This is the way to calculate the irreversible time of EOS-EVM:
- ensure the spring node & eos-evm-node are fully sync-up.
- do a get_info request to spring node.
{
"server_version": "943d1134",
"chain_id": "aca376f206b8fc25a6ed44dbdc66547c36c6c33e3a119ffbeaef943642f0e906",
"head_block_num": 316609050,
...
"earliest_available_block_num": 302853021,
"last_irreversible_block_time": "2023-06-23T03:10:35.500"
}
- in the above example all EVM blocks before
"last_irreversible_block_time": "2023-06-23T03:10:35.500"are irreversible. Use the time conversion script:python3 -c 'from datetime import datetime; print(hex(int((datetime.strptime("2023-06-23T03:10:35.500","%Y-%m-%dT%H:%M:%S.%f")-datetime(1970,1,1)).total_seconds())))'to get the EVM irreversible blocktime in hex0x64950d2b. As EVM block time is 1 second most of the time, exchanges or node operators can easily estimate the EVM block number given the block time. By locating the recent (around 180) EVM blocks, we found out that the EVM blocks up to6828746are irreversible, because its timestamp is0x64950d2b:
curl --location --request POST '127.0.0.1:8881/' --header 'Content-Type: application/json' --data-raw '{"method":"eth_getBlockByNumber","params":["6828746",false],"id":0}' {"id":0,"jsonrpc":"2.0","result":{"difficulty":"0x1","extraData":"0x","gasLimit":"0x7ffffffffff","gasUsed":"0x0","hash":"0x563fe6290cf38d55e4c4d2c86886032a1734ad1e467b7ce06ff52f12ee378b0d","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","miner":"0xbbbbbbbbbbbbbbbbbbbbbbbb5530ea015b900000","mixHash":"0x12df121840088703a9fe2f305eefe25dbe97bc57f7e127d922ffa8d005aceea6","nonce":"0x0000000000000000","number":"0x6832ca","parentHash":"0xafebdcf129bd506cee25892b2f20703e5ae98bd95557a04b91ac0f56a3433824","receiptsRoot":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x0000000000000000000000000000000000000000000000000000000000000000","size":"0x202","stateRoot":"0x0000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x64950d2b","totalDifficulty":"0x6832cb","transactions":[],"transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","uncles":[]}}
- please ensure the evm block has a non-empty "mixHash" field, which is important because it corresponds to the block ID of the EOS (L1) chain. If not, then go for the earlier EVM block until the block with mixHash is found, or simply wait sometime and do it again from the beginning.
- take the first 4 bytes of mixHash. In the above example it is 0x12df1218, convert to decimal number 316609048, which is the block number of the EOS (L1) chain.
- send a "get_block" request to nodeos and make sure the mixHash is equal to the block id:
./cleos get block 316609048
{
"timestamp": "2023-06-23T03:10:34.500",
"producer": "atticlabeosb",
"confirmed": 0,
"previous": "12df121796cd96182814f42440afe5d77924f53619bdd92c5855042a3df59516",
...
...
"id": "12df121840088703a9fe2f305eefe25dbe97bc57f7e127d922ffa8d005aceea6",
"block_num": 316609048,
"ref_block_prefix": 808451753
}
- If yes (which is in the above example), it means that the EVM chain is irreversible up to block 6828746 (where EOS block is 316609048 at 2023-06-23T03:10:34.500). If not, wait sometime until the EVM chain sync up and check again.
- For EOS tokens on EOS-EVM: Since this is the native token, similar to other ETH compatible networks, exchanges can use similar way to query EVM blocks (such as using eth_getBlockByNumber) up to the last irreversible EVM blocks as explained above. Or query the account balance using eth_getBalance if needed.
- For ERC20 tokens on EOS-EVM: Also similar to other ETH networks, exchanges can execute the ETH view action (eth_call) to extract the balance of any EVM account, or monitor each EVM blocks.
In order to monitoring fund withdrawal, exchanges need to consider:
- The
EXPIRE_SECvalue set in the eos-evm-miner. This value will control how long will the EOS trasaction expires in such a way that it will never be included in the blockchain after expiration. - The irreversible EVM block number.
For example:
-
- At 9:00:00AM UTC, the upstream signed the ETH transaction with ETH compatible private key and then call eth_sendRawTransaction
-
- The eos-evm-miner packages the EVM transaction into EOS transaction and signed it with EOS private key, and push to native EOS network.
-
- If
EXPIRE_SECis set to 60, the EOS transaction will expire at 9:01:00AM. So we need to wait until the result of./cleos get infoshows that the last_irreversible_block_time >= 9:01:00AM. At most cases, the EOS Network will have around 3 minute finality time, so we probably need to wait until 9:04:00AM.
- If
-
- Since all transactions up 9:01:00AM are irreversible, we scan each EVM block between 9:00:00AM and 9:01:01AM (1 sec max timestamp difference between EOS and EOS-EVM blocks) to confirm whether the transaction is included in the EVM blockchain (so as the native EOS blockchain). We can confirm the withdrawal is successfull if we find the transaction in this range. Otherwise, the transaction is already expired and can not be included in the blockchain.
-
- Alternative to 4, instead of scanning all blocks in the time range, we can get the nonce number of the EVM account to confirm if the withdrawal is successful. But this method only works if there is only one withdrawal pending under that EVM account.
Follow the build process in https://github.com/eosnetworkfoundation/eos-evm-node/tree/main/peripherals/proxy
The proxy program will separate Ethereum's write requests (such as eth_sendRawTransaction,eth_gasPrice) from other requests (treated as read requests). The write requests should go to Transaction Wrapper (which wrap the ETH transaction into Antelope transaction and sign it and push to the Antelope blockchain). The read requests should go to eos-evm-rpc.
In order to get it working, docker is required. To install docker in Linux, see https://docs.docker.com/engine/install/ubuntu/
cd eos-evm-node/peripherals/proxy/- Edit the file
nginx.conf, find the follow settings:
upstream write {
server 192.168.56.101:18888;
}
upstream read {
server 192.168.56.101:8881;
}- Change the IP and port of the write session to your Transaction Wrapper server endpoint.
- Change the IP and port of the read session to your eos-evm-rpc server endpoint
- Build the docker image for the proxy program:
sudo docker build .- Check the image ID after building the image
sudo docker image lsExample output:
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> 49564d312df7 2 hours ago 393MB
debian jessie 3aaeab7a4777 19 months ago 129MB- Run the proxy in docker:
sudo docker run -p 81:80 -v ${PWD}/nginx.conf:/etc/nginx.conf 49564d312df7In the commmand above we map the host port 81 to the port 80 inside the docker.
- Check if the proxy is responding:
echo "=== test 8881 rpc ==="
curl --location --request POST '127.0.0.1:8881/' --header 'Content-Type: application/json' --data-raw '{"method":"eth_blockNumber","params":["0x1",false],"id":0}'
echo "=== test 80 proxy->rpc ==="
curl --location --request POST '127.0.0.1:80/' --header 'Content-Type: application/json' --data-raw '{"method":"eth_blockNumber","params":["0x1",false],"id":0,"jsonrpc":"2.0"}'
echo "=== test 18888 wrapper ==="
curl http://127.0.0.1:18888 -X POST -H "Accept: application/json" -H "Content-Type: application/json" --data '{"method":"eth_gasPrice","params":[],"id":1,"jsonrpc":"2.0"}'
echo ""
echo "=== test 80 proxy->wrapper ==="
curl http://127.0.0.1:80 -X POST -H "Accept: application/json" -H "Content-Type: application/json" --data '{"method":"eth_gasPrice","params":[],"id":1,"jsonrpc":"2.0"}'
echo ""Example response:
=== test 8881 rpc ===
{"id":0,"jsonrpc":"2.0","result":"0x2f0d68f"}
=== test 80 proxy->rpc ===
{"id":0,"jsonrpc":"2.0","result":"0x2f0d68f"}
=== test 18888 wrapper ===
{"jsonrpc":"2.0","id":1,"result":"0x45d964b800"}
=== test 80 proxy->wrapper ===
{"jsonrpc":"2.0","id":1,"result":"0x45d964b800"}
You can now use endpoint http://127.0.0.1:80 in metamask for your own exSat EVM account operations.
The following python program provides a simple way for the proxy (change the READ_RPC_ENDPOINT, WRITE_RPC_ENDPOINT and SERVER_PORT if necessary):
#!/usr/bin/env python3
import random
import os
import json
import time
import calendar
from datetime import datetime
from flask import Flask, request, jsonify
from flask_cors import CORS
from eth_hash.auto import keccak
import requests
import json
from binascii import unhexlify
readEndpoint = os.getenv("READ_RPC_ENDPOINT","http://127.0.0.1:8881")
writeEndpoint = os.getenv("WRITE_RPC_ENDPOINT", "http://127.0.0.1:18888")
listenPort = os.getenv("SERVER_PORT", 5000)
ssl_cert = os.getenv("SSL_CERT","")
ssl_keyfile = os.getenv("SSL_KEYFILE","")
writemethods = {"eth_sendRawTransaction","eth_gasPrice"}
try:
app = Flask(__name__)
CORS(app)
@app.route("/", methods=["POST"])
def default():
def forward_request(req):
if type(req) == dict and ("method" in req) and (req["method"] in writemethods):
print("write req:" + str(req))
resp = requests.post(writeEndpoint, json.dumps(req), headers={"Accept":"application/json","Content-Type":"application/json"}).json()
print("write resp:" + str(resp))
return resp
else:
resp = requests.post(readEndpoint, json.dumps(req), headers={"Accept":"application/json","Content-Type":"application/json"}).json()
print("resp is:" + str(resp))
return resp;
request_data = request.get_json()
if type(request_data) == dict:
print("req is:" + str(request_data));
return jsonify(forward_request(request_data))
res = []
for r in request_data:
res.append(forward_request(r))
return jsonify(res)
if len(ssl_cert) > 0 and len(ssl_keyfile) > 0:
print("Running in SSL mode")
app.run(host='0.0.0.0',port=listenPort,ssl_context=(ssl_cert, ssl_keyfile))
else:
print("Running in non-SSL mode")
app.run(host='0.0.0.0', port=listenPort)
finally:
exit(0)
follow https://github.com/eosnetworkfoundation/eos-evm/blob/main/docs/local_testnet_deployment_plan.md to setup your own EOS-EVM Explorer
Sometime full EVM chain is required if there's a major version upgrade of eos-evm-node. This is the suggested replay process:
-
- Use the backup VM (in which spring node is running in irreversible mode so that it won't be any forks) for replaying
-
- Gracefully shutdown eos-evm-rpc & eos-evm-node, keep spring node running.
-
- Backup the eos-evm-node data folder (specified in --chain-data parameter).
-
- Delete everything in the --chain-data folder, but keep the folder itself
-
- Replace the eos-evm-node & eos-evm-rpc to the new version
-
- Start eos-evm-node & eos-evm-rpc again. Replay will be started automatically.
-
- Query the current replay process, normally replay will finished within hours:
curl --location --request POST '127.0.0.1:8881/' --header 'Content-Type: application/json' --data-raw '{"method":"eth_blockNumber","params":["0x1",false],"id":0}'
-
- After replay finishes, gracefully shutdown eos-evm-rpc & eos-evm-node. Make the evm backup of data-dir folder
-
- Apply the new binaries & the backup to other eos-evm node.
-
Eos-evm-node will gracefully stop if the state-history-plugin connection in spring node is dropped. Exchanges or node operators need to have auto-restart script to restart eos-evm-node (and choose the available spring end-point if high availability setup exist)
-
In some rare case, eos-evm-node can not handle forks happened in Native EOS (L1) chain. Exchanges or node operators may need to run the recovery process.
-
If eos-evm-node crashes, in some case it may not able to start due to database error. Exchanges or node operators may need to run the recovery process.