renaming datadir (seems?) to have restarted the whole sync #7990

mans17 · 2023-08-10T12:29:05Z

mans17
Aug 10, 2023

Hi, I'm pretty new here.
I've synced the görli network (AFAIK), logs told me :

[INFO] [08-08|00:20:12.142] [13/15 LogIndex] Progress                number=8497812 alloc=3.3GB sys=6.3GB
[INFO] [08-08|00:20:20.531] [13/15 LogIndex] Flushed buffer file     name=/home/data-enc/etherum/10-execution-ERIGON/temp/erigon-sortable-buf-466108629
[INFO] [08-08|00:20:24.269] [13/15 LogIndex] Flushed buffer file     name=/home/data-enc/etherum/10-execution-ERIGON/temp/erigon-sortable-buf-3560227779
[INFO] [08-08|00:20:30.474] [p2p] GoodPeers                          eth68=77 eth67=70 eth66=5
[INFO] [08-08|00:20:30.486] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=4.1GB sys=6.3GB
[INFO] [08-08|00:20:54.270] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=000000000000000000000000000000000000000000000000000000000000643cffffffff
[INFO] [08-08|00:21:24.270] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=000000000000000000000000000000000000000000000000000000000001c554ffffffff
[INFO] [08-08|00:21:30.487] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=2.2GB sys=6.3GB
[INFO] [08-08|00:21:54.270] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=00000000000000000000000000000000000000000000000000000000000cc167ffffffff
[INFO] [08-08|00:22:24.273] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=0000000000000000000000000a8156e7ee392d885d10eaa86afd0e323afdcd95001d200a
[INFO] [08-08|00:22:30.478] [p2p] GoodPeers                          eth66=5 eth68=77 eth67=70
[INFO] [08-08|00:22:30.493] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=3.4GB sys=6.3GB
[INFO] [08-08|00:22:54.269] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=0000000000000000000000004e36e298808813a0a4b6ae3a97a945e62912327effffffff
[INFO] [08-08|00:23:24.269] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=000000000000000000000000a0a2815496bb19495aa31261d481279c749a879cffffffff
[INFO] [08-08|00:23:30.488] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=2.3GB sys=6.3GB
[INFO] [08-08|00:23:54.269] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=000000000000000000000000e6ce978338260b4e1ac7cfc6ad60114360d50a22ffffffff
[INFO] [08-08|00:24:24.271] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=21bf80de
[INFO] [08-08|00:24:30.474] [p2p] GoodPeers                          eth68=77 eth67=70 eth66=5
[INFO] [08-08|00:24:30.489] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=3.3GB sys=6.3GB
[INFO] [08-08|00:24:54.270] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=6135b536
[INFO] [08-08|00:25:24.271] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=9744d0a1
[INFO] [08-08|00:25:30.487] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=2.7GB sys=6.3GB
[INFO] [08-08|00:25:54.270] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=db80dd48
[INFO] [08-08|00:26:16.088] [13/15 LogIndex] Flushed buffer file     name=/home/data-enc/etherum/10-execution-ERIGON/temp/erigon-sortable-buf-2265096545
[INFO] [08-08|00:26:30.480] [p2p] GoodPeers                          eth68=77 eth67=70 eth66=5
[INFO] [08-08|00:26:30.488] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=4.0GB sys=6.3GB
[INFO] [08-08|00:26:46.093] [13/15 LogIndex] ETL [2/2] Loading       into=LogAddressIndex current_prefix=335e2d61
[INFO] [08-08|00:27:16.089] [13/15 LogIndex] ETL [2/2] Loading       into=LogAddressIndex current_prefix=6f3a314c
[INFO] [08-08|00:27:30.487] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=3.7GB sys=6.3GB
[INFO] [08-08|00:27:46.096] [13/15 LogIndex] ETL [2/2] Loading       into=LogAddressIndex current_prefix=b807b3d3
[INFO] [08-08|00:28:16.090] [13/15 LogIndex] ETL [2/2] Loading       into=LogAddressIndex current_prefix=f65032bf
[INFO] [08-08|00:28:27.657] [13/15 LogIndex] DONE                    in=49m45.905639229s
[INFO] [08-08|00:28:27.672] Timings (slower than 50ms)               Snapshots=825ms Headers=1.715s Execution=7h12m33.251s HashState=18m18.194s IntermediateHashes=9m57.859s CallTraces=33m57.269s AccountHistoryIndex=16m42.492s StorageHistoryIndex=1h2m39.646s LogIndex=49m45.905s
[INFO] [08-08|00:28:27.676] [2/15 Headers] Waiting for Consensus Layer...
[INFO] [08-08|00:28:30.475] [p2p] GoodPeers                          eth68=77 eth66=5 eth67=70
[INFO] [08-08|00:28:30.486] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=2.5GB sys=6.3GB
[INFO] [08-08|00:29:30.486] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=2.8GB sys=6.3GB
[INFO] [08-08|00:30:30.474] [p2p] GoodPeers                          eth68=76 eth67=69 eth66=5
[INFO] [08-08|00:30:30.487] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.1GB sys=6.3GB
[INFO] [08-08|00:31:30.486] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.5GB sys=6.3GB
[INFO] [08-08|00:32:30.475] [p2p] GoodPeers                          eth67=69 eth66=5 eth68=77
[INFO] [08-08|00:32:30.498] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.2GB sys=6.3GB
[INFO] [08-08|00:33:30.488] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.5GB sys=6.3GB
[INFO] [08-08|00:34:30.474] [p2p] GoodPeers                          eth66=5 eth68=77 eth67=70
[INFO] [08-08|00:34:30.496] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.2GB sys=6.3GB
[INFO] [08-08|00:35:30.487] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.5GB sys=6.3GB
[INFO] [08-08|00:36:30.474] [p2p] GoodPeers                          eth68=77 eth67=70 eth66=5
[INFO] [08-08|00:36:30.487] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.1GB sys=6.3GB
[INFO] [08-08|00:37:30.489] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.4GB sys=6.3GB
[INFO] [08-08|00:38:30.475] [p2p] GoodPeers                          eth68=77 eth67=70 eth66=5
[INFO] [08-08|00:38:30.488] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.1GB sys=6.3GB
[INFO] [08-08|00:39:30.489] [txpool] stat                            pending=10000 baseFee=0 queued=30000 alloc=1.5GB sys=6.3GB

Then, I figured out I messed up an "etherum" directory, and was like "let's change it to etherEum".
I THINK I was clever enough to stop erigon, THEN rename datadir then restart, but I might have let erigon run a few secs after renaming (created the problem ?).

/usr/bin/erigon --datadir=/home/data-enc/etherum/10-execution-ERIGON --chain=goerli
## was fine, let's rename !
/usr/bin/erigon --datadir=/home/data-enc/ethereum/10-execution-ERIGON --chain=goerli
### oh, no ! restart syncing !

1/ Is it OK to rename/move datadir, usually ?
2/ Logs during redownload did not tell something weird IMHO, here is "full" log with snaps on irrelevant parts.

[INFO] [08-09|18:04:02.924] [p2p] GoodPeers                          eth68=43 eth67=41 eth66=1
[INFO] [08-09|18:04:02.944] [txpool] stat                            pending=665 baseFee=0 queued=19848 alloc=1.5GB sys=1.9GB
[INFO] [08-09|18:05:02.941] [txpool] stat                            pending=670 baseFee=0 queued=19950 alloc=1.3GB sys=1.9GB
[INFO] [08-09|18:06:02.924] [p2p] GoodPeers                          eth66=1 eth68=43 eth67=43
[INFO] [08-09|18:06:02.943] [txpool] stat                            pending=679 baseFee=0 queued=20054 alloc=1.3GB sys=1.9GB
[INFO] [08-09|18:07:02.941] [txpool] stat                            pending=683 baseFee=0 queued=20150 alloc=1.6GB sys=1.9GB
[INFO] [08-09|18:08:02.925] [p2p] GoodPeers                          eth68=42 eth67=43 eth66=1
[INFO] [08-09|18:08:02.945] [txpool] stat                            pending=690 baseFee=0 queued=20247 alloc=1.2GB sys=1.9GB
[INFO] [08-09|18:08:41.349] Got interrupt, shutting down...
[INFO] [08-09|18:08:41.349] Exiting...
[EROR] [08-09|18:08:41.349] Staged Sync                              err="[2/15 Headers] server is stopping"
[INFO] [08-09|18:08:41.349] RPC server shutting down
[INFO] [08-09|18:08:41.351] HTTP endpoint closed                     url=127.0.0.1:8545
[INFO] [08-09|18:08:41.349] Exiting Engine...
[INFO] [08-09|18:08:41.351] RPC server shutting down
[INFO] [08-09|18:08:41.351] RPC server shutting down
[INFO] [08-09|18:08:41.351] Engine HTTP endpoint close               url=127.0.0.1:8551
[INFO] [08-09|18:08:42.861] logging to file system                   log dir=/home/data-enc/ethereum/10-execution-ERIGON/logs file prefix=erigon log level=info json=false
[INFO] [08-09|18:08:42.861] Build info                               git_branch=master git_tag=v2.48.0 git_commit=d487ecffc43239313d4590866a86b6048296b148
[INFO] [08-09|18:08:42.861] Starting Erigon on Görli testnet...
[INFO] [08-09|18:08:42.863] Maximum peer count                       ETH=100 total=100
[INFO] [08-09|18:08:42.863] starting HTTP APIs                       APIs=eth,erigon,engine
[INFO] [08-09|18:08:42.863] torrent verbosity                        level=WRN
[INFO] [08-09|18:08:45.194] [torrent] Public IP                      ip=82.65.187.169
[INFO] [08-09|18:08:45.198] Set global gas cap                       cap=50000000
[INFO] [08-09|18:08:45.204] [Downloader] Runnning with               ipv6-enabled=true ipv4-enabled=true download.rate=16mb upload.rate=4mb
[INFO] [08-09|18:08:45.206] Opening Database                         label=chaindata path=/home/data-enc/ethereum/10-execution-ERIGON/chaindata
[INFO] [08-09|18:08:46.284] Initialised chain configuration          config="{ChainID: 5, Homestead: 0, DAO: <nil>, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 1561651, Muir Glacier: <nil>, Berlin: 4460644, London: 5062605, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: 10790000, Merge Netsplit: <nil>, Shanghai: 1678832736, Cancun: <nil>, Prague: <nil>, Engine: clique}" genesis=0xbf7e331f7f7c1dd2e05159666b3bf8bc7a8a3a9eb1d518969eab529dd9b88c1a
[INFO] [08-09|18:08:46.884] Initialising Ethereum protocol           network=5
[INFO] [08-09|18:08:46.892] Starting private RPC server              on=127.0.0.1:9090
[INFO] [08-09|18:08:46.892] new subscription to logs established
[INFO] [08-09|18:08:46.893] rpc filters: subscribing to Erigon events
[INFO] [08-09|18:08:46.893] New txs subscriber joined
[INFO] [08-09|18:08:46.893] new subscription to newHeaders established
[INFO] [08-09|18:08:46.893] Reading JWT secret                       path=/home/data-enc/ethereum/10-execution-ERIGON/jwt.hex
[INFO] [08-09|18:08:46.894] HTTP endpoint opened for Engine API      url=127.0.0.1:8551 ws=true ws.compression=true
[INFO] [08-09|18:08:46.894] HTTP endpoint opened                     url=127.0.0.1:8545 ws=false ws.compression=true grpc=false
[INFO] [08-09|18:08:46.902] Started P2P networking                   version=67 self=enode://d511d4af3384c1d3f3eaae8c21d4207dfc0ea5ef900bb940aa41948f57000984e0d645c1b5f93a4ff6b0264965c3276ef3d4fc31877be1ac6f18a260b14f5ba4@82.65.187.169:30304 name=erigon/v2.48.0-stable-d487ecff/linux-amd64/go1.20.7
[INFO] [08-09|18:08:46.904] [1/15 Snapshots] Fetching torrent files metadata
[INFO] [08-09|18:08:46.905] Started P2P networking                   version=68 self=enode://d511d4af3384c1d3f3eaae8c21d4207dfc0ea5ef900bb940aa41948f57000984e0d645c1b5f93a4ff6b0264965c3276ef3d4fc31877be1ac6f18a260b14f5ba4@127.0.0.1:0 name=erigon/v2.48.0-stable-d487ecff/linux-amd64/go1.20.7
[INFO] [08-09|18:08:46.936] [txpool] Started
[INFO] [08-09|18:08:47.305] Mapped network port                      proto=tcp extport=30304 intport=30304 interface="UPNP IGDv1-IP1"
[INFO] [08-09|18:08:47.662] [snapshots] Blocks Stat                  blocks=8500k indices=8500k alloc=820.5MB sys=1.3GB
[INFO] [08-09|18:08:48.313] Mapped network port                      proto=udp extport=30304 intport=30304 interface="UPNP IGDv1-IP1"
[INFO] [08-09|18:08:48.515] Mapped network port                      proto=tcp extport=30303 intport=30303 interface="UPNP IGDv1-IP1"
[INFO] [08-09|18:08:48.717] Mapped network port                      proto=udp extport=30303 intport=30303 interface="UPNP IGDv1-IP1"
[INFO] [08-09|18:08:49.049] [2/15 Headers] Waiting for Consensus Layer...
[INFO] [08-09|18:09:46.894] [txpool] stat                            pending=2 baseFee=0 queued=75 alloc=719.1MB sys=1.3GB
[INFO] [08-09|18:10:46.880] [p2p] GoodPeers                          eth67=1
[INFO] [08-09|18:10:46.893] [txpool] stat                            pending=5 baseFee=0 queued=164 alloc=928.7MB sys=1.3GB
[INFO] [08-09|18:11:46.893] [txpool] stat                            pending=5 baseFee=0 queued=384 alloc=724.7MB sys=1.3GB
[...]
[INFO] [08-09|18:34:46.894] [txpool] stat                            pending=105 baseFee=0 queued=16417 alloc=1.0GB sys=1.4GB
[INFO] [08-09|18:35:46.894] [txpool] stat                            pending=108 baseFee=0 queued=16513 alloc=801.8MB sys=1.4GB
[INFO] [08-09|18:36:46.880] [p2p] GoodPeers                          eth68=12 eth67=15 eth66=1
[INFO] [08-09|18:36:46.897] [txpool] stat                            pending=111 baseFee=0 queued=16609 alloc=1.0GB sys=1.4GB
[INFO] [08-09|18:37:30.120] [2/15 Headers] Downloading PoS headers... height=unknown hash=0x7d342f255abd202d5563d793614625447579c03fdc0d1d7633dc101a89ea7c95 requestId=1
[INFO] [08-09|18:37:30.120] [2/15 Headers] DONE                      in=28m42.458383504s
[INFO] [08-09|18:37:30.139] Timings (slower than 50ms)               Snapshots=758ms Headers=28m42.458s
[INFO] [08-09|18:37:30.139] [2/15 Headers] Postponing PoS download since another one is in progress height=0 hash=0x7d342f255abd202d5563d793614625447579c03fdc0d1d7633dc101a89ea7c95
[INFO] [08-09|18:37:46.893] [txpool] stat                            pending=115 baseFee=0 queued=16693 alloc=1014.2MB sys=1.4GB
[INFO] [08-09|18:38:17.085] [downloader] Downloaded PoS Headers      now=9471969 blk/sec=422.400
[INFO] [08-09|18:38:46.879] [p2p] GoodPeers                          eth67=15 eth66=1 eth68=13
[INFO] [08-09|18:38:46.897] [txpool] stat                            pending=121 baseFee=0 queued=16829 alloc=887.5MB sys=1.6GB
[INFO] [08-09|18:38:47.378] [downloader] Downloaded PoS Headers      now=9450273 blk/sec=723.200
[INFO] [08-09|18:39:16.953] [downloader] Downloaded PoS Headers      now=9407649 blk/sec=1420.800
[INFO] [08-09|18:39:46.893] [txpool] stat                            pending=125 baseFee=0 queued=16932 alloc=1.0GB sys=1.8GB
[INFO] [08-09|18:39:46.917] [downloader] Downloaded PoS Headers      now=9327585 blk/sec=2668.800
[INFO] [08-09|18:40:16.918] [downloader] Downloaded PoS Headers      now=9223905 blk/sec=3456.000
[...]
[INFO] [08-09|18:43:50.880] [5/15 Bodies] Processing bodies...       from=8499999 to=9489441
[INFO] [08-09|18:44:10.881] [5/15 Bodies] Downloading block bodies   block_num=8500087 delivery/sec=3.0MB wasted/sec=0B remaining=989353 delivered=651 cache=53.6MB alloc=1022.1MB sys=2.5GB
[INFO] [08-09|18:44:30.883] [5/15 Bodies] Downloading block bodies   block_num=8500087 delivery/sec=12.3MB wasted/sec=0B remaining=989353 delivered=3136 cache=256.0MB alloc=1.7GB sys=2.5GB
[INFO] [08-09|18:44:46.879] [p2p] GoodPeers                          eth68=12 eth67=19 eth66=1

Answered by AskAlexSharov

Aug 10, 2023

can see progress in "integration print_stages" or in eth_syncing
we using transactional db - and not-very-often doing Commit - some recent progress may be lost.
erigon always start from step 1 - it's expected

View full answer

AskAlexSharov · 2023-08-10T14:35:54Z

AskAlexSharov
Aug 10, 2023
Maintainer

can see progress in "integration print_stages" or in eth_syncing
we using transactional db - and not-very-often doing Commit - some recent progress may be lost.
erigon always start from step 1 - it's expected

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

renaming datadir (seems?) to have restarted the whole sync #7990

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

renaming datadir (seems?) to have restarted the whole sync #7990

Uh oh!

mans17 Aug 10, 2023

Replies: 1 comment

Uh oh!

AskAlexSharov Aug 10, 2023 Maintainer

mans17
Aug 10, 2023

AskAlexSharov
Aug 10, 2023
Maintainer