Skip to content

Commit 7e71936

Browse files
committed
Merge branch 'main' into bal-devnet-2
2 parents b4ef1b4 + 58dba7c commit 7e71936

File tree

95 files changed

+3325
-1435
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

95 files changed

+3325
-1435
lines changed
Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
---
2+
name: erigondb-sync-integration-test-plan
3+
description: Integration test plan for erigondb.toml settings resolution across 3 runtime scenarios (legacy, fresh+downloader, fresh+no-downloader)
4+
allowed-tools: Bash, Read
5+
user-invocable: false
6+
---
7+
8+
# erigondb.toml Integration Test Plan
9+
10+
Verify that `erigondb.toml` settings are correctly resolved across three distinct runtime scenarios: legacy datadirs, fresh sync with downloader, and fresh sync without downloader.
11+
12+
Use this plan after any changes to the erigondb.toml resolution logic (creation, defaults, legacy detection, downloader delivery) to confirm all paths still work.
13+
14+
## Parameters
15+
16+
Two values must be provided by the caller:
17+
18+
| Parameter | Description | Example |
19+
|-----------|-------------|---------|
20+
| **chain** | Chain name passed to `--chain=` | `hoodi`, `mainnet` |
21+
| **legacy datadir** | Path to an existing Erigon datadir that has `preverified.toml` but **no** `erigondb.toml` | `~/eth-nodes/erigon33-hoodi-stable` |
22+
23+
All scenarios use `--prune.mode=archive`.
24+
25+
## Pre-requisites
26+
27+
1. **Built binary** — invoke `/erigon-build` if `./build/bin/erigon` does not exist.
28+
2. **Legacy datadir** — the user-provided path must contain `snapshots/preverified.toml` and must **not** contain `snapshots/erigondb.toml`.
29+
3. **Port availability** — use `/erigon-network-ports` for reference. Each scenario uses a different port offset (+100, +200, +300) via `/erigon-ephemeral` conflict detection.
30+
31+
## Scenario A: Legacy datadir
32+
33+
**Goal**: Starting on a legacy datadir (has `preverified.toml`, no `erigondb.toml`) creates `erigondb.toml` with legacy settings (`step_size = 1562500`).
34+
35+
### Steps
36+
37+
1. Create an ephemeral clone of the user's legacy datadir using `/erigon-ephemeral` (Mode B: clone). This gives an isolated copy so the original is untouched.
38+
2. Confirm pre-conditions on the clone:
39+
- `snapshots/preverified.toml` exists
40+
- `snapshots/erigondb.toml` does **not** exist
41+
3. Start erigon with port offset **+100** (via `/erigon-ephemeral` port conflict detection):
42+
```
43+
./build/bin/erigon \
44+
--datadir=<clone-path> \
45+
--chain=<chain> \
46+
--prune.mode=archive \
47+
<port flags at +100 offset>
48+
```
49+
4. Wait for startup (~15s). Check logs for a message indicating legacy settings detection (e.g., `Creating erigondb.toml with LEGACY settings`).
50+
5. Verify the generated file:
51+
```bash
52+
cat <clone-path>/snapshots/erigondb.toml
53+
```
54+
Expected: `step_size = 1562500`
55+
6. Kill the process and clean up the ephemeral datadir.
56+
57+
## Scenario B: Fresh sync with downloader
58+
59+
**Goal**: A fresh datadir (no `preverified.toml`, no `erigondb.toml`) starts with code defaults (`step_size = 1562500`), then the downloader delivers the network's `erigondb.toml` during the header-chain phase, which may have different settings (e.g., `step_size = 390625` for hoodi).
60+
61+
### Steps
62+
63+
1. Create an empty ephemeral datadir using `/erigon-ephemeral` (Mode A: empty).
64+
2. Start erigon with port offset **+200**:
65+
```
66+
./build/bin/erigon \
67+
--datadir=<empty-path> \
68+
--chain=<chain> \
69+
--prune.mode=archive \
70+
<port flags at +200 offset>
71+
```
72+
3. Check early logs for a message indicating defaults are being used (e.g., `erigondb.toml not found, using defaults`) with `step_size=1562500`.
73+
4. Wait for the header-chain download phase to complete (~30-120s depending on network). Watch logs for:
74+
- `[1/6 OtterSync] Downloader completed header-chain`
75+
- `Reading DB settings from existing erigondb.toml`
76+
- `erigondb stepSize changed, propagating` (if the network's settings differ from the code defaults)
77+
5. Verify the delivered file:
78+
```bash
79+
cat <empty-path>/snapshots/erigondb.toml
80+
```
81+
The values come from the network's published `erigondb.toml` (check the chain's webseed for the canonical values).
82+
6. Kill the process and clean up the ephemeral datadir.
83+
84+
## Scenario C: Fresh sync with `--no-downloader`
85+
86+
**Goal**: A fresh datadir with `--no-downloader` immediately writes `erigondb.toml` with code defaults (`step_size = 1562500`).
87+
88+
### Steps
89+
90+
1. Create an empty ephemeral datadir using `/erigon-ephemeral` (Mode A: empty).
91+
2. Start erigon with port offset **+300** and `--no-downloader`:
92+
```
93+
./build/bin/erigon \
94+
--datadir=<empty-path> \
95+
--chain=<chain> \
96+
--prune.mode=archive \
97+
--no-downloader \
98+
<port flags at +300 offset>
99+
```
100+
3. Check logs for a message indicating defaults were written immediately (e.g., `Initializing erigondb.toml with DEFAULT settings`) with `step_size=1562500`.
101+
4. Verify the file exists immediately:
102+
```bash
103+
cat <empty-path>/snapshots/erigondb.toml
104+
```
105+
Expected: `step_size = 1562500` (code defaults, since no downloader to provide network settings)
106+
5. Kill the process and clean up the ephemeral datadir.
107+
108+
## Success Criteria
109+
110+
| Scenario | Condition | Expected step_size | File timing |
111+
|----------|-----------|-------------------|-------------|
112+
| A (legacy) | `Creating erigondb.toml with LEGACY settings` log | 1,562,500 | Written immediately on startup |
113+
| B (fresh+downloader) | `erigondb.toml not found, using defaults` then `erigondb stepSize changed, propagating` | Code default 1,562,500 then network value (chain-dependent) | After downloader delivers it |
114+
| C (fresh+no-downloader) | `Initializing erigondb.toml with DEFAULT settings (nodownloader)` log | 1,562,500 | Written immediately on startup |
115+
116+
## Cleanup
117+
118+
After all scenarios complete, ensure all ephemeral datadirs and processes are cleaned up. Use `/erigon-ephemeral` Step 5 (cleanup) for each instance, or Step 6 (leftover detection) to find any stragglers.

.github/workflows/qa-constrained-tip-tracking.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ jobs:
2222
runs-on: [ self-hosted, qa, "${{ matrix.backend }}", tip-tracking ]
2323
timeout-minutes: 780 # 13 hours
2424
env:
25-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version-main/datadir
25+
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
2626
ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir
2727
ERIGON_QA_PATH: /home/qarunner/erigon-qa
2828
TRACKING_TIME_SECONDS: 7200 # 2 hours

.github/workflows/qa-rpc-integration-tests-gnosis.yml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ jobs:
2929
cancel-in-progress: true
3030
runs-on: [ self-hosted, qa, Gnosis, rpc-integration ]
3131
env:
32-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/gnosis-reference-version/datadir
3332
ERIGON_TESTBED_AREA: /opt/erigon-testbed
3433
ERIGON_QA_PATH: /home/qarunner/erigon-qa
3534
ERIGON_ASSERT: true
@@ -40,6 +39,21 @@ jobs:
4039
- name: Check out repository
4140
uses: actions/checkout@v6
4241

42+
- name: Set reference data dir based on branch
43+
run: |
44+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
45+
BRANCH="${{ github.base_ref }}"
46+
if [ -z "$BRANCH" ]; then
47+
BRANCH="${{ github.ref }}"
48+
BRANCH="${BRANCH#refs/heads/}"
49+
fi
50+
if [[ "$BRANCH" == release/* ]]; then
51+
VERSION="${BRANCH#release/}"
52+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/gnosis-reference-version-${VERSION}/datadir" >> $GITHUB_ENV
53+
else
54+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/gnosis-reference-version/datadir" >> $GITHUB_ENV
55+
fi
56+
4357
- name: Clean Erigon Build Directory
4458
run: |
4559
make clean

.github/workflows/qa-rpc-integration-tests-latest.yml

Lines changed: 19 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,11 @@ on:
88
type: boolean
99
required: false
1010
default: false
11+
schedule:
12+
- cron: '0 0 * * *' # Run nightly at 00:00 AM UTC
1113
push:
1214
branches:
13-
- main
14-
pull_request:
15-
branches:
16-
- main
17-
types:
18-
- opened
19-
- reopened
20-
- synchronize
21-
- ready_for_review
22-
15+
- 'release/3.*'
2316

2417
jobs:
2518
mainnet-rpc-integ-tests-latest:
@@ -33,10 +26,9 @@ jobs:
3326
cancel-in-progress: true
3427
runs-on: [ self-hosted, qa, Ethereum, rpc-latest-erigon ]
3528
env:
36-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
3729
ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir
3830
ERIGON_QA_PATH: /home/qarunner/erigon-qa
39-
TOTAL_TIME_SECONDS: 900 # 15 minutes
31+
TOTAL_TIME_SECONDS: 1800 # 30 minutes
4032
REFERENCE_SYSTEM_HOST_ADDRESS: 157.180.55.78:8545
4133
ERIGON_ASSERT: true
4234
RPC_PAST_TEST_DIR: /opt/rpc-past-tests
@@ -46,6 +38,21 @@ jobs:
4638
- name: Check out repository
4739
uses: actions/checkout@v6
4840

41+
- name: Set reference data dir based on branch
42+
run: |
43+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
44+
BRANCH="${{ github.base_ref }}"
45+
if [ -z "$BRANCH" ]; then
46+
BRANCH="${{ github.ref }}"
47+
BRANCH="${BRANCH#refs/heads/}"
48+
fi
49+
if [[ "$BRANCH" == release/* ]]; then
50+
VERSION="${BRANCH#release/}"
51+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/reference-version-${VERSION}/datadir" >> $GITHUB_ENV
52+
else
53+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/reference-version/datadir" >> $GITHUB_ENV
54+
fi
55+
4956
- name: Clean Erigon Build Directory
5057
run: |
5158
make clean

.github/workflows/qa-rpc-integration-tests-remote.yml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ jobs:
2929
cancel-in-progress: true
3030
runs-on: [ self-hosted, qa, Ethereum, rpc-integration-commitment ]
3131
env:
32-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
3332
ERIGON_TESTBED_AREA: /opt/erigon-testbed
3433
ERIGON_QA_PATH: /home/qarunner/erigon-qa
3534
ERIGON_ASSERT: true
@@ -40,6 +39,21 @@ jobs:
4039
- name: Check out repository
4140
uses: actions/checkout@v6
4241

42+
- name: Set reference data dir based on branch
43+
run: |
44+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
45+
BRANCH="${{ github.base_ref }}"
46+
if [ -z "$BRANCH" ]; then
47+
BRANCH="${{ github.ref }}"
48+
BRANCH="${BRANCH#refs/heads/}"
49+
fi
50+
if [[ "$BRANCH" == release/* ]]; then
51+
VERSION="${BRANCH#release/}"
52+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/reference-version-${VERSION}/datadir" >> $GITHUB_ENV
53+
else
54+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/reference-version/datadir" >> $GITHUB_ENV
55+
fi
56+
4357
- name: Clean Erigon Build Directory
4458
run: |
4559
make clean

.github/workflows/qa-rpc-integration-tests.yml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ jobs:
2929
cancel-in-progress: true
3030
runs-on: [ self-hosted, qa, Ethereum, rpc-integration-commitment ]
3131
env:
32-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
3332
ERIGON_TESTBED_AREA: /opt/erigon-testbed
3433
ERIGON_QA_PATH: /home/qarunner/erigon-qa
3534
ERIGON_ASSERT: true
@@ -40,6 +39,21 @@ jobs:
4039
- name: Check out repository
4140
uses: actions/checkout@v6
4241

42+
- name: Set reference data dir based on branch
43+
run: |
44+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
45+
BRANCH="${{ github.base_ref }}"
46+
if [ -z "$BRANCH" ]; then
47+
BRANCH="${{ github.ref }}"
48+
BRANCH="${BRANCH#refs/heads/}"
49+
fi
50+
if [[ "$BRANCH" == release/* ]]; then
51+
VERSION="${BRANCH#release/}"
52+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/reference-version-${VERSION}/datadir" >> $GITHUB_ENV
53+
else
54+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/reference-version/datadir" >> $GITHUB_ENV
55+
fi
56+
4357
- name: Clean Erigon Build Directory
4458
run: |
4559
make clean

.github/workflows/qa-rpc-performance-comparison-tests.yml

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,14 +96,29 @@ jobs:
9696
env:
9797
RUN_ID: ${{ needs.setup.outputs.run_id }}
9898
NETWORK: ${{ needs.setup.outputs.network }}
99-
ERIGON_REFERENCE_DIR: /opt/erigon-versions/reference-version
100-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
10199
ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir
102100
TOTAL_TIME_SECONDS: 1200 # 20 minutes
103101
GETH_INSTALL_DIR: /opt/go-ethereum
104102
CLIENT: ${{ matrix.client }}
105103

106104
steps:
105+
- name: Set reference data dir based on branch
106+
run: |
107+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
108+
BRANCH="${{ github.base_ref }}"
109+
if [ -z "$BRANCH" ]; then
110+
BRANCH="${{ github.ref }}"
111+
BRANCH="${BRANCH#refs/heads/}"
112+
fi
113+
if [[ "$BRANCH" == release/* ]]; then
114+
VERSION="${BRANCH#release/}"
115+
ERIGON_REFERENCE_DIR="/opt/erigon-versions/reference-version-${VERSION}"
116+
else
117+
ERIGON_REFERENCE_DIR="/opt/erigon-versions/reference-version"
118+
fi
119+
echo "ERIGON_REFERENCE_DIR=${ERIGON_REFERENCE_DIR}" >> $GITHUB_ENV
120+
echo "ERIGON_REFERENCE_DATA_DIR=${ERIGON_REFERENCE_DIR}/datadir" >> $GITHUB_ENV
121+
107122
- name: Checkout Erigon repository
108123
if: matrix.client == 'erigon'
109124
uses: actions/checkout@v6

.github/workflows/qa-rpc-performance-tests.yml

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,14 +87,29 @@ jobs:
8787
env:
8888
RUN_ID: ${{ needs.setup.outputs.run_id }}
8989
NETWORK: ${{ needs.setup.outputs.network }}
90-
ERIGON_REFERENCE_DIR: /opt/erigon-versions/reference-version
91-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
9290
ERIGON_TESTBED_AREA: /opt/erigon-testbed
9391
ERIGON_QA_PATH: /home/qarunner/erigon-qa
9492
GETH_INSTALL_DIR: /opt/go-ethereum
9593
CLIENT: ${{ matrix.client }}
9694

9795
steps:
96+
- name: Set reference data dir based on branch
97+
run: |
98+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
99+
BRANCH="${{ github.base_ref }}"
100+
if [ -z "$BRANCH" ]; then
101+
BRANCH="${{ github.ref }}"
102+
BRANCH="${BRANCH#refs/heads/}"
103+
fi
104+
if [[ "$BRANCH" == release/* ]]; then
105+
VERSION="${BRANCH#release/}"
106+
ERIGON_REFERENCE_DIR="/opt/erigon-versions/reference-version-${VERSION}"
107+
else
108+
ERIGON_REFERENCE_DIR="/opt/erigon-versions/reference-version"
109+
fi
110+
echo "ERIGON_REFERENCE_DIR=${ERIGON_REFERENCE_DIR}" >> $GITHUB_ENV
111+
echo "ERIGON_REFERENCE_DATA_DIR=${ERIGON_REFERENCE_DIR}/datadir" >> $GITHUB_ENV
112+
98113
- name: Checkout Erigon repository
99114
if: matrix.client == 'erigon'
100115
uses: actions/checkout@v6

.github/workflows/qa-tip-tracking-gnosis.yml

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ jobs:
1717
runs-on: [self-hosted, qa, Gnosis, tip-tracking]
1818
timeout-minutes: 1200
1919
env:
20-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version-3.3/datadir
2120
ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir
2221
ERIGON_QA_PATH: /home/qarunner/erigon-qa
2322
TRACKING_TIME_SECONDS: 7200 # 2 hours
@@ -28,6 +27,24 @@ jobs:
2827
- name: Check out repository
2928
uses: actions/checkout@v6
3029

30+
- name: Set reference data dir based on branch
31+
run: |
32+
# For pull_request events base_ref is the target branch name; for push/dispatch parse from ref.
33+
BRANCH="${{ github.base_ref }}"
34+
if [ -z "$BRANCH" ]; then
35+
BRANCH="${{ github.ref }}"
36+
BRANCH="${BRANCH#refs/heads/}"
37+
fi
38+
if [[ "$BRANCH" == release/* ]]; then
39+
VERSION="${BRANCH#release/}"
40+
MAJOR="${VERSION%.*}"
41+
MINOR="${VERSION##*.}"
42+
PREV_VERSION="${MAJOR}.$((MINOR - 1))"
43+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/gnosis-reference-version-${PREV_VERSION}/datadir" >> $GITHUB_ENV
44+
else
45+
echo "ERIGON_REFERENCE_DATA_DIR=/opt/erigon-versions/gnosis-reference-version/datadir" >> $GITHUB_ENV
46+
fi
47+
3148
- name: Clean Erigon Build Directory
3249
run: |
3350
make clean

.github/workflows/qa-tip-tracking-with-load.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ jobs:
4747
runs-on: ${{ matrix.runs-on }}
4848
timeout-minutes: 300 # 5 hours
4949
env:
50-
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version-main/datadir
50+
ERIGON_REFERENCE_DATA_DIR: /opt/erigon-versions/reference-version/datadir
5151
ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir
5252
ERIGON_QA_PATH: /home/qarunner/erigon-qa
5353
TRACKING_TIME_SECONDS: 7200 # 2 hours

0 commit comments

Comments
 (0)