Merge branch 'main' into fix/bal-selfdestruct-netzero

Author: mh0lt

.claude/skills/erigondb-sync-integration-test-plan/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: erigondb-sync-integration-test-plan
+description: Integration test plan for erigondb.toml settings resolution across 3 runtime scenarios (legacy, fresh+downloader, fresh+no-downloader)
+allowed-tools: Bash, Read
+user-invocable: false
+---
+
+# erigondb.toml Integration Test Plan
+
+Verify that `erigondb.toml` settings are correctly resolved across three distinct runtime scenarios: legacy datadirs, fresh sync with downloader, and fresh sync without downloader.
+
+Use this plan after any changes to the erigondb.toml resolution logic (creation, defaults, legacy detection, downloader delivery) to confirm all paths still work.
+
+## Parameters
+
+Two values must be provided by the caller:
+
+| Parameter | Description | Example |
+|-----------|-------------|---------|
+| **chain** | Chain name passed to `--chain=` | `hoodi`, `mainnet` |
+| **legacy datadir** | Path to an existing Erigon datadir that has `preverified.toml` but **no** `erigondb.toml` | `~/eth-nodes/erigon33-hoodi-stable` |
+
+All scenarios use `--prune.mode=archive`.
+
+## Pre-requisites
+
+1. **Built binary** — invoke `/erigon-build` if `./build/bin/erigon` does not exist.
+2. **Legacy datadir** — the user-provided path must contain `snapshots/preverified.toml` and must **not** contain `snapshots/erigondb.toml`.
+3. **Port availability** — use `/erigon-network-ports` for reference. Each scenario uses a different port offset (+100, +200, +300) via `/erigon-ephemeral` conflict detection.
+
+## Scenario A: Legacy datadir
+
+**Goal**: Starting on a legacy datadir (has `preverified.toml`, no `erigondb.toml`) creates `erigondb.toml` with legacy settings (`step_size = 1562500`).
+
+### Steps
+
+1. Create an ephemeral clone of the user's legacy datadir using `/erigon-ephemeral` (Mode B: clone). This gives an isolated copy so the original is untouched.
+2. Confirm pre-conditions on the clone:
+   - `snapshots/preverified.toml` exists
+   - `snapshots/erigondb.toml` does **not** exist
+3. Start erigon with port offset **+100** (via `/erigon-ephemeral` port conflict detection):
+   ```
+   ./build/bin/erigon \
+     --datadir=<clone-path> \
+     --chain=<chain> \
+     --prune.mode=archive \
+     <port flags at +100 offset>
+   ```
+4. Wait for startup (~15s). Check logs for a message indicating legacy settings detection (e.g., `Creating erigondb.toml with LEGACY settings`).
+5. Verify the generated file:
+   ```bash
+   cat <clone-path>/snapshots/erigondb.toml
+   ```
+   Expected: `step_size = 1562500`
+6. Kill the process and clean up the ephemeral datadir.
+
+## Scenario B: Fresh sync with downloader
+
+**Goal**: A fresh datadir (no `preverified.toml`, no `erigondb.toml`) starts with code defaults (`step_size = 1562500`), then the downloader delivers the network's `erigondb.toml` during the header-chain phase, which may have different settings (e.g., `step_size = 390625` for hoodi).
+
+### Steps
+
+1. Create an empty ephemeral datadir using `/erigon-ephemeral` (Mode A: empty).
+2. Start erigon with port offset **+200**:
+   ```
+   ./build/bin/erigon \
+     --datadir=<empty-path> \
+     --chain=<chain> \
+     --prune.mode=archive \
+     <port flags at +200 offset>
+   ```
+3. Check early logs for a message indicating defaults are being used (e.g., `erigondb.toml not found, using defaults`) with `step_size=1562500`.
+4. Wait for the header-chain download phase to complete (~30-120s depending on network). Watch logs for:
+   - `[1/6 OtterSync] Downloader completed header-chain`
+   - `Reading DB settings from existing erigondb.toml`
+   - `erigondb stepSize changed, propagating` (if the network's settings differ from the code defaults)
+5. Verify the delivered file:
+   ```bash
+   cat <empty-path>/snapshots/erigondb.toml
+   ```
+   The values come from the network's published `erigondb.toml` (check the chain's webseed for the canonical values).
+6. Kill the process and clean up the ephemeral datadir.
+
+## Scenario C: Fresh sync with `--no-downloader`
+
+**Goal**: A fresh datadir with `--no-downloader` immediately writes `erigondb.toml` with code defaults (`step_size = 1562500`).
+
+### Steps
+
+1. Create an empty ephemeral datadir using `/erigon-ephemeral` (Mode A: empty).
+2. Start erigon with port offset **+300** and `--no-downloader`:
+   ```
+   ./build/bin/erigon \
+     --datadir=<empty-path> \
+     --chain=<chain> \
+     --prune.mode=archive \
+     --no-downloader \
+     <port flags at +300 offset>
+   ```
+3. Check logs for a message indicating defaults were written immediately (e.g., `Initializing erigondb.toml with DEFAULT settings`) with `step_size=1562500`.
+4. Verify the file exists immediately:
+   ```bash
+   cat <empty-path>/snapshots/erigondb.toml
+   ```
+   Expected: `step_size = 1562500` (code defaults, since no downloader to provide network settings)
+5. Kill the process and clean up the ephemeral datadir.
+
+## Success Criteria
+
+| Scenario | Condition | Expected step_size | File timing |
+|----------|-----------|-------------------|-------------|
+| A (legacy) | `Creating erigondb.toml with LEGACY settings` log | 1,562,500 | Written immediately on startup |
+| B (fresh+downloader) | `erigondb.toml not found, using defaults` then `erigondb stepSize changed, propagating` | Code default 1,562,500 then network value (chain-dependent) | After downloader delivers it |
+| C (fresh+no-downloader) | `Initializing erigondb.toml with DEFAULT settings (nodownloader)` log | 1,562,500 | Written immediately on startup |
+
+## Cleanup
+
+After all scenarios complete, ensure all ephemeral datadirs and processes are cleaned up. Use `/erigon-ephemeral` Step 5 (cleanup) for each instance, or Step 6 (leftover detection) to find any stragglers.

.github/workflows/scripts/run_rpc_tests_ethereum.sh

@@ -41,4 +41,4 @@ DISABLED_TEST_LIST=(
 DISABLED_TESTS=$(IFS=,; echo "${DISABLED_TEST_LIST[*]}")
 
 # Call the main test runner script with the required and optional parameters
-"$(dirname "$0")/run_rpc_tests.sh" mainnet v1.120.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR"
+"$(dirname "$0")/run_rpc_tests.sh" mainnet v1.121.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR"

.github/workflows/scripts/run_rpc_tests_ethereum_latest.sh

@@ -34,4 +34,4 @@ DISABLED_TEST_LIST=(
 DISABLED_TESTS=$(IFS=,; echo "${DISABLED_TEST_LIST[*]}")
 
 # Call the main test runner script with the required and optional parameters
-"$(dirname "$0")/run_rpc_tests.sh" mainnet v1.120.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR" "latest" "$REFERENCE_HOST" "do-not-compare-error-message" "$DUMP_RESPONSE"
+"$(dirname "$0")/run_rpc_tests.sh" mainnet v1.121.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR" "latest" "$REFERENCE_HOST" "do-not-compare-error-message" "$DUMP_RESPONSE"

.github/workflows/scripts/run_rpc_tests_gnosis.sh

@@ -22,5 +22,5 @@ DISABLED_TEST_LIST=(
 DISABLED_TESTS=$(IFS=,; echo "${DISABLED_TEST_LIST[*]}")
 
 # Call the main test runner script with the required and optional parameters
-"$(dirname "$0")/run_rpc_tests.sh" gnosis v1.120.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR"
+"$(dirname "$0")/run_rpc_tests.sh" gnosis v1.121.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR"
 

.github/workflows/scripts/run_rpc_tests_remote_ethereum.sh

@@ -28,4 +28,4 @@ DISABLED_TEST_LIST=(
 DISABLED_TESTS=$(IFS=,; echo "${DISABLED_TEST_LIST[*]}")
 
 # Call the main test runner script with the required and optional parameters
-"$(dirname "$0")/run_rpc_tests.sh" mainnet v1.120.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR"
+"$(dirname "$0")/run_rpc_tests.sh" mainnet v1.121.0 "$DISABLED_TESTS" "$WORKSPACE" "$RESULT_DIR"

.github/workflows/test-all-erigon.yml

@@ -119,7 +119,7 @@ jobs:
     concurrency:
       group: >-
         ${{
-          (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/heads/release/')) && 
+          (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/heads/release/')) &&
           format('{0}-{1}-{2}', github.workflow, matrix.os, github.run_id) ||
           format('{0}-{1}-{2}', github.workflow, matrix.os, github.ref)
         }}
@@ -130,6 +130,10 @@ jobs:
     runs-on:
       group: "selfhosted-win"
 
+    env:
+      GOMODCACHE: C:\go-cache\mod
+      GOCACHE: C:\go-cache\build
+
     steps:
       # not needed on self-hosted runner. already configured.
       #- name: Configure Pagefile
@@ -147,6 +151,12 @@ jobs:
           lfs: true
           clean: true
 
+      # Prune LFS blobs that are no longer referenced by the current checkout.
+      - name: Prune stale Git LFS objects
+        if: needs.source-of-changes.outputs.changed_files != 'true'
+        shell: bash
+        run: git lfs prune --verify-remote=false
+
       - name: Setup Go environment on ${{ matrix.os }}
         if: needs.source-of-changes.outputs.changed_files != 'true'
         uses: actions/setup-go@v6
@@ -161,15 +171,20 @@ jobs:
       - name: Run all tests on ${{ matrix.os }}
         if: needs.source-of-changes.outputs.changed_files != 'true'
         shell: bash
-        env:
-          GOMODCACHE: C:\go-cache\mod
-          GOCACHE: C:\go-cache\build
         run: make test-all
 
-      - name: Cleanup
+      # Clears the Go build cache after the run so compiled artifacts do not
+      # accumulate on the persistent self-hosted runner across runs.
+      # GOMODCACHE is intentionally kept: it is bounded by the number of unique
+      # dependency versions and saves re-downloading modules on the next run.
+      - name: Post-run cleanup
         if: always()
+        shell: bash
         run: |
-          Remove-Item -Path "$env:TEMP\*" -Recurse -Force -ErrorAction SilentlyContinue
+          go clean -cache
+          rm -f coverage-test-all.out
+          rm -rf "$TEMP/"* 2>/dev/null || true
+          rm -rf "$RUNNER_TEMP/"* 2>/dev/null || true
 
       - name: This ${{ matrix.os }} check does not make sense for changes within out-of-scope directories
         if: needs.source-of-changes.outputs.changed_files == 'true'

cl/p2p/p2p_localnode.go

@@ -26,20 +26,44 @@ func newLocalNode(
 	}
 	localNode := enode.NewLocalNode(db, privKey)
 
-	ipEntry := enr.IP(ipAddr)
 	udpEntry := enr.UDP(udpPort)
 	tcpEntry := enr.TCP(tcpPort)
 
-	localNode.Set(ipEntry)
 	localNode.Set(udpEntry)
 	localNode.Set(tcpEntry)
-
-	localNode.SetFallbackIP(ipAddr)
 	localNode.SetFallbackUDP(udpPort)
 
+	if ipAddr.IsUnspecified() {
+		if detected := detectOutboundIP(ipAddr); detected != nil {
+			logger.Info("[Caplin] Discovery address is unspecified, using detected outbound IP for ENR. Set --caplin.discovery.addr explicitly to override", "detected", detected)
+			ipAddr = detected
+		} else {
+			logger.Warn("[Caplin] Discovery address is unspecified and outbound IP detection failed, ENR will have no IP. Set --caplin.discovery.addr to your public IP")
+		}
+	}
+	if !ipAddr.IsUnspecified() {
+		localNode.Set(enr.IP(ipAddr))
+		localNode.SetFallbackIP(ipAddr)
+	}
+
 	return localNode, nil
 }
 
+// detectOutboundIP determines the preferred outbound IP address by asking the
+// OS routing table (no actual traffic is sent). Returns nil if detection fails.
+func detectOutboundIP(unspecified net.IP) net.IP {
+	network, target := "udp4", "8.8.8.8:80"
+	if unspecified.To4() == nil {
+		network, target = "udp6", "[2001:4860:4860::8888]:80"
+	}
+	conn, err := net.Dial(network, target)
+	if err != nil {
+		return nil
+	}
+	defer conn.Close()
+	return conn.LocalAddr().(*net.UDPAddr).IP
+}
+
 func NewUDPv5Listener(ctx context.Context, cfg *P2PConfig, discCfg discover.Config, logger log.Logger) (*discover.UDPv5, error) {
 	var (
 		ipAddr = cfg.IpAddr

cl/sentinel/sentinel.go

@@ -274,21 +274,22 @@ func (s *Sentinel) Identity() (pid, enrStr string, p2pAddresses, discoveryAddres
 	}
 	discoveryAddresses = []string{}
 
-	if s.listener.LocalNode().Node().TCP() != 0 {
+	nodeIP := s.listener.LocalNode().Node().IP()
+	if nodeIP == nil {
+		s.logger.Warn("[Sentinel] Discovery node has nil IP address, skipping discovery address advertisement. Check caplin.discovery.addr configuration and host IPv6 setup")
+	} else {
 		protocol := "ip4"
-		if s.listener.LocalNode().Node().IP().To4() == nil {
+		if nodeIP.To4() == nil {
 			protocol = "ip6"
 		}
-		port := s.listener.LocalNode().Node().TCP()
-		discoveryAddresses = append(discoveryAddresses, fmt.Sprintf("/%s/%s/tcp/%d/p2p/%s", protocol, s.listener.LocalNode().Node().IP(), port, pid))
-	}
-	if s.listener.LocalNode().Node().UDP() != 0 {
-		protocol := "ip4"
-		if s.listener.LocalNode().Node().IP().To4() == nil {
-			protocol = "ip6"
+		if s.listener.LocalNode().Node().TCP() != 0 {
+			port := s.listener.LocalNode().Node().TCP()
+			discoveryAddresses = append(discoveryAddresses, fmt.Sprintf("/%s/%s/tcp/%d/p2p/%s", protocol, nodeIP, port, pid))
+		}
+		if s.listener.LocalNode().Node().UDP() != 0 {
+			port := s.listener.LocalNode().Node().UDP()
+			discoveryAddresses = append(discoveryAddresses, fmt.Sprintf("/%s/%s/udp/%d/p2p/%s", protocol, nodeIP, port, pid))
 		}
-		port := s.listener.LocalNode().Node().UDP()
-		discoveryAddresses = append(discoveryAddresses, fmt.Sprintf("/%s/%s/udp/%d/p2p/%s", protocol, s.listener.LocalNode().Node().IP(), port, pid))
 	}
 	subnetField := bitfield.NewBitvector64()
 	syncnetField := bitfield.NewBitvector8()

cmd/integration/commands/stages.go

@@ -1133,7 +1133,11 @@ func allSnapshots(ctx context.Context, db kv.RoDB, logger log.Logger) (*freezebl
 		blockReader := freezeblocks.NewBlockReader(_allSnapshotsSingleton, _allBorSnapshotsSingleton)
 		txNums := blockReader.TxnumReader()
 
-		_aggSingleton = dbstate.New(dirs).Logger(logger).MustOpen(ctx, db)
+		var erigonDBSettings *dbstate.ErigonDBSettings
+		if erigonDBSettings, err = dbstate.ResolveErigonDBSettings(dirs, logger, false); err != nil {
+			return
+		}
+		_aggSingleton = dbstate.New(dirs).Logger(logger).WithErigonDBSettings(erigonDBSettings).MustOpen(ctx, db)
 
 		_aggSingleton.SetProduceMod(snapCfg.ProduceE3)
 

cmd/mcp/README.md

@@ -118,41 +118,46 @@ for datadir mode.
 ## Available Tools
 
 ### Ethereum Standard (eth_*)
+
 `eth_blockNumber`, `eth_getBlockByNumber`, `eth_getBlockByHash`,
 `eth_getBalance`, `eth_getTransactionByHash`, `eth_getTransactionReceipt`,
 `eth_getBlockReceipts`, `eth_getLogs`, `eth_getCode`, `eth_getStorageAt`,
 `eth_getTransactionCount`, `eth_call`, `eth_estimateGas`, `eth_gasPrice`,
 `eth_chainId`, `eth_syncing`, `eth_getProof`, and more.
 
 ### Erigon-Specific (erigon_*)
+
 `erigon_forks`, `erigon_blockNumber`, `erigon_getHeaderByNumber`,
 `erigon_getHeaderByHash`, `erigon_getBlockByTimestamp`,
 `erigon_getBalanceChangesInBlock`, `erigon_getLogsByHash`,
 `erigon_getLogs`, `erigon_getBlockReceiptsByBlockHash`, `erigon_nodeInfo`.
 
 ### Otterscan (ots_*)
+
 `ots_getApiLevel`, `ots_getInternalOperations`,
 `ots_searchTransactionsBefore`, `ots_searchTransactionsAfter`,
 `ots_getBlockDetails`, `ots_getBlockTransactions`, `ots_hasCode`,
 `ots_traceTransaction`, `ots_getTransactionError`,
 `ots_getTransactionBySenderAndNonce`, `ots_getContractCreator`.
 
 ### Log Analysis
+
 `logs_tail`, `logs_head`, `logs_grep`, `logs_stats` — requires `--log.dir`
 or `--datadir` to locate Erigon/torrent log files.
 
 ### Metrics
+
 `metrics_list`, `metrics_get` — only available in embedded mode (inside Erigon).
 In standalone mode, these return an informational message.
 
 ## Flags
 
-| Flag | Default | Description |
-|------|---------|-------------|
-| `--rpc.url` | `http://127.0.0.1:8545` | Erigon JSON-RPC endpoint URL |
-| `--port` | 0 | JSON-RPC port shorthand |
-| `--datadir` | | Erigon data directory (enables direct DB mode) |
-| `--private.api.addr` | `127.0.0.1:9090` | gRPC private API (with --datadir) |
-| `--transport` | `stdio` | Transport: `stdio` or `sse` |
-| `--sse.addr` | `127.0.0.1:8553` | SSE listen address |
-| `--log.dir` | | Log directory (overrides datadir detection) |
+| Flag                 | Default                 | Description                                    |
+|----------------------|-------------------------|------------------------------------------------|
+| `--rpc.url`          | `http://127.0.0.1:8545` | Erigon JSON-RPC endpoint URL                   |
+| `--port`             | 0                       | JSON-RPC port shorthand                        |
+| `--datadir`          |                         | Erigon data directory (enables direct DB mode) |
+| `--private.api.addr` | `127.0.0.1:9090`        | gRPC private API (with --datadir)              |
+| `--transport`        | `stdio`                 | Transport: `stdio` or `sse`                    |
+| `--sse.addr`         | `127.0.0.1:8553`        | SSE listen address                             |
+| `--log.dir`          |                         | Log directory (overrides datadir detection)    |