db/state, snapcfg: download erigondb.toml via torrent; move settings resolution out of Open()#19587
Merged
AskAlexSharov merged 6 commits intomainfrom Mar 4, 2026
Merged
Conversation
…ut of Open() Move the ResolveErigonDBSettings call from inside Open() to each caller, making settings resolution explicit at each call site. This lets the node/eth/backend.go caller pass the real noDownloader flag, while all other callers pass false. Add a pure setter builder WithErigonDBSettings(*ErigonDBSettings) on AggOpts that assigns the resolved settings to opts.stepSize and opts.stepsInFrozenFile. Open() no longer does settings I/O — it just uses whatever is on opts. If a caller forgets the builder, stepSize=0 causes ConfigureDomains to fail fast. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
wmitsuda
requested review from
AskAlexSharov,
Giulio2002,
canepat,
lupin012,
mh0lt,
sudeepdino008 and
yperbasis
as code owners
- Remove ReloadErigonDBSettings/ConfigureDomains from OpenFolder() since they overwrote test-specified stepSize with disk defaults; initial setup is done in Open(), live reload is explicit in stage_snapshots.go - Add StepSize default to NewTest() so callers without explicit .StepSize() get a valid value - Move AdjustReceiptCurrentVersionIfNeeded to Open() before capturing schema, so savedSchema reflects the adjusted receipt versions - Delegate execctx NewTest() to state.NewTest() to avoid divergence
All consumers of statecfg.Schema read the global directly. The savedSchema copy on Aggregator created an illusion of isolation that didn't exist. Move AdjustReceiptCurrentVersionIfNeeded into ConfigureDomains() right before Configure() where the schema is consumed.
Reusable test plan for verifying erigondb.toml settings resolution across 3 scenarios: legacy datadir, fresh sync with downloader, and fresh sync with --no-downloader.
… use fsync Remove dead CreateOrReadErigonDBSettings (replaced by ResolveErigonDBSettings in step_cmd.go with noDownloader=true). Extract readErigonDBSettings and writeErigonDBSettings helpers to consolidate the TOML marshal/unmarshal patterns. Use dir.WriteFileWithFsync for durable writes, consistent with salt and registry files. Reorder ResolveErigonDBSettings to return early when erigondb.toml exists, skipping the unnecessary preverified.toml stat.
Member
Author
|
@anacrolix I'd appreciate your review on this one as well, since it touches some codepaths involving the sync process |
AskAlexSharov
approved these changes
Mar 4, 2026
anacrolix
reviewed
Mar 4, 2026
| return nil, err | ||
| return err | ||
| } | ||
| return dir.WriteFileWithFsync(path, data, 0644) |
Contributor
There was a problem hiding this comment.
I think there's a const for this somewhere to avoid inconsistencies
Member
Author
There was a problem hiding this comment.
can't find it, actually I see plenty of 0644 hardcoded in the code
Contributor
|
I put some questions in Discord |
sudeepdino008
pushed a commit
that referenced
this pull request
Mar 4, 2026
…resolution out of Open() (#19587) ## Summary - Move `ResolveErigonDBSettings` out of `Aggregator.Open()` so callers resolve settings explicitly before constructing the aggregator, enabling the downloader to provide the real `erigondb.toml` before domains are configured. - Whitelist `erigondb.toml` in the header-chain download phase so the torrent downloader delivers it alongside headers/bodies. - After the header-chain download completes, `ReloadErigonDBSettings()` re-reads the file and propagates any stepSize changes to all Domain/InvertedIndex instances. - Add `WithErigonDBSettings()` builder on `AggOpts` so all call sites pass pre-resolved settings. Three runtime scenarios handled: 1. **Legacy datadir** (has `preverified.toml`, no `erigondb.toml`): writes `erigondb.toml` with legacy settings (step_size=1,562,500) on startup. 2. **Fresh sync with downloader**: starts with defaults, downloader delivers real `erigondb.toml` during header-chain phase, settings are reloaded and propagated. 3. **Fresh sync with `--no-downloader`**: writes defaults to disk immediately since no downloader will provide the file. ## Test plan - [x] Scenario 1 (legacy datadir): confirmed log `Creating erigondb.toml with LEGACY settings step_size=1562500` and file written on startup - [x] Scenario 2 (fresh + downloader): confirmed log `erigondb stepSize changed, propagating` after header-chain download delivers `erigondb.toml` with step_size=390625 - [x] Scenario 3 (fresh + `--no-downloader`): confirmed log `Initializing erigondb.toml with DEFAULT settings (nodownloader)` and file written immediately --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
sudeepdino008
pushed a commit
that referenced
this pull request
Mar 4, 2026
…resolution out of Open() (#19587) ## Summary - Move `ResolveErigonDBSettings` out of `Aggregator.Open()` so callers resolve settings explicitly before constructing the aggregator, enabling the downloader to provide the real `erigondb.toml` before domains are configured. - Whitelist `erigondb.toml` in the header-chain download phase so the torrent downloader delivers it alongside headers/bodies. - After the header-chain download completes, `ReloadErigonDBSettings()` re-reads the file and propagates any stepSize changes to all Domain/InvertedIndex instances. - Add `WithErigonDBSettings()` builder on `AggOpts` so all call sites pass pre-resolved settings. Three runtime scenarios handled: 1. **Legacy datadir** (has `preverified.toml`, no `erigondb.toml`): writes `erigondb.toml` with legacy settings (step_size=1,562,500) on startup. 2. **Fresh sync with downloader**: starts with defaults, downloader delivers real `erigondb.toml` during header-chain phase, settings are reloaded and propagated. 3. **Fresh sync with `--no-downloader`**: writes defaults to disk immediately since no downloader will provide the file. ## Test plan - [x] Scenario 1 (legacy datadir): confirmed log `Creating erigondb.toml with LEGACY settings step_size=1562500` and file written on startup - [x] Scenario 2 (fresh + downloader): confirmed log `erigondb stepSize changed, propagating` after header-chain download delivers `erigondb.toml` with step_size=390625 - [x] Scenario 3 (fresh + `--no-downloader`): confirmed log `Initializing erigondb.toml with DEFAULT settings (nodownloader)` and file written immediately --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ResolveErigonDBSettingsout ofAggregator.Open()so callers resolve settings explicitly before constructing the aggregator, enabling the downloader to provide the realerigondb.tomlbefore domains are configured.erigondb.tomlin the header-chain download phase so the torrent downloader delivers it alongside headers/bodies.ReloadErigonDBSettings()re-reads the file and propagates any stepSize changes to all Domain/InvertedIndex instances.WithErigonDBSettings()builder onAggOptsso all call sites pass pre-resolved settings.Three runtime scenarios handled:
preverified.toml, noerigondb.toml): writeserigondb.tomlwith legacy settings (step_size=1,562,500) on startup.erigondb.tomlduring header-chain phase, settings are reloaded and propagated.--no-downloader: writes defaults to disk immediately since no downloader will provide the file.Test plan
Creating erigondb.toml with LEGACY settings step_size=1562500and file written on startuperigondb stepSize changed, propagatingafter header-chain download deliverserigondb.tomlwith step_size=390625--no-downloader): confirmed logInitializing erigondb.toml with DEFAULT settings (nodownloader)and file written immediately