chore: add sync strategies #437

shane-moore · 2025-05-06T04:21:47Z

Changes

added how full sync and snap sync function

Note

this should probably go in the devp2p section. I'm thinking we make a gh issue to stating to move this context over there once the open devp2p PR's are closed out

github-actions · 2025-05-06T04:22:17Z

Hi @shane-moore,

⚠️ Potential typos found in your pull request:

📄 ./docs/wiki/EL/el-architecture.md (line(s) 187):
1. ❌ trie’s
📄 ./docs/wiki/EL/el-architecture.md (line(s) 324):
1. ❌ Taverna
📄 ./docs/wiki/EL/el-architecture.md (line(s) 324):
1. ❌ Synchronising
📄 ./docs/wiki/pectra-faq.md (line(s) 152):
1. ❌ consolidatations
📄 ./docs/wiki/pectra-faq.md (line(s) 152):
1. ❌ BBdDc
📄 ./docs/wiki/pectra-faq.md (line(s) 152):
1. ❌ fb
📄 ./docs/wiki/pectra-faq.md (line(s) 198):
1. ❌ maxmium

ℹ️ How to fix this error:

If these are actual typos:
- Open the files at the specified line numbers and fix them
If these are names or one-off nouns:
- Wrap them in <name> tags
- Example: <name>Alex Pereira</name>
- Use this for people's names or unique terms that appear rarely
If these are valid terms:
- Add them to wordlist.txt (one word per line)
- Terms must be plain text without spaces/special chars
- The list is case-insensitive
If these are code terms:
- Wrap them in backticks (`) in your markdown

ℹ️ Checking for typos locally

Install aspell for your platform
Navigate to project root and run:

for f in **/*.md ; do echo $f ; aspell --lang=en_US --mode=markdown --home-dir=. --personal=wordlist.txt --ignore-case=true --camel-case --add-sgml-skip nospellcheck list < $f | sort | uniq -c ; done

Learn more about wordlist format

github-actions · 2025-05-06T04:26:55Z

Hi @shane-moore,

⚠️ Potential typos found in your pull request:

📄 ./docs/wiki/pectra-faq.md (line(s) 152):
1. ❌ BBdDc
📄 ./docs/wiki/pectra-faq.md (line(s) 152):
1. ❌ fb

ℹ️ How to fix this error:

If these are actual typos:
- Open the files at the specified line numbers and fix them
If these are names or one-off nouns:
- Wrap them in <name> tags
- Example: <name>Alex Pereira</name>
- Use this for people's names or unique terms that appear rarely
If these are valid terms:
- Add them to wordlist.txt (one word per line)
- Terms must be plain text without spaces/special chars
- The list is case-insensitive
If these are code terms:
- Wrap them in backticks (`) in your markdown

ℹ️ Checking for typos locally

Install aspell for your platform
Navigate to project root and run:

for f in **/*.md ; do echo $f ; aspell --lang=en_US --mode=markdown --home-dir=. --personal=wordlist.txt --ignore-case=true --camel-case --add-sgml-skip nospellcheck list < $f | sort | uniq -c ; done

Learn more about wordlist format

shane-moore · 2025-05-06T04:31:54Z

@taxmeifyoucan @raxhvl, this PR is ready for review whenever you get a chance. I recall from the wiki contributors call a couple weeks ago that we wanted to split up PR reviewers by categories, but I don't remember who was assigned to which category, i.e. EL like for this PR

taxmeifyoucan

Thanks Shane!

taxmeifyoucan · 2025-05-06T06:43:07Z

docs/wiki/EL/el-architecture.md

+Let's look at how the flows of both **full sync** and **snap sync** work.

+### Full Sync
+Full Sync trades absolute trustlessness for time and resources. You replay every block and transaction from genesis to tip, rebuilding the state trie step by step:


Suggested change

Full Sync trades absolute trustlessness for time and resources. You replay every block and transaction from genesis to tip, rebuilding the state trie step by step:

Full Sync trades absolute trustlessness for time and resources. The client replays every block and transaction from genesis to tip, rebuilding the state trie step by step:

I would maybe add that full sync is the default naive strategy and because it's starting from genesis, it's not sustainable for the future (4444)

solid point! added 4444 context in latest commit 2bde312

Other thoughts I didn't include in the PR:
On top of 4444, once we get statelessness, syncing should be significantly sped up, since each block will have execution witnesses, containing each account and storage slots accessed, and block validation can just run txs against an in-memory state trie created from those accessed accounts and storage slots.
^^ maybe too early to add this context to the sync wiki section but def going to be a game changer

taxmeifyoucan · 2025-05-06T06:45:34Z

docs/wiki/EL/el-architecture.md

+5. Send `GetByteCodes` for every code hash found in the account bodies to get back the contract code for that account
+6. Locally insert every fetched leaf into a fresh trie, verifying each batch against the pivot root via Merkle proofs.
+
+After step 6, we have a snapshot of the pivot block's state. However, the blockchain is also progressing at the same time and invalidating some of the regenerated state data. This means it is also necessary to have a healing phase where errors in the state are fixed.  During healing, the client walks the reconstructed trie and verifies that it hashes to the pivot block’s stateRoot. Any missing or inconsistent trie nodes or bytecode blobs are fetched using targeted snap requests. Healing ensures the final state is complete and cryptographically valid, even if parts of the snapshot were stale, incomplete, or assembled from mixed sources.  


One small thing I would highlight here is that snapshot has a specific database like format, different from state merkle so it needs to be converted

no doubt @taxmeifyoucan, added this context in 2bde312

taxmeifyoucan · 2025-05-06T06:45:44Z

docs/wiki/EL/el-architecture.md

 - [Engine Api Spec](https://github.com/ethereum/execution-apis/blob/main/src/engine/paris.md#payload-validation) • [archived](https://web.archive.org/web/20250318111700/https://github.com/ethereum/execution-apis/blob/main/src/engine/paris.md#payload-validation)
 - [Engine API: A Visual Guide](https://hackmd.io/@danielrachi/engine_api) • [archived](https://web.archive.org/web/20241006232802/https://hackmd.io/@danielrachi/engine_api)
 - [Engine API | Mikhail | Lecture 21](https://youtu.be/fR7LBXAMH7g)
+- ["Snapping Snap Sync: Practical Attacks on Go Ethereum Synchronising Nodes" (ETH Zurich)](https://appliedcrypto.ethz.ch/content/dam/ethz/special-interest/infk/inst-infsec/appliedcrypto/research/TavernaPaterson-SnappingSnapSync.pdf)


also eth.org has these https://ethereum.org/en/developers/docs/nodes-and-clients/#execution-layer-sync-modes

nice source, added as well 2bde312

shane-moore force-pushed the chore/sync-strategies branch from 5c398ed to 3bdda2b Compare May 6, 2025 04:26

taxmeifyoucan approved these changes May 6, 2025

View reviewed changes

chore: add sync strategies

f169c17

shane-moore force-pushed the chore/sync-strategies branch from 3bdda2b to f169c17 Compare May 7, 2025 17:22

chore: updates per pr review

2bde312

taxmeifyoucan merged commit 69c2502 into eth-protocol-fellows:main May 16, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: add sync strategies #437

chore: add sync strategies #437

Uh oh!

shane-moore commented May 6, 2025

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

shane-moore commented May 6, 2025

Uh oh!

taxmeifyoucan left a comment

Uh oh!

taxmeifyoucan May 6, 2025

Uh oh!

taxmeifyoucan May 6, 2025

Uh oh!

shane-moore May 7, 2025

Uh oh!

taxmeifyoucan May 6, 2025 •

edited

Loading

Uh oh!

shane-moore May 7, 2025 •

edited

Loading

Uh oh!

taxmeifyoucan May 6, 2025

Uh oh!

shane-moore May 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	Full Sync trades absolute trustlessness for time and resources. You replay every block and transaction from genesis to tip, rebuilding the state trie step by step:
	Full Sync trades absolute trustlessness for time and resources. The client replays every block and transaction from genesis to tip, rebuilding the state trie step by step:

chore: add sync strategies #437

chore: add sync strategies #437

Uh oh!

Conversation

shane-moore commented May 6, 2025

Changes

Note

Uh oh!

github-actions bot commented May 6, 2025

ℹ️ How to fix this error:

ℹ️ Checking for typos locally

Uh oh!

github-actions bot commented May 6, 2025

ℹ️ How to fix this error:

ℹ️ Checking for typos locally

Uh oh!

shane-moore commented May 6, 2025

Uh oh!

taxmeifyoucan left a comment

Choose a reason for hiding this comment

Uh oh!

taxmeifyoucan May 6, 2025

Choose a reason for hiding this comment

Uh oh!

taxmeifyoucan May 6, 2025

Choose a reason for hiding this comment

Uh oh!

shane-moore May 7, 2025

Choose a reason for hiding this comment

Uh oh!

taxmeifyoucan May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shane-moore May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taxmeifyoucan May 6, 2025

Choose a reason for hiding this comment

Uh oh!

shane-moore May 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

taxmeifyoucan May 6, 2025 •

edited

Loading

shane-moore May 7, 2025 •

edited

Loading