History Network Fully Operational Meta Tracking Issue #398
Description
Motivations
We are on the cusp of the Portal History Network being fully operational. A meta coordination issue should ensure all independent teams can track the progress of clients and infrastructure as they go live.
Breakdown
The History Network going live can be broken into 2 parts
a.The infrastructure to keep the network live and healthy - i.e. bridgesb.Support for the specifications within the clients themselves
Required Header Validation methods
1.pre-merge HistoricalAccumulators2.post-merge to pre-capella HistoricalRoots3.post-capella to pre-deneb HistoricalSummariesCapella4.post-deneb to (onward minus 8192 blocks) HistoricalSummariesDeneb5.ephemeral headers last 8192 blocks
Plan for a. bridging infrastructure
- for
1 to 4, the Trin Team already has infrastructure for this. The bridge will be exclusively using E2HS files, no live providers.- We will have a script which adds the new E2HS file https://e2hs.ethportal.net/index.html every time there are 8192 execution blocks finalized from the point of view of HistoricalSummaries. HistoricalSummaries don't line up 1 for 1, so most E2HS files will include proofs from at least 2 different accumulators, this is only not the case for pre-merge E2HS files as HistoricalAccumulators line up 1 for 1.
- for
5.. I am building an ephemeral headers bridge which will be compliant with the specification for ephemeral offer's
Other teams are welcome to develop their own supporting infrastructure/bridges, if a team is planning to build one please share it in your respective teams update. Developing a bridge is an extra development burden, since Trin's Team has the most resources it is assumed that they take on this initial burden. As we ship code into production their may be more time for other teams to develop their own infrastructure for supporting the History Network. For full roll out of the History Network, the infrastructure built by the Trin Team is sufficient, any team is free to work on this, but for all teams completing b. should be the highest priority.
Plan for b. Clients being compliant with the History Network Specifications.
- For clients to support
3 to 5they must implement some kind of Oracle such as the Portal Beacon Network, alternative approaches exist for Portal Clients being ran within an Execution Client, but for running a Portal Client independently it is expected clients support thePortal Beacon Network - for
1 to 4- Clients should support full Validation from Genesis Onwards using HeaderWithProofs so from Frontier to Pectra
- for
5.clients must implement the History Network Ephemeral specifications - On the full roll out of the History Network it is expected that all clients perform full validation. Meaning any content they store is validated for Canonicalness
Testing
Currently we have no negative tests on Hive this is bad because it means we have no signal if clients properly implement validation. I want to finish this off, but my bandwidth has been limited due to higher priorities such as getting the History/State Network's into production.
Call to Action
All Teams should post a response which outlines their plans on where they are and what they still need to complete to be fully compliant with the full History Network Specification. It would be productive if each team links to an issue on their own repo which contains a checklist of tasks to do to support the Full History Network.
An example list can be and roughly the one I made for Trin
## Trin's Full History Network Tracking Issue
### `z.` Bridging infrastructure
- [x] E2HS Bridge
- [x] Generate E2HS files for Genesis to pre-capella
- [ ] Generate E2HS files for post-capella
- [x] We have code to generate HeaderWithProofs from Genesis to pre-deneb
- [ ] We have code to generate HeaderWithProofs from post-deneb onwards
- [ ] Script for generating E2HS files every 8192 finalized execution Blocks
- [ ] Implement Ephemeral Header's Bridge
### `x.` Content Validation | Full History Network Client Support
- [x] Trin supports the Portal Beacon Network to be used as an oracle for HistoricalSummaries and the HEAD of the chain
- [ ] We support Electra HistoricalSummaries
- [x] Trin supports Genesis to pre-capella HeaderWithProof Validation
- [ ] Trin supports post-capella HeaderWithProof Validation
- [x] Trin has implemented the Ephemeral Header Content Types
- [ ] Trin supports/implemented Ephemeral Header's specification
### `y.` Testing
- [ ] Hive Validation Negation tests
- [ ] Others?Every team should share and maintain a list like shown above on their progress for supporting the Full History Network. Most Teams shouldn't need to worry above Bridging infrastructure and prioritize Full History Network Client Support. From the example from Trin if we check off all the points for z. and x. the Full History on the History Network would be live and henceforth Portal would finally have a working product to release to the world.
I expect for Trin we should be able to finish all pre-ephemeral content tasks in the next 1 to 2 weeks max.