Preserve orphaned blocks

[michaelsproul]

Description

We currently have no way of preserving orphaned blocks, even though they are useful for block explorers and analytics.

Steps to resolve

• Add a flag to lighthouse bn called something like --keep-orphaned-blocks
• Add a new column to the database for mapping slots to vecs of orphaned block roots (DBColumn::OrphanedBlocks). The blocks themselves can be stored in the hot DB's main block storage, where all other blocks are currently stored.
• In the pruning code that runs on finalization add logic to keep the blocks when --keep-orphaned-blocks is set, storing their block roots under the slot => Vec<Hash256> mapping.
• In the API:
  • Modify /eth/v1/beacon/headers?slot so that it reads block roots from the orphaned block roots table, and then looks them up in the main block table.
  • Work out what to do with /eth/v1/beacon/headers?parent_root (I don't know what we do currently)
  • The orphaned blocks themselves can then be accessed by their root using /eth/v1/beacon/blocks/{root} but not by their slot directly.

Version

Lighthouse v1.4.0

[eklm]

I would like to take this.

[michaelsproul]

Thanks @eklm that would be awesome!

I'm working on some other database changes at the moment which are making me reconsider this part of the design:

The blocks themselves can be stored in the hot DB's main block storage, where all other blocks are currently stored.

I think it might be better to move the orphaned blocks to a separate part of the database. So maybe the column DBColumn::OrphanedBlocks could store values of Vec<SignedBeaconBlock<E>>, or we keep the layer of redirection and add two columns: OrphanedBlockRoots: Slot => Vec<Hash256> and OrphanedBlocks: Hash256 => SignedBeaconBlock<E>.