EIP-7917: Deterministic proposer lookahead

specs/fulu/beacon-chain.md

@@ -10,6 +10,18 @@
   - [Block processing](#block-processing)
     - [Execution payload](#execution-payload)
       - [Modified `process_execution_payload`](#modified-process_execution_payload)
+- [Containers](#containers)
+  - [Extended Containers](#extended-containers)
+    - [`BeaconState`](#beaconstate)
+- [Helper functions](#helper-functions)
+  - [Misc](#misc)
+    - [New `compute_proposer_indices`](#new-compute_proposer_indices)
+  - [Beacon state accessors](#beacon-state-accessors)
+    - [Modified `get_beacon_proposer_index`](#modified-get_beacon_proposer_index)
+    - [New `get_beacon_proposer_indices`](#new-get_beacon_proposer_indices)
+  - [Epoch processing](#epoch-processing)
+    - [Modified `process_epoch`](#modified-process_epoch)
+    - [New `process_proposer_lookahead`](#new-process_proposer_lookahead)
 
 <!-- mdformat-toc end -->
 
@@ -71,3 +83,162 @@ def process_execution_payload(state: BeaconState, body: BeaconBlockBody, executi
         excess_blob_gas=payload.excess_blob_gas,
     )
 ```
+
+## Containers
+
+### Extended Containers
+
+#### `BeaconState`
+
+*Note*: The `BeaconState` container is extended with the `proposer_lookahead`
+field, which is a list of validator indices covering the full lookahead period,
+starting from the beginning of the current epoch. For example,
+`proposer_lookahead[0]` is the validator index for the first proposer in the
+current epoch, `proposer_lookahead[1]` is the validator index for the next
+proposer in the current epoch, and so forth. The length of the
+`proposer_lookahead` list is `(MIN_SEED_LOOKAHEAD + 1) * SLOTS_PER_EPOCH`,
+reflecting how far ahead proposer indices are computed based on the
+`MIN_SEED_LOOKAHEAD` parameter.
+
+```python
+class BeaconState(Container):
+    # Versioning
+    genesis_time: uint64
+    genesis_validators_root: Root
+    slot: Slot
+    fork: Fork
+    # History
+    latest_block_header: BeaconBlockHeader
+    block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
+    state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
+    historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT]
+    # Eth1
+    eth1_data: Eth1Data
+    eth1_data_votes: List[Eth1Data, EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH]
+    eth1_deposit_index: uint64
+    # Registry
+    validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
+    balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
+    # Randomness
+    randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR]
+    # Slashings
+    slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR]  # Per-epoch sums of slashed effective balances
+    # Participation
+    previous_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]
+    current_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]
+    # Finality
+    justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH]  # Bit set for every recent justified epoch
+    previous_justified_checkpoint: Checkpoint
+    current_justified_checkpoint: Checkpoint
+    finalized_checkpoint: Checkpoint
+    # Inactivity
+    inactivity_scores: List[uint64, VALIDATOR_REGISTRY_LIMIT]
+    # Sync
+    current_sync_committee: SyncCommittee
+    next_sync_committee: SyncCommittee
+    # Execution
+    latest_execution_payload_header: ExecutionPayloadHeader
+    # Withdrawals
+    next_withdrawal_index: WithdrawalIndex
+    next_withdrawal_validator_index: ValidatorIndex
+    # Deep history valid from Capella onwards
+    historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT]
+    deposit_requests_start_index: uint64
+    deposit_balance_to_consume: Gwei
+    exit_balance_to_consume: Gwei
+    earliest_exit_epoch: Epoch
+    consolidation_balance_to_consume: Gwei
+    earliest_consolidation_epoch: Epoch
+    pending_deposits: List[PendingDeposit, PENDING_DEPOSITS_LIMIT]
+    pending_partial_withdrawals: List[PendingPartialWithdrawal, PENDING_PARTIAL_WITHDRAWALS_LIMIT]
+    pending_consolidations: List[PendingConsolidation, PENDING_CONSOLIDATIONS_LIMIT]
+    proposer_lookahead: Vector[ValidatorIndex, (MIN_SEED_LOOKAHEAD + 1) * SLOTS_PER_EPOCH]  # [New in Fulu:EIP7917]
+```
+
+## Helper functions
+
+### Misc
+
+#### New `compute_proposer_indices`
+
+```python
+def compute_proposer_indices(state: BeaconState, epoch: Epoch, seed: Bytes32, indices: Sequence[ValidatorIndex]) -> List[ValidatorIndex, SLOTS_PER_EPOCH]:
+    """
+    Return the proposer indices for the given ``epoch``.
+    """
+    start_slot = compute_start_slot_at_epoch(epoch)
+    seeds = [hash(seed + uint_to_bytes(Slot(start_slot + i))) for i in range(SLOTS_PER_EPOCH)]
+    return [compute_proposer_index(state, indices, seed) for seed in seeds]
+```
+
+### Beacon state accessors
+
+#### Modified `get_beacon_proposer_index`
+
+*Note*: The function `get_beacon_proposer_index` is modified to use the
+pre-calculated `current_proposer_lookahead` instead of calculating it on-demand.
+
+```python
+def get_beacon_proposer_index(state: BeaconState) -> ValidatorIndex:
+    """
+    Return the beacon proposer index at the current slot.
+    """
+    return state.proposer_lookahead[state.slot % SLOTS_PER_EPOCH]
+```
+
+#### New `get_beacon_proposer_indices`
+
+```python
+def get_beacon_proposer_indices(state: BeaconState, epoch: Epoch) -> List[ValidatorIndex, SLOTS_PER_EPOCH]:
+    """
+    Return the proposer indices for the given ``epoch``.
+    """
+    indices = get_active_validator_indices(state, epoch)
+    seed = get_seed(state, epoch, DOMAIN_BEACON_PROPOSER)
+    return compute_proposer_indices(state, epoch, seed, indices)
+```
+
+### Epoch processing
+
+#### Modified `process_epoch`
+
+*Note*: The function `process_epoch` is modified in Fulu to call
+`process_proposer_lookahead` to update the `proposer_lookahead` in the beacon
+state.
+
+```python
+def process_epoch(state: BeaconState) -> None:
+    process_justification_and_finalization(state)
+    process_inactivity_updates(state)
+    process_rewards_and_penalties(state)
+    process_registry_updates(state)
+    process_slashings(state)
+    process_eth1_data_reset(state)
+    process_pending_deposits(state)
+    process_pending_consolidations(state)
+    process_effective_balance_updates(state)
+    process_slashings_reset(state)
+    process_randao_mixes_reset(state)
+    process_historical_summaries_update(state)
+    process_participation_flag_updates(state)
+    process_sync_committee_updates(state)
+    process_proposer_lookahead(state)  # [New in Fulu:EIP7917]
+```
+
+#### New `process_proposer_lookahead`
+
+*Note*: This function updates the `proposer_lookahead` field in the beacon state
+by shifting out proposer indices from the earliest epoch and appending new
+proposer indices for the latest epoch. With `MIN_SEED_LOOKAHEAD` set to `1`,
+this means that at the start of epoch `N`, the proposer lookahead for epoch
+`N+1` will be computed and included in the beacon state's lookahead.
+
+```python
+def process_proposer_lookahead(state: BeaconState) -> None:
+    last_epoch_start = len(state.proposer_lookahead) - SLOTS_PER_EPOCH
+    # Shift out proposers in the first epoch
+    state.proposer_lookahead[:last_epoch_start] = state.proposer_lookahead[SLOTS_PER_EPOCH:]
+    # Fill in the last epoch with new proposer indices
+    last_epoch_proposers = get_beacon_proposer_indices(state, Epoch(get_current_epoch(state) + MIN_SEED_LOOKAHEAD + 1))
+    state.proposer_lookahead[last_epoch_start:] = last_epoch_proposers
+```

specs/fulu/fork.md

@@ -9,6 +9,7 @@
 - [Helper functions](#helper-functions)
   - [Misc](#misc)
     - [Modified `compute_fork_version`](#modified-compute_fork_version)
+    - [New `initialize_proposer_lookahead`](#new-initialize_proposer_lookahead)
 - [Fork to Fulu](#fork-to-fulu)
   - [Fork trigger](#fork-trigger)
   - [Upgrading the state](#upgrading-the-state)
@@ -54,6 +55,23 @@ def compute_fork_version(epoch: Epoch) -> Version:
     return GENESIS_FORK_VERSION
 ```
 
+#### New `initialize_proposer_lookahead`
+
+```python
+def initialize_proposer_lookahead(
+    state: electra.BeaconState
+) -> List[ValidatorIndex, (MIN_SEED_LOOKAHEAD + 1) * SLOTS_PER_EPOCH]:
+    """
+    Return the proposer indices for the full available lookahead starting from current epoch.
+    Used to initialize the ``proposer_lookahead`` field in the beacon state at genesis and after forks.
+    """
+    current_epoch = get_current_epoch(state)
+    lookahead = []
+    for i in range(MIN_SEED_LOOKAHEAD + 1):
+        lookahead.extend(get_beacon_proposer_indices(state, Epoch(current_epoch + i)))
+    return lookahead
+```
+
 ## Fork to Fulu
 
 ### Fork trigger
@@ -129,6 +147,7 @@ def upgrade_to_fulu(pre: electra.BeaconState) -> BeaconState:
         pending_deposits=pre.pending_deposits,
         pending_partial_withdrawals=pre.pending_partial_withdrawals,
         pending_consolidations=pre.pending_consolidations,
+        proposer_lookahead=initialize_proposer_lookahead(pre),  # [New in Fulu:EIP7917]
     )
 
     return post

tests/core/pyspec/eth2spec/test/fulu/epoch_processing/test_process_proposer_lookahead.py

@@ -0,0 +1,43 @@
+from eth2spec.test.context import spec_state_test, with_fulu_and_later
+from eth2spec.test.helpers.epoch_processing import run_epoch_processing_with
+from eth2spec.test.helpers.state import next_epoch
+
+
+@with_fulu_and_later
+@spec_state_test
+def test_next_epoch_proposer_lookahead_shifted_to_front(spec, state):
+    """Test that the next epoch proposer lookahead is shifted to the front at epoch transition."""
+    # Transition few epochs to pass the MIN_SEED_LOOKAHEAD
+    next_epoch(spec, state)
+    next_epoch(spec, state)
+    # Get initial lookahead
+    initial_lookahead = state.proposer_lookahead.copy()
+
+    # Run epoch processing
+    yield "pre", state
+    next_epoch(spec, state)
+    yield "post", state
+
+    # Verify lookahead was shifted correctly
+    assert (
+        state.proposer_lookahead[: spec.SLOTS_PER_EPOCH]
+        == initial_lookahead[spec.SLOTS_PER_EPOCH :]
+    )
+
+
+@with_fulu_and_later
+@spec_state_test
+def test_proposer_lookahead_in_state_matches_computed_lookahead(spec, state):
+    """Test that the proposer lookahead in the state matches the lookahead computed on the fly."""
+    # Transition few epochs to pass the MIN_SEED_LOOKAHEAD
+    next_epoch(spec, state)
+    next_epoch(spec, state)
+
+    # Run epoch processing
+    yield "pre", state
+    next_epoch(spec, state)
+    yield "post", state
+
+    # Verify lookahead in state matches the lookahead computed on the fly
+    computed_lookahead = spec.initialize_proposer_lookahead(state)
+    assert state.proposer_lookahead == computed_lookahead

tests/core/pyspec/eth2spec/test/fulu/fork/test_fork_lookahead_consistency.py

@@ -0,0 +1,37 @@
+from eth2spec.test.context import (
+    spec_test,
+    with_phases,
+    with_state,
+)
+from eth2spec.test.helpers.constants import (
+    ELECTRA,
+    FULU,
+)
+from eth2spec.test.helpers.fulu.fork import (
+    FULU_FORK_TEST_META_TAGS,
+    run_fork_test,
+)
+from eth2spec.test.helpers.state import next_slot
+from eth2spec.test.utils import with_meta_tags
+from tests.core.pyspec.eth2spec.test.helpers.state import simulate_lookahead
+
+
+@with_phases(phases=[ELECTRA], other_phases=[FULU])
+@spec_test
+@with_state
+@with_meta_tags(FULU_FORK_TEST_META_TAGS)
+def test_lookahead_consistency_at_fork(spec, phases, state):
+    """
+    Test that lookahead is consistent before/after the Fulu fork.
+    """
+
+    # Calculate the current and next epoch lookahead by simulating the state progression
+    # with empty slots and calling `get_beacon_proposer_index` (how it was done pre-Fulu)
+    pre_fork_proposers = simulate_lookahead(spec, state)
+
+    # Upgrade to Fulu
+    spec = phases[FULU]
+    state = yield from run_fork_test(spec, state)
+
+    # Check if the pre-fork simulation matches the post-fork `state.proposer_lookahead`
+    assert pre_fork_proposers == state.proposer_lookahead

tests/core/pyspec/eth2spec/test/fulu/sanity/test_effective_balance_increase_changes_lookahead.py

@@ -0,0 +1,90 @@
+from eth2spec.test.context import (
+    spec_state_test,
+    with_phases,
+)
+from eth2spec.test.helpers.attestations import (
+    state_transition_with_full_block,
+)
+from eth2spec.test.helpers.constants import ELECTRA, FULU
+from eth2spec.test.helpers.state import (
+    next_epoch,
+    simulate_lookahead,
+)
+from eth2spec.test.helpers.withdrawals import (
+    set_compounding_withdrawal_credential,
+)
+
+
+def run_test_effective_balance_increase_changes_lookahead(
+    spec, state, randao_setup_epochs, expect_lookahead_changed
+):
+    # Advance few epochs to adjust the RANDAO
+    for _ in range(randao_setup_epochs):
+        next_epoch(spec, state)
+
+    # Set all active validators to have balance close to the hysteresis threshold
+    current_epoch = spec.get_current_epoch(state)
+    active_validator_indices = spec.get_active_validator_indices(state, current_epoch)
+    for validator_index in active_validator_indices:
+        # Set compounding withdrawal credentials for the validator
+        set_compounding_withdrawal_credential(spec, state, validator_index)
+        state.validators[validator_index].effective_balance = 32000000000
+        # Set balance to close the next hysteresis threshold
+        state.balances[validator_index] = 33250000000 - 1
+
+    # Calculate the lookahead of next epoch
+    next_epoch_lookahead = simulate_lookahead(spec, state)[spec.SLOTS_PER_EPOCH :]
+
+    blocks = []
+    yield "pre", state
+
+    # Process 1-epoch worth of blocks with attestations
+    for _ in range(spec.SLOTS_PER_EPOCH):
+        block = state_transition_with_full_block(
+            spec, state, fill_cur_epoch=True, fill_prev_epoch=True
+        )
+        blocks.append(block)
+
+    yield "blocks", blocks
+    yield "post", state
+
+    # Calculate the actual lookahead
+    actual_lookahead = simulate_lookahead(spec, state)[: spec.SLOTS_PER_EPOCH]
+
+    if expect_lookahead_changed:
+        assert next_epoch_lookahead != actual_lookahead
+    else:
+        assert next_epoch_lookahead == actual_lookahead
+
+
+def run_test_with_randao_setup_epochs(spec, state, randao_setup_epochs):
+    if spec.fork == ELECTRA:
+        # Pre-EIP-7917, effective balance changes due to attestation rewards
+        # changes the next epoch's lookahead
+        expect_lookahead_changed = True
+    else:
+        # Post-EIP-7917, effective balance changes due to attestation rewards
+        # do not change the next epoch's lookahead
+        expect_lookahead_changed = False
+
+    yield from run_test_effective_balance_increase_changes_lookahead(
+        spec, state, randao_setup_epochs, expect_lookahead_changed=expect_lookahead_changed
+    )
+
+
+@with_phases(phases=[ELECTRA, FULU])
+@spec_state_test
+def test_effective_balance_increase_changes_lookahead(spec, state):
+    # Since this test relies on the RANDAO, we adjust the number of next_epoch transitions
+    # we do at the setup of the test run until the assertion passes.
+    # We start with 4 epochs because the test is known to pass with 4 epochs.
+    for randao_setup_epochs in range(4, 20):
+        try:
+            state_copy = state.copy()
+            yield from run_test_with_randao_setup_epochs(spec, state_copy, randao_setup_epochs)
+            return
+        except AssertionError:
+            # If the randao_setup_epochs is not the right one to make the test pass,
+            # then try again in the next iteration
+            pass
+    assert False, "The test should have succeeded with one of the iterations."