RFC: Feature Threshold Mechanism: Accounts or Blocks

[buffalojoec]

Feature Threshold Mechanism: Accounts or Blocks

The network requires a more robust solution for activating features. Currently,
contributors can submit feature activations with a single keypair, often
ignoring the network's stake distribution across compatible client versions.

Since features are forking events, this is a critical infrastructure risk as
well as a centralization vector. Validators should be able to collectively
activate or block feature activations through quorum.

SIMD-0072 proposed an enshrined on-chain protocol that would solve this
problem. In short, the protocol would require validator approval for any staged
feature activations to become live. However, an equally compelling alternative
design has been proposed, leading to SIMD-0072's long stalemate.

This RFC presents both designs and their tradeoffs to reach a decision on which
to implement.

SIMD-0072: Account-Based Signaling

SIMD-0072 introduces an on-chain protocol where validators explicitly signal
support for staged features via transactions. The Feature Gate program tallies
stake-weighted support, and the runtime activates staged features that meet the
threshold at epoch boundaries.

The process works as follows:

1. A contributor creates a feature account and configures a staging authority.
2. The staging authority stages the feature for activation.
3. During the next epoch, validators submit transactions signaling which staged
   features they support.
4. At the epoch boundary, the runtime activates features with sufficient stake
   support.

Feature accounts use a new FeatureV2 state that tracks status, staging
authority, and accumulated stake support:

enum FeatureV2 {
    Inactive {
        staging_authority: Pubkey,
    },
    Staged {
        signal_epoch: u64,
        staging_authority: Pubkey,
        stake_support: u64,
    },
    Active {
        activation_epoch: u64,
    }
}

Validators signal support by submitting a SignalSupportForStagedFeatures
transaction that includes all feature accounts they wish to support. The
instruction requires the validator's authorized voter as a signer and their
vote account as a reference.

A signal account PDA prevents double-signaling within an epoch:

/// PDA seeds: ["support_signal", vote_account_address]
struct SupportSignal {
    /// The epoch this validator last submitted a support signal.
    epoch: u64,
}

The Feature Gate program uses the SolGetEpochStake syscall to retrieve each
signaling validator's stake weight and accumulates it in the stake_support
field of each included feature account.

Features must be explicitly staged before they can be activated; the signaling
mechanism only determines whether a staged feature has sufficient support to go
live. At the epoch boundary, the runtime compares each staged feature's
accumulated support against total cluster stake and activates those meeting the
threshold.

Pros

• All validators can signal support, regardless of whether they produce blocks
  during the epoch.
• Stake support is stored in feature accounts and queryable via RPC without
  special tooling.
• If a validator fails to signal, the omission is observable and diagnosable.

Cons

• Validators must submit a transaction each epoch. Failure to do so
  (misconfiguration, downtime, neglect) effectively votes against all staged
  features.
• Feature accounts and signal PDAs require rent-exempt balances. Each signal
  transaction incurs fees.

Alternative: Block Footer Signaling

Block footer signaling embeds feature support directly in block production.
Rather than submitting explicit transactions, leaders include a bitmask in the
block footer indicating which features their client supports. The runtime
maintains a running tally of stake-weighted support and activates staged
features at epoch boundaries.

SIMD-0307 introduced block footers as mandatory structured data appended to
each block. Feature signaling would extend the footer with a bitmask field:

         ...existing footer fields...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| feature_support_bitmask     (N bytes) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The bitmask encodes support for features in the current major version's feature
set. The exact encoding is a design detail to be determined, but one possibility
is to assign bit positions based on source order in the feature-set crate, with
universally activated features pruned on each major release to keep the bitmask
compact.

As with SIMD-0072, features must still be staged before they can be activated.
The signaling mechanism only determines whether a staged feature has sufficient
support to go live.

Unlike SIMD-0072, stake support is accumulated in the runtime rather than
on-chain. As each block is processed, the runtime accumulates the leader's stake
toward each supported feature. At the epoch boundary, the runtime compares
accumulated support against total cluster stake and activates staged features
meeting the threshold.

Pros

• Leaders signal support automatically by producing blocks; no separate
  transaction or operational step required.
• No on-chain accounts or transaction fees required for signaling.
• Signaling cannot be forgotten or misconfigured; it is inherent to block
  production.

Cons

• Only validators who produce blocks during the epoch can signal support,
  though leader selection should cover most stake over a full epoch.
• Stake support is tallied in runtime memory and is ephemeral. Observing current
  support levels requires RPC extensions or reconstructing tallies from block
  data.

Call to Action

This RFC requests explicit review from the following core contributors:

• @t-nelson (Anza)
• @jstarry (Anza)
• @nbelenkov (Anza)
• @topointon-jump (Firedancer)
• @mjain-jump (Firedancer)
• @tigarcia (Solana Foundation)

Any additional community members are welcome to provide feedback and their input
will be equally considered. The above merely lists contributors already involved
in this debate or with relevant domain expertise.

Reviewers should weigh in on which design they prefer, considering factors such
as:

• Implementation complexity across clients
• Operational burden on validators
• Reliability of stake coverage
• Observability of signaling state

[nbelenkov]

Thanks for writting this up @buffalojoec! I think I am personally leaning towards block footer signalling for a few reasons:

1. With alpenglow incoming we basically will no longer have on chain votes, hence one of the only on-chain txs that the validator will now need to send will be the feature gate voting.
2. Both vote based stake counting and leader based runtime bitmask counting I think represent the validator stake pretty well, as the leader selection is stake based so we would have a decent representation, unless the feature is triggered close to epoch boundary. But then if we do reach majority in that small period with some crazy staked validators, the same can be achieved with them voting in simd-72, so I don't see this is a concern.

Complexity wise the bitmask one would probably be more complicated to implement than on chain voting. We also potentially need to account for cases where you might have 2 feature gates due to be activated and you only support one of them, with account based voting its straight forward, but here we need to encode that into the bitmask and keep that in mind for the runtime.

Re observability account based voting is simpler for sure, with block signaling we can either calculate it off-chain as you suggested or expose metrics from the validators somehow, will require an engineering effort here.

Concerns:

1. In account based signaling, it is probably easier for validators to specify they don't want to support a certain feature than in the block header one. But this is probably more of an implementation detail
2. I think there will be more performance overhead for block footer voting than the account signaling, as you would need to tally each block with relevant stake of the block producer, instead of sending one tx per epoch, but I am not sure how significant that overhead will be.
3. I am not too informed on how MCP/MCL is supposed to work in the current design, but I wonder if affects how we would need to do block footer voting.
4. With block footer signaling we would need to be careful in cases where a validator downgrades versions and no longer has the feature gate after producing a block with the new expected feature gate, which might lead to accounting issues. (or maybe if the validator fails over to their backup with older version)

[buffalojoec]

Thanks for the review! Some comments on your concerns:

1. I agree that this is an implementation detail. I think for either design, we could get to a place where operators pass a --disallow-feature <PUBKEY> flag to vote "no".
2. Actually I believe all of the information we'd need is already available. The leader is packing the block, and the current Bank needs to know that leader's stake so it can calculate their block rewards.
3. AFAIK, each leader will be creating shreds of the same block, so we'd probably need to expand the design for MCP to support more than one feature signal bitmask per block. Just like most of the Banking Stage, it will need an overhaul when we're ready for MCP.
4. This is a great point. Once the block is produced, you're cooked. Maybe this is a salient reason to favor the transaction-based approach. Alternatively, you could have a program manage only "rewrites" or send these messages over gossip. At that point though you start to wonder if the whole mechanism belongs there...

[nbelenkov]

This is a great point. Once the block is produced, you're cooked. Maybe this is a salient reason to favor the transaction-based approach. Alternatively, you could have a program manage only "rewrites" or send these messages over gossip. At that point though you start to wonder if the whole mechanism belongs there...

Yeha thinking more about this case, it basically makes the block approach way too complicated, as we now have to account for feature set changes, which the on-chain program can easily handle. The MCP stuff would also be easier to handle with on-chain program actually

[ksn6]

A few questions with the block footer route:

• All validators need to agree on the total stake votes associated with each feature. Otherwise, not all validators may end up enabling desired features at epoch boundaries (which would be very bad). How do we enforce this in the block footer route? E.g., with the "transaction" approach, things are essentially enforced via bank hash matches, right? How are we thinking about this with the block footer approach?

• How do we handle cases where the validator switches back and forth with decisions across multiple blocks (i.e., inclusion / exclusion of particular features across multiple block footers)? E.g., should this be allowed?

• What is the proposed layout / format for feature_support_bitmask?

[buffalojoec]

Nice, gud data. This is really nice to know!

[nbelenkov]

oh this is much quicker than I thought, thanks for running the data

[nbelenkov]

Yep, this was my thinking as well; it sounds like we'll have to write some data structures which keep track of voting state here, but that's essentially what chain state gets for us anyways.

We need to basically track the latest signal from each validator, so that if they change feature sets we can update our internal accouting, which does become quite a large datastructure that will now need to have key as validator id and feature gate they support potentially. Its a good point that chain state basically keeps that for us if we go with program approach

[topointon-jump]

Storing metadata off-chain is fine for non-critical things, like observability, seeing what clients are running etc - when it's fine to not have a perfectly synchronised view of the world. Using this data to make on-chain decisions is much more fragile - now you have a whole new system and data-structure that is tied into consensus which every client must get exactly right.

[topointon-jump]

Also - if we accumulate this data in the runtime from block footers, any nodes which were restarted too close to the epoch boundary wouldn't have had enough time to tally the votes, and would fork off. Feature gate activations become non-deterministic for each node based on their uptime.

[topointon-jump]

I lean towards the transaction sending on-chain account based approach. Stake support information is primarily needed for on-chain logic: feature activations. Therefore it should live in the same place as the other on-chain state: in an account.

This has other advantages:

• Each client implementation will be much simpler. A bug here could have severe consequences, so we should aim for simplicity here.
• On-chain programs, such as the feature gate program, can programatically inspect it. This lets us build further feature gate automation on-chain using stake support as a first-class primitive.
• Operators and core devs can query the current stake support for a feature in a simple and reliable way, by inspecting the state, without being affected by any bugs or changes in turbine or gossip.

Relying on off-chain protocols for this information is fragile. We already have a mechanism for synchronising state needed by the runtime - the on-chain accounts - which each client must keep synchronised correctly. That's the perfect place to store this information.

AFAICT, the main downside of this approach is that each validator has to send a transaction every epoch. Are we concerned about the cost? Or the transaction sending mechanism? I would think the cost of a single transaction every epoch is negligible compared to the overhead of operating a validator.

[buffalojoec]

Solid points.

AFAICT, the main downside of this approach is that each validator has to send a transaction every epoch. Are we concerned about the cost? Or the transaction sending mechanism? I would think the cost of a single transaction every epoch is negligible compared to the overhead of operating a validator.

I don't view cost as a major issue. With Tower voting the cost of sending a feature signal transaction pales in comparison to vote transaction costs per epoch. With Alpenglow, there's still a cost involved via the 2 SOL bond mechanism, so it's a similar metric.

My main concern with transactions was the fact that validators must submit these, retry them, etc. and there's nothing we can do at a protocol level to course correct if any nodes just simply don't send them. However, if it's ingrained as part of the client then maybe it's not really an issue unless someone intentionally disables the transactions?

I suppose - to that point - the transaction approach also gives a bit more power to validators by way of abstaining.

[topointon-jump]

Yeah, it would be automatic - you would have to try hard to disable it, which is the same as abstaining. The nice thing about being on-chain is also that the automatic loop can easily check if the transaction has landed - just look up it's PDA and see if it contains the correct information. If it doesn't, send the transaction. Do this in a loop every hour or something.

[tigarcia]

The downside I see with tx signaling is that the identity account is required to sign the tx. With the Alpenglow rollout, I believe the desire is to have the VAT be charged in the vote account. Once that's in place, the protocol could put rewards from tx fees and priority fees in the vote account as well. Then the validator operator wouldn't need to have access to a key with funds. At least that's my understanding. @wen-coding for confirmation.

[buffalojoec]

Also, the cost of sending tx per epoch, if most of the time default answer is yes, can we assume yes by default and people only send in tx to vote no? That will reduce the cost a lot.

I would love it if we could do this, since it would be so much easier on the operators, but unfortunately we don't have a way to detect automatically which nodes are running compatible software. Each signal is an attestation saying "I have this feature ID and I'm on a compatible version and I support it". If we invert that and start from 100%, we're not going to be able to filter out nodes that might be missing the same feature ID.

[wen-coding]

Hmm I have two questions:

1. an operator can downgrade his software after sending the attestation right? How can he revert his attestation?
2. why can't a cli tool talk with the running validator to retrieve the list of features this validator supports, compare with the features to be activated in next epoch, and then send nacks for the difference?

[buffalojoec]

1. an operator can downgrade his software after sending the attestation right? How can he revert his attestation?

He would have to send a new signal transaction with whatever features removed, and it will update his entry. If he doesn't, he'd fork off on activation.

2. why can't a cli tool talk with the running validator to retrieve the list of features this validator supports, compare with the features to be activated in next epoch, and then send nacks for the difference?

It can! I'm saying the leader at the epoch boundary - if it was to assume everyone has the feature and only acknowledge "nay" - wouldn't actually know for sure if that calculated stake support is actually compliant.

Imagine 10% of the network is behind by one version and they never submit any "nay" signals. The leader at the epoch boundary is going to be assuming they're in support, but in reality they're going to fork off.

[wen-coding]

Hmm, I suppose "my software doesn't support this feature" is different from "I have this feature in my software but I don't want to activate it because I'm against the philosophy/mechanism/etc"

For the latter we definitely need the operators to send in nay signals. The former seems to be something the software can solve themselves without involving human, just update their state somewhere on the chain. (And, human can make mistakes, they can say "yes" but they forgot to update software version of their standby validator, then we have the same problem.)

[buffalojoec]

The former seems to be something the software can solve themselves without involving human, just update their state somewhere on the chain.

Precisely - this is the core reason for the "yay" signaling mechanism.

(And, human can make mistakes, they can say "yes" but they forgot to update software version of their standby validator, then we have the same problem.)

Yep, but the price they would pay for this oversight is a forking event, which is probably fair.