Add version to AccountMeta

[roryharr]

This is to solve two issues:

1. The format of an account in the snapshot cannot be changed, since there is only a single interpretation of accounts. Adding a version to the AccountMeta would allow changing the snapshot format to remove or add fields in the future. Currently there are 48 bytes of deprecated fields stored in every account.

2. The account hash cannot be changed as it requires mixing out the old hash. For example: Whether the rent epoch was included in the hash or not when the account was hashed in is required for adding the last written slot required for dynamic state rent (

To resolve both these issues: Version should be added to the AccountMeta, and should be included as part of the accounts hash.

To make this backwards compatible the initial version would be defined as 0, and bytes that are guaranteed to be zero in both the AccountMeta and the lattice hash calculation would be reused to store the version value: The upper 4 bits of the executable byte. This should provide support for up to 15 version rolls, while leaving 3 bits available in the byte for other data/changes in the future.

Any version updates will require a feature activation, and upon features activation, account versions would only be updated when an account is rewritten to a new bank.

[igor56D]

+1

I think we can not expose this field in the runtime, for the sake of simplicity, and circle back if exposing it is necessary.

[roryharr]

RPC calls may need a new API as well

[roryharr]

Talked to @brooksprumo.

His feedback is to focus on the account version in this SIMD, and leave the storage version alone.

If storage format changes are wanted to save bytes, it should be done using TieredStorage which has even more gains.

So the SIMD will just be Add Version to Accounts. Effects will be two fold:

1. Version is added to the lattice hash
2. Version is added to the accounts meta data.

[brooksprumo]

The storage of accounts is also client implementation details, so wouldn't need a SIMD. (Explicitly excluding the current snapshot format from the previous statement too.)

[brooksprumo]

Other background on versioning the per-account hash: #319 (comment)

[brooksprumo]

Note we should only do this once there is a need.

[brooksprumo]

Let's call this "account hash version" instead. This is needed for the accounts lattice hash, so that if we want to change the account structure, or how we hash a single account, that we can still compute the accounts lattice hash correctly.

Some examples without changing the account structure:

1. We want to change the hash function.

Today we use blake3 for hashing. In the future there may be a new hash function, e.g. blake4, that we want to use. We want a way to update the hash function independent of the account structure.

2. We want to change the order that we pass fields into the hash function.

Here's the order for hashing today:

solana-improvement-documents/proposals/0215-accounts-lattice-hash.md

Lines 101 to 107 in 6a41b4d

	lthash.init()
	lthash.append( account.lamports )
	lthash.append( account.data )
	lthash.append( account.is_executable )
	lthash.append( account.owner )
	lthash.append( account.pubkey )
	return lthash.fini()

Note that account data is in the middle. There have been discussions/recommendations that all variable length fields are moved to the end. Account data is the only variable length field today, so we would want to support moving account data to the end.

3. Adding non-account fields to be hashed, e.g. a hash domain.

The account hash currently does not have hash domain, so it may collide with other non-account hashes. Yes, it is invalid to compare an account hash with a non-account hash. There has been a long-standing goal of putting all protocol hashes into their own hash domains to prevent these issues entirely.

For the account hash, we'd want to support adding a hash domain. This would change the account hash algorithm itself, but not the account structure. E.g. we'd include the string "account" or "account v3" as the first field sent to the hash function.

These were all examples of the how the account hash could change that is independent of the account structure itself changing. We'd also need to change the version if we change the account structure, yes. This is more to motivate why this concept is larger than versioning the account (or even more narrowly, the account metadata) itself.

[brooksprumo]

Oh, I think maybe we're trying to solve different problems. Let me back up and share some background/context.

Before the accounts lt hash feature was activated, the bank hash contained the accounts delta hash. The ADH was a merkle tree of per-account hashes for all the accounts modified in that slot. Computing the ADH only required the new, modified state of the accounts. As such, we could change the per-account hash algorithm with just a feature gate.

As an example, we changed the per-account hash algorithm to remove slot. Here's the issue solana-labs/solana#28404, and PR solana-labs/solana#28405. This change facilitated moving the physical storage location of accounts between storage files. Previously this was not possible, as the slot information is not stored with the account, only in the storage's file name/AccountStorageEntry.

If we wanted to change the Account structure/layout, we could've done it with a feature gate as well. (Note back then we could change things much easier: single client world, much smaller account state and user base.)

As part of removing rent collection, we first stopped rewriting unchanged accounts (aka skipping rewrites). We first required all accounts to be rent exempt. We also stopped serializing the rent epoch into the VM (or rather, made it a constant). At that point we could've removed rent_epoch from the per-account hash algorithm as well. Instead we combined it with the activation of the first accounts lt hash feature gate.

We could've stopped including rent_epoch in the per-account hash separate/before the accounts lt hash feature if we wanted. And at that point, we could've then also repurposed the rent_epoch bytes for a new use. That repurposing would've only needed a feature gate as well. No account versioning required. This would also extend to/be true for removing executable from Account as well (again, in the old ADH world, not the new accounts lt hash world).

This has changed with the accounts lt hash. Now, we need more than just the new, modified version of the accounts; we also need their previous version. We must mix out the old per-account hash, and mix in the new one. And the mixing out must use the same per-hash algorithm from when that old account was mixed into the accounts lt hash.

Given that, there was/is a push to remove executable from the VM as well (#319 (comment)). To accomplish this, when an account is modified, we still need to mix out the old state, which means we need to know what its executable used to be. When hashing the account, we need a way to know the old version should be hashed the old way.

This is the problem being solved by the account version.

The account version is not intended to solve the question "how do we change the storage format of accounts". Instead, it is meant to solve "how do we still compute the correct per-account hash of the account's old version".

This does tie into account storage, due to snapshots and their use of append vecs. As such, we're limited to reusing account fields without incur a lot of additional pain. If we wanted to grow the account structure, then we'd also need versioning for the storage file format as well. This versioning is implemented in the agave Tiered Storage file format for precisely this reason.

In conclusion, the main point I'd like to emphasize is that the account version is intended to solve the new complications when changing how we hash an account, due to the accounts lt hash. It is not intended to handle account storage file format changes.

[roryharr]

I had originally planned on solving two problems

1. The accounts hash calculation
2. Allow changes to the storage format so it can be optimized.

But changed direction as of this comment last week: Brook's feedback is to focus on the account version in this SIMD, and leave the storage version alone.

However I somewhat disagree with this statement:

We could've stopped including rent_epoch in the per-account hash separate/before the accounts lt hash feature if we wanted. And at that point, we could've then also repurposed the rent_epoch bytes for a new use. That repurposing would've only needed a feature gate as well. No account versioning required

I don't rent epoch could've been repurposed for a new use without a version. Scanning of the accounts needs to know whether the data stored in the rent_epoch field has been rewritten or not. I guess it could've been handled at that time by waiting for 2 days for rent rewrites to clear out all the old data...

The scenario I am thinking of, (which is not just account hashing).

1. SlotX: bit Y is deprecated
2. SlotX + N: bit Y is re-used for another purpose
3. RPC Provider gets a request for an account
4. RPC provider needs to know how to interpret bit Y, and there are three scenarios

• a) It has a legacy value written before SlotX - Ignore bit status and use default value for the new field
• b) it has a legacy value written after SlotX, but before SlotX+N - Ignore bit status and use default value for the new field
• c) it has a new value written after SlotX + N - Report bit

The account version helps the provider differentiate between the three types.

This SIMD will change how the account metadata is interpreted and the new meaning needs to be agreed upon due to snapshots being shared across clients, even though it doesn't effect consensus. So the two options I see are calling it out in the SIMD itself, or having side channel discussions and agreements. I think calling it out in the SIMD is the better path, but am open to other options.

[brooksprumo]

I don't rent epoch could've been repurposed for a new use without a version. Scanning of the accounts needs to know whether the data stored in the rent_epoch field has been rewritten or not. I guess it could've been handled at that time by waiting for 2 days for rent rewrites to clear out all the old data...

For rent_epoch, we would've still had to:

• enable skipping rewrites feature gate
• enable the feature gate that stopped updating rent_epoch on accounts (I think that was Disable rent fees collection)
• enable serializing a constant into the VM for rent_epoch feature gate
• enable an imaginary feature gate that removed rent_epoch from the account hash algorithm

At that point, then we could've had another feature gate that repurposed the rent_epoch bytes, since those bytes would've been out of consensus, and thus available to use.

After that feature gate was activated, then it would be mostly the same situation as if there was an account version field or not.

The account state is valid as of a slot, say S. So if an account was written in the latest slot S, or if it was written millions of slots in the past, loading/accessing an account is interpreted according to the rules as of slot S. What I find neat is that if we add account versioning, there can be additional rules that we interpret next, like a tree.

---

I think the RPC case you're bringing up is interesting, so let's go through that.

First, this appears to be based on the implementation of the RPC that lives inside agave. So other RPC implementations may have differences, and I'll call those out too.

Let's consider three different scenarios:

1. Account Version has not been added

This is the behavior today. An RPC request for an account would return the account as it exists today. There's no account versioning, so no further rules to consider. Mostly uninteresting.

2. Account Version has been activated, and has been for a long time (1000s of slots)

In this scenario, any RPC request for an account will be based on a bank whose slot is after the feature activation of Account Version. E.g. I want to lookup account Alice, as of slot S. Let's say Alice was last written at slot before the feature was activated, like S - 1,000,000. Even though Alice's account was written pre Account Version, when loading, Alice's account it will be interpreted using the rules of slot S, not slot S - 1,000,000. We now consult Alice's account's version to interpret and return it. Note in this scenario the cluster is significantly past the feature activation slot of Account Version that it is not possible for the RPC to fulfill a request to get an account as of a slot pre feature activation.

And here's where other RPC impls could vary. One could imagine a theoretical impl, separate from agave, that creates and maintains a full copy of the whole account state plus chain state for every single slot. This imaginary impl never prunes old slots, and thus can fulfill any/all requests for accounts, regardless of slot. In our example, we could ask for Alice's account at slot S and at slot S - 1,000,000. This RPC impl would be return the correct slot-appropriate interpretation of the account for both slots, since it would have the chain state/feature status/etc for those slots.

With mainnet-beta today, I would say it is impractical to have an RPC impl that does not prune any old state ever. However, we have historical snapshots and ledger, thus we could use ledger-tool and replay to get the interpretation of an account at very old slots.

3. Account Version was just activated, like this very slot.

In this scenario, let's say slot S is the current root, and the Account Version feature gate was just activated. So slot S - 1 does not have Account Version. It is possible that an RPC request could come in for Alice's account as of slot S - 1 and the agave RPC node has not yet squashed/cleaned it up. Thus it would be able to fulfill that request. In this scenario, the account would be interpreted with the rules as of slot S - 1, which is pre Account Version.

I think the main thing is that there isn't ambiguity on how to interpret the account data because the RPC requests are always attached to a bank/slot.

This also applies to snapshots, which is great! That's because a snapshot is for a specific slot. So the account state will be interpreted in only one way, and that is based on the rules as of the snapshot slot.

[roryharr]

I don't think we are disagreeing on anything now are we? In general the version needs to be in the metadata, and its needed for hashing as well as interpreting the data loaded from the account meta data.

Whether it is interpreted when the account meta data is loaded compared with later is an implementation discussion, but not relevant to the SIMD.

The part i'm still stuck (before posting the SIMD is: How to handle the account meta data interpretation issue in the SIMD itself.

Any piece of code loading a snapshot needs to be aware of this information, and at this time, the agave client is not the only user of snapshots. Whether FD should roll their own snapshots is a separate discussion. My thought is to cover the change that describes that byteX of the account metadata should now be parsed as account version in the SIMD itself.

Maybe you also missed the part where I said adding it to the stored metadata would provide greater flexibility but I do not plan on going down that path.

[brooksprumo]

I don't think we are disagreeing on anything now are we?

Great!

Yeah, for the storing, I think we should repurpose the executable field. We know it's a whole byte, so can grab the top bits. We'll store the version in the top 7 bits (or however many we decide), and assuming we need to retain executable, that'll stay in the lsb.