RFC-0078 adaptation to Metadata V16 and Extrinsic V5

[carlosala]

With the Metadata V16 and extrinsic V5 currently stabilised, we need to find a way to pass the necessary information to the offline signers so they can decode it. I'm talking about extensionVersion and the enabled extensions for each of those versions.
These will most likely require a V2 version of the CheckMetadataHash.
Opening this to start the discussion, I'm still not sure which is the best way to proceed.

Note that, with Extrinsic V4, the CheckMetadataHash can be constructed with both Metadata V15 and V16 without further issues, and therefore the discussion doesn't need to be rushed.

Besides that, #124 has to be discussed (and eventually, approved) and Metadata V16 should be RFC-ed as well before landing on Polkadot.

cc @bkchr

[bkchr]

Thank you for starting the discussion. I think the easiest way would be probably to have one metadata hash per extension version?

[carlosala]

Actually, I was thinking that we need an update to CheckMetadataHash for Metadata V16 and extrinsic V4.
Which Extension should be used to build an extrinsic v4? Which extrinsicVersion?

That is something that we should probably specify in Metadata V16 rather than CheckMetadataHash, but it'll directly benefit from it.
WDYT @bkchr ?

[carlosala]

Thank you for starting the discussion. I think the easiest way would be probably to have one metadata hash per extension version?

Not a bad idea. It wouldn't be bad either to have all the extension versions together (just change the ExtrinsicMetadata in CheckMetadataHash V2). Having a constant in System.Version or elsewhere showing which CheckMetadataHash versions are available would be interesting as well.

[bkchr]

Which Extension should be used to build an extrinsic v4? Which extrinsicVersion?

Extrinsic V4 will always use extrinsicVersion = 0. This is nothing we need to take care of in CheckMetadataHash.

[bkchr]

It wouldn't be bad either to have all the extension versions together (just change the ExtrinsicMetadata in CheckMetadataHash V2)

Yeah should probably also work.

Having a constant in System.Version or elsewhere showing which CheckMetadataHash versions are available would be interesting as well.

Should just be exposed via the metadata of CheckMetadataHash.

[carlosala]

Extrinsic V4 will always use extrinsicVersion = 0. This is nothing we need to take care of in CheckMetadataHash.

But then the hash v1 is generated with only the signedExtensions present in extrinsicVersion = 0? That's an open question that should be clarified. If we generate the hash with all signed extensions we need then to transfer to Ledger or other devices a list of the signed extensions to be used.

Besides that, it should be specified that extrinsic v4 uses extrinsic version 0. It is the first time I read that.

Should just be exposed via the metadata of CheckMetadataHash.

There is no such a thing of "metadata" of an extension afaik. Maybe I misunderstood you.

[carlosala]

@bkchr this is becoming relevant.

Kusama Asset Hub (1007001) currently builds two different hashes when using metadata versions 15 and 16, when using the current algorithm. This is due to the lookup type ordering, which differs between versions 15 and 16.
I see two solutions, both adding a CheckMetadataHash V2:

1. Keep the current ordering (the one found in the lookup at every metadata), but clearly stating that CheckMetadataHash V2 is only for metadata v16.
2. Find a new lookup ordering that is resilient to lookup reordering in future metadata versions (or simply due to refactors in Polkadot-SDK code). E.g. numerical sort using the SCALE encoding of each entry.

I think I prefer option 2, as it is a bit more future-proof. It has one issue, though, that different metadata versions could have different lookups, or even produce a different number of types in the lookup; however, this would require a new CheckMetadataHash version anyway.

thanks @0xKheops for the report <3

[bkchr]

Discussed this with @carlosala in DM. The problem is that the frame_metadata_id is different across different versions of the metadata. Which is kind of expectable, because different metadata versions include different types (more or less). While writing the original RFC, I did not take into account that there are multiple versions of the metadata and that we need to pin the hash to a particular version. This is kind of implicitly expected, because the same metadata should lead to the same hash.

So, for we should always only generate the metadata hash for metadata V15. When writing the RFC for this issue here, we will make sure to directly include these information in the RFC.

[Lohann]

Just gave my 2 cents about this topic here:
paritytech/frame-metadata#110 (comment)

[bkontur]

Just asking, is there any progress or blocker? Is the frame_metadata_id the only issue? Or we are waiting for RFC: #124?