List of supported XCM assets Runtime API

[mrshiposha]

The current version of the pallet-xcm makes it easy to transfer assets without thinking of what transfer type you need as the pallet computes that by itself (thanks to the transferAssets extrinsic). The ORML pallet-xtokens does a similar thing with its transferMultiassets extrinsic.

This simplifies the XCM interaction from the JS side drastically and allows the building of more convenient UIs.

However, at the moment, users cannot discover the complete list of supported XCM assets uniformly in the ecosystem chains.
Note: "Supported" means the asset can be withdrawn/transferred/deposited on that chain. It doesn't matter if the asset is supported as a fee asset.

This leads to the creation of custom XCM asset registries that must be constantly maintained.
Such lists of supported XCM assets are essential for building convenient UIs for the end-users.

If we had a uniform way of asking chains what they support, we'd have an opportunity to build general (and simple!) XCM JS tooling and much better UIs that are always up-to-date with chains' valid state.

Suggestion

Of course, we could develop a Runtime API that returns a simple list of supported assets.
This would've been enough for fungibles, as their asset location and amount fully identifies the balances.
For non-fungibles, however, we must also know the asset location's supported AssetInstance variant (Index, Array4, etc.).
So, the Runtime API should return the list of supported asset locations and also report their fungibility variant.

Native approach

We could add additional types for such Runtime API responses. However, I think this is neither a future-proof nor a convenient approach (the alternative is in the next section).
Nonetheless, I will outline these possible extra types:

enum SupportedFungibility {
    Fungible,
    NonFungible(VersionedSupportedNonFungible)
}

// Must also have a `Versioned*` type
enum SupportedNonFungible {
    Index,
    Array4,
    /*
      <...>
       the same variants as in the `AssetInstance` but without parameters
    */
}

// Must also have a `Versioned*` type
struct SupportedAsset {
    id: VersionedAssetId,
    fun: VersionedSupportedFungibility,
}

And the Runtime API could look like this:

trait SupportedXcmAssets<Metadata> {
    fn query_supported_assets(version: Version) -> Result<Vec<VersionedSupportedAsset>, Error>;

    // chain-specific info like `is_sufficient`. Useful for UIs
    fn query_asset_metadata(asset: VersionedAsset) -> Result<Metadata, Error>;
}

URI approach

I believe a better approach is to define a URI scheme for XCM resources to report information about them via Runtime API.
With it, we aren't tied to a particular programming language and its types. Yet, it is still easy to process, as the URI parsing tools are available in basically every programming language. Also, the URI form is easier to type and read if you need to deal with XCM assets more or less directly.

Note: if other XCM Runtime APIs also used the URI, it would've been better because of the uniform and language-independent approach.

So, instead of returning the extra types shown above, the Runtime API would return just a URI and chain-specific metadata:

type Xcmi = String;
type Metadata = String; // A simple string of `(key=value)*`

trait SupportedXcmAssets {
    fn query_supported_assets(version: Version) -> Result<Vec<Xcmi>, Error>;

    // chain-specific info like `is_sufficient`. Useful for UIs
    fn query_asset_metadata(xcmi: Xcmi) -> Result<Metadata, Error>;
}

We might define the scheme as xcmi:// [asset-location] [?fungibility-variant].

The example output of the query_supported_assets(3):

• DOT:
  xcmi://v3/GlobalConsensus(Polkadot)?fungible
• USDT:
  xcmi://v3/GlobalConsensus(Polkadot)/Parachain(1000)/PalletInstance(50)/GeneralIndex(1984)?fungible
• Unique Network's NFT collection 2:
  xcmi://v3/GlobalConsensus(Polkadot)/Parachain(2037)/GeneralIndex(2)?nonfungible=Index(*)
• Moonbeam's NFT collection of a contract 0x...:
  xcmi://v3/GlobalConsensus(Polkadot)/Parachain(2004)/AccountKey20(0x...)?nonfungible=Array32(*)

This makes it easier to read and use asset information in any language and allows the unification of certain interfaces.

Index(*) and Array32(*) mark the non-fungible ID variant used but don't specify a concrete token. Instead, the * is reported here, notifying the user that the NFT collection is supported regardless of a concrete token.

For instance, look at the query_asset_metadata in the last code snippet.
We can:

• Query the metadata of a token collection/class:
  query_asset_metadata("xcmi://v3/GlobalConsensus(Polkadot)")
• Query the tokenUri of Moonbeam's NFT:
  query_asset_metadata("xcmi://v3/GlobalConsensus(Polkadot)/Parachain(2004)/AccountKey20(0x...)?nonfungible=Array32(0x...)")
• Query the properties of Unique's NFT (collection=2, token=15):
  query_asset_metadata("xcmi://v3/GlobalConsensus(Polkadot)/Parachain(2037)/GeneralIndex(2)?nonfungible=Index(15)")
• Query the properties of the Unique's collection 2:
  query_asset_metadata("xcmi://v3/GlobalConsensus(Polkadot)/Parachain(2037)/GeneralIndex(2)")

BTW, it would've also been great to have a method for uniformly querying an account's balance.
fn query_balance(account: AccountId, xcmi: Xcmi) -> Result<u128, Error>.

This would work consistently with fungibles and non-fungibles:

• if a user has an NFT fully identified by the xcmi, then its balance is 1
• if we identify the NFT collection instead, the balance will equal the number of owned NFTs in that collection.

CC @franciscoaguirre @xlc

[xlc]

I prefer the native approach. For a future proof solution, we should use XCQ

[mrshiposha]

Sure, we can use the native approach. However, I'd propose introducing a proc-macro to generate versioned types for XCM rather than maintaining them by hand. Actually, we at Unique Network have a similar proc-macro. We can extend it a bit and publish it to crates.io so we can reuse it in XCM for versioned types.

As for the URI (BTW, it is better to use the URL WHATWG spec as the tools work with this better), I think we need to define it for clients at least -- a standard language-independent representation that can be converted to and from SCALE. Then, we could implement a simple JS library for the conversions. Such representation is easier to type and read and could facilitate interoperability between different ecosystems. I found an RFC draft in XCM format about a similar representation. However, it doesn't include the XCM version and fungibility variant, so it can neither convey information about the fungibility of the asset (PalletInstance(50)/GeneralIndex(1984)?fungible) nor fully represent assets (PalletInstance(50)/GeneralIndex(1984)?fungible=700). Maybe we should continue this discussion there.

About XCQ: I briefly looked at it but didn't quite understand how it could be used. Is there any documentation? Is it something similar to ChainQL?

[xlc]

You can read this about XCQ: https://forum.polkadot.network/t/cross-consensus-query-language-xcq/7583
It is a replacement of https://forum.polkadot.network/t/wasm-view-functions/1045

We will define an XCQ extension, requires chains to support such extension, and client uses a pvm program to query information.

It is will also possible for the pvm program to query raw storages directly and do whatever it wants so for some use cases, it is possible to support chains that did not integrate XCQ.

[mrshiposha]

@xlc on second thought, the URI form will become hard to read if we consider junctions with several parameters (even AccountId32 has two, where one of them is optional in V3/V4 but mandatory in V2). So, this fact defeats the readability of the URI form of locations/assets.

About the native approach: to make it more future-proof, we'd need to return a pattern of supported assets, not a list. For instance, if a chain supports many assets of one location pattern, returning a list with repeating data would be wasteful.
It won't be a problem now, but imagine thousands of XCM-enabled FT/NFT collections for various purposes (including utilitarian/technical ones).
With this approach, the client can still check if the concrete asset is supported. The client can fill in one of the supported patterns with actual values to form the concrete asset and then check if the chain supports it.

[xlc]

yeah in that case it could be easier to have a runtime API to check if an asset is supported. the UI have to somehow figure out the asset list anyway.

[kianenigma]

Similarly, in favor of the native approach. Runtime APIs are under-utilized, and a standard set of Runtime APIs that all chains implement can significantly simplify the code of wallets IMO.

But, not keen on keeping this limited to "what assets are available for XCM transfer", but rather "list all assets that I have". A field of this can then be whether this asset can be transferred locally, or cross chain.

enum Transferrable {
    Yes, 
    /// Un-opinionated max-16 char description of why an asset is not transferrable.
    /// Abstraction around Hold/Freeze reasons, or any other reason why a runtime 
    /// might lock assets. 
    No([u8; 16])
} 

struct AssetInfo {
    /// XCM AssetId of this asset in the local consensus system. 
    asset_id: MultiAsset,
    transferrable: Transferrable,
    cross_consensus_transferrable: bool 
}

trait StandardAssetApi {
    fn assets(who: AccountId) -> Vec<AssetInfo>
}

[mrshiposha]

trait StandardAssetApi {
   fn assets(who: AccountId) -> Vec<AssetInfo>
}

It looks good and is easy to use. I like it.

However, I'm concerned about how this will work for NFTs.

Note: I see that AssetInfo uses MultiAsset as the type of asset_id, but judging by the field's name and the doc comment, I assume it is a typo.

If the AssetInfo reports only AssetIds, a user would need to guess the AssetInstance for NFTs or the maximum amount they could transfer for fungibles.

It seems it would've been more convenient to return Asset in AssetInfo so the user would know precisely what they can transfer or not. However, this looks like a DoS possibility if an account has an absurd number of NFT tokens. Can we do this without introducing an attack vector?

Another issue is that the user can't know if another chain can accept a given asset. Maybe the new DryRunApi for XCM could report whether an asset would've been trapped if transferred to a given location.

[xlc]

it is not always possible to return list of all support assets. think about EVM chains that there could be a lot of ERC20s. it is infeasible for the runtime to iterate all the contracts to identify the one implements ERC20. or just AssetHub with a large amount of user created assets. it will be inefficient and requires pagination at least.

[mrshiposha]

I have to agree with @xlc. A large number of asset collections (as well as non-fungible tokens) preclude reporting the lists even if we inspect them for a single selected account (the EVM case).

Is there a possibility of having a standard API (maybe a rate-limited one) for scanners, at least? So that the scanners could retrieve the collections (fungible/non-fungible) and tokens info in a standard way across the ecosystem?

[kianenigma]

@xlc @mrshiposha possibility of difficulty implementing an API IMO should not prevent that API from existence, if it does solve a problem for some, and it is sensible. I argue that a general assets API is both sensible, and indeed does solve a problem for at least some chains. For the rest, more complicated solutions are needed.

• regarding NFTs, yes my idea is only about fungible tokens, not NFTs.
• regarding complexity, a particular runtime that has permissionless ERC20 can return a standard error, or decide to paginate, or anything in between¹. This is why how this API is implemented should be up to each runtime, and not one universal solution. But many runtimes can already benefit from this, so I would argue it is a good idea.

Footnotes

1. For the case of AH, we can say this API will only return the sufficient assets, for example, which have a more limited count than any shitcoing created on the fly. ↩

[xlc]

I guess there isn't much harm to have an optional API to return all "main" assets as long as it is clear what the API is doing.