Enhanced KV-store / Key-value experience

[taoeffect]

Problem

Creating apps that use the KV store is a bit too cumbersome.

1. It requires having pubsub listeners in different files
2. It requires complex handlers that take onConflict parameters and etag stuff, e.g. const getUpdatedUnreadMessages = ({ currentData = {}, etag } = {}) => {
3. We're currently adding multiple selectors for every type of KV we are manipulating

Solution

We want a very simple API that is super easy to use for handling KV updates that uses Chelonia to automatically decide where in the state these keys are stored, and creates a simple API for creating updates and handling conflicts.

See okTurtles/group-income#2903 for details.

Also see the convenience function updateKVPreferences from Sebin's recently merged PR in identity-kv.js.

The user should not be responsible for anything related to pubsub events when dealing with KV stuff.

The pubsub KV logic could still be available for more complicated use-cases.

EDIT: since we would be moving most of the contract actions into the KV store (e.g. state.attributes) we should also think about how this new friendlier KV API would handle validation. So we could for example say that "whenever this key is validated please use this zod schema to validate".

[corrideat]

we should also think about how this new friendlier KV API would handle validation

Adding some context from today's conversation in relation to this point. There are two kinds of validation of note when it comes to the KV store (and compared to the current on-chain processing).

When we use a schema, we can check for well-formedness, which is an overall improvement, and we should do this for sending data out and for reading data in.

However, with contracts we have a different kind of 'validation', which is the ability to reject specific messages (be it because of invalid form or because of invalid semantics (e.g., trying to perform an action you're not allowed to)). Because we can reject specific messages in the chain and leave the state effectively unchanged, on-chain invalid messages don't break the state.

With the KV store, since there's no concept of a history / chain (which is the point of the KV store), a change, once written, becomes the 'truth', and the checks that can be done to accept or reject a write are server-side and pretty rudimentary.

What this means is that if we were to use the KV store for things where invalid data (due to form or semantics) would break something (and it couldn't be broken in the same way with on-chain actions), then we need to find a way of preventing this. One thing that comes to mind is voting, but surely there are other ways.

Concrete example: say we use the KV store to store permissions ({ uid1: [permission1, permission2], uid2: [permission1], ... }). Someone with delegator access could write to the 'permissions' KV-key and make themselves admin of a group. With on-chain, this isn't possible because, even if they sent such an action, it'd be rejected by other clients and ignored.

[taoeffect]

What this means is that if we were to use the KV store for things where invalid data (due to form or semantics) would break something (and it couldn't be broken in the same way with on-chain actions), then we need to find a way of preventing this. One thing that comes to mind is voting, but surely there are other ways.

So, I'm thinking that instead of voting what we could do (if it was desirable to have advanced logic on what is considered a valid value under a key), would be to create a special KV namespace (e.g. contractKeys_*), and for creating a key under this namespace would require referencing the contractID of a special contract chain that existed for the sole purpose of validating values under this key. This would enable the creation of arbitrary logic for validating KV keys in a historically consistent way.

EDIT: Or, actually, as it would be a bit wasteful to create a new chain for every single key under this namespace, what could be done would be to reference an existing chain (e.g. think of chatroom.js as an example) that implemented a specific API (e.g. a specific action selector) that would validate key-value pairs in the KV namespace. And actually, it would make sense that the contract to be used for this would be the contract of the KV namespace that we're using (since all KV stuff has to happen under a specific contractID namespace).

So actually we'd dedicate a specific contract method (not action) for validating KV keys. So we define an API to be implemented that looks something like this in the contract:

methods: {
  "gi.contracts/chatroom/_kvValidation" ({ key, value }, { state }) {
    // here we'd either do nothing, or throw an exception based on the values of `key` and `value`
  }
}

// OR:

methods: {
  "gi.contracts/chatroom/_kvValidation/<key>" ({ value }, { state }) {
    // here we'd either do nothing, or throw an exception based on the values of `key` and `value`
  }
}

And so any time anyone attempts to update a key, it would first be validated by this method.

Downsides:

• this doesn't prevent invalid values from being written to the KV store (but it does identify them)
• it would result in inconsistent validation rules being applied at different points in time because the contract version can change

I'm not saying we should do this, just jotting down this idea.

[taoeffect]

The enhanced KV API should primarily focus on the kv/set side of things, so that when a key is set, it will automatically result in an update to all clients listening for that key and update their corresponding Vuex state under a key like state._kv.<keyName> to something to that effect.

Another selector would be used to mark which keys you're interested in receiving updates for, so that you can restrict what keys exist under state._kv.