[Deliverable] Isolate private chat traffic

[plopezlpz]

Summary

Dedicate a shard for private chat traffic, so that users are not impacted by other features (Communities, settings backup, pair/sync) that may be enabled by other features.
Ensure that features other than private chats can be disabled in status-go. Removed those for which consensus was reached.

https://github.com/vacp2p/rfc-index/blob/34aa3f3647cd5f0ae6a9af7fad47c3c8ee32c866/status/raw/status-app-protocols.md#global-general-scope

TODOs

• Investigate moving one-to-one and group chats in their own shard, including breaking changes and possible migration options.
• Agree on a shard to move traffic to and set it up in the fleet
• Move the private chats to that shard
• Test private chats with other features disabled: communities, settings backup and pair/sync

Tasks

• Investigate moving self-addressed messages to separate shard status-im/status-go#6234

[plopezlpz]

Status currently uses shards 32, 64 and, when setup, communities might use their custom shard. We are moving communities away from 32 and 64 (if they use their custom shard, they keep using it). We might just consider what to do with settings backup and pair sync and just leave private chats as they are in the current shards.
WDYT? @jm-clius @fryorcraken @jazzz

[jm-clius]

Yes, I think private chats can stay on the current shards with Communities, Backup and Sync split off.

[jazzz]

Leaving privatechats on their current shards makes sense to me.

[jazzz]

I will add that splitting pair/datasync messages from the chat messages may have some added complexity. The messages currently exist in the same topics, and handle acknowledgements as well as retransmission of missing messages. Separating the two will require that private chat clients connect to two separate shard to maintain normal operation. No strong opnions here other than a word of warning.

Backups are completely independent, though glad to see those moved.

[jm-clius]

Separating the two will require that private chat clients connect to two separate shard to maintain normal operation.

Thanks for the info! Are you saying then that the device sync procedures are somehow integrated with private chats? In this case, I wouldn't separate into two shards - we specifically want to remove "local only" traffic to a separate shard so that the client does not have to subscribe to that shard.

[fryorcraken]

Yes, I think private chats can stay on the current shards with Communities, Backup and Sync split off.

Is the intent to force migration of communities to the new shards? or just have "newly created communities" on the new shards?

If we force migration with a roll out strategy then fine to keep private chats where they are.
If not, then either let's force migration xor move private chats to their own shards.

[plopezlpz]

Is the intent to force migration of communities to the new shards? or just have "newly created communities" on the new shards?

the intent is to migrate to new shards current and new communities

[fryorcraken]

I will add that splitting pair/datasync messages from the chat messages may have some added complexity. The messages currently exist in the same topics, and handle acknowledgements as well as retransmission of missing messages. Separating the two will require that private chat clients connect to two separate shard to maintain normal operation. No strong opnions here other than a word of warning.

Backups are completely independent, though glad to see those moved.

Ok to re-iterate on this.

What I understand is that to achieve this current deliverable, we first deliver #266 (@plopezlpz is on it).

Then, we expect that backups are dropped. If not dropped, should not be too hard to move away (if we want to do the work).

Finally, we would need to move pair/datasync messages in their own shard (easier than moving 1:1 messages), but that's tricky.

Right?

I am still clarifying whether to do this work.

[jazzz]

move pair/datasync messages in their own shard

I realized I was assuming that "Pair sync" was referring to "Peersyncing" the process which Status handles resharing of messages from other peers using datasyncing. Is that true? or does pairsync refer to clients relaying copies of outbound messages to paired installations?

[jazzz]

@jm-clius

Are you saying then that the device sync procedures are somehow integrated with private chats?

First, I'm not entirely sure of the distinction between devicesyncing and datasyncing in this context, and feels like we might be referring to different things - In which case disregard.

The point I was clarifying is:
While peersyncing uses used datasync for synchronizing community messages, datasync is also used in 1:1 chat for read-receipts and acknowledgements. Without datasyncing all messages in Status would appear to users as being lost or "not delivered". In that way datasyncing and 1:1 messages are coupled together.