Support inserting messages out-of-order by timestamp?

[alphapapa]

Hi,

We're looking into using LUI for our Matrix client: alphapapa/matrix-client.el#15 However, Matrix messages need to be deduplicated by message ID and inserted in order by timestamp, e.g. when resyncing with a room or loading earlier messages. LUI already supports timestamps and messages IDs, but the insertion function seems to just insert new messages at the end of the buffer.

Would you be open to enhancing LUI to support insertion by timestamp, and conditionally on whether a message ID already exists in the buffer? Maybe the insertion function could be set with a variable so we could just rewrite that function for our own use, binding to our function around our code that calls LUI functions.

Thanks.

[jorgenschaefer]

Hello, and thanks for the suggestion! Support for deletion of messages is rather new, and introduced the message ID concept at all. I don't see a reason not to support updating or out of order insertion now as well. :-)

[alphapapa]

Thanks, that sounds great.

About deduplication, if you are interested in that: I guess scanning a large buffer of messages for a text-property with a certain value might get slower over time, as the buffer grows, so perhaps a hash-table could optionally be used for deduplication as well.

[jorgenschaefer]

Hm. If you add that, make sure it's an optional feature, since many applications do not use it at all. That might even be something for the user of lui, not lui itself, to handle. I.e. lui provides lui-insert and lui-update, and the client code has to remember if it already sent the message or not.

[alphapapa]

FYI we've added this feature to matrix-client-el. It's probably not implemented as cleanly as you could do it, but it seems to work so far. :)

The issue of scanning the whole buffer for message IDs is mitigated by using date headers and message timestamps, so after finding the correct place to insert a message (go to correct header, then go to correct timestamp position before the next header), only the following message ID needs to be compared.

[Thaodan]

Hello, and thanks for the suggestion! Support for deletion of messages is rather new, and introduced the message ID concept at all. I don't see a reason not to support updating or out of order insertion now as well. :-)

Could be useful for:
https://ircv3.net/specs/extensions/message-ids