Memory Optimizations for Large Sessions

[chezsmithy]

During longer sessions where the compaction feature is utilized, the chat history can become quite large in the memory / DOM. This can cause the extension to become sluggish and in extreme cases can cause certain components to render incorrectly.

I'd like to recommend a couple of approachs to managing the chat history in memory post compaction:

1. Prune everything before the compaction and update the session stored on disk to reflect this. (simple)
2. Update session management so that when compaction occurs the messages before the latest compaction are stored in the session on disk but they are removed from memory. Only bring them back into memory if the session is reloaded or perhaps add a UX element to allow a user to reload messages before the latest compaction, but prune them again as soon as the session is resumed.

[gamesguru]

Hi, I agree the performance is an issue. I am working on a PR which has improvements via the Virtuoso library.

#9487

Unfortunately, the PR is much broader than this one enhancement, and it is not stable enough to recommend for general usage.

You can try it out though and let me know if you have any issues, either here or email is fine. We can come up with some more test cases to add. I am mostly manually testing in PyCharm and Linux. I can configure my pipeline to output artifacts for your desired IDE. I would appreciate any help in getting this tested and flushed out, thanks for bringing this to everyone's attention.