Skip to content

perf(chat): improve performance of the chat buffer#2941

Merged
olimorris merged 2 commits into
mainfrom
perf/chat-buffer
Mar 28, 2026
Merged

perf(chat): improve performance of the chat buffer#2941
olimorris merged 2 commits into
mainfrom
perf/chat-buffer

Conversation

@olimorris

@olimorris olimorris commented Mar 26, 2026

Copy link
Copy Markdown
Owner

Description

I've been meaning to tighten performance on the chat buffer for sometime. It's felt slow when chat history gets big. Every request at a context over 100,000 tokens seems to be delayed by a split second.

I asked Opus 4.6 to conduct a review on the chat class and identify performance improvements. It's findings were:

Priority Issue Impact
1 vim.deepcopy(self.messages) on submit High — scales with conversation length
2 check_context() O(n^2) High — compounds with many tools/contexts
3 make_safe() called 3x per submit Medium — builds tables + iterates schema each time
4 sync_all_buffer_content nested iteration Medium — per-submit cost
5 Chat.new() synchronous initialization Medium — affects perceived startup time
6 _G.codecompanion_buffers linear scan Low — only on close, small N
7 Stable context values recomputed Low — cheap individually

The first 3 were all great catches, which I missed. Possibly because I'm too close to the chat class itself ("can't see the wood for the trees") and the fact I hadn't appreciated the impact of what vim.deepcopy does when it traverses a complex table structure.

For those who are still unsure of what value AI brings to their codebase, I'd reference the commits in this PR and the overall scope which I gave Opus 4.6. I liken it to having an experienced pair of eyes look at my code for what it is with no pre-conceived ideas and biases.

AI Usage

Opus 4.6 (via Claude Code)

Related Issue(s)

Screenshots

Checklist

  • I've read the contributing guidelines and have adhered to them in this PR
  • I confirm that this PR has been majority created by me, and not AI (unless stated in the "AI Usage" section above)
  • I've run make all to ensure docs are generated, tests pass and StyLua has formatted the code
  • (optional) I've added test coverage for this fix/feature
  • (optional) I've updated the README and/or relevant docs pages

@olimorris olimorris marked this pull request as ready for review March 28, 2026 09:56
@olimorris olimorris merged commit f0c6487 into main Mar 28, 2026
11 checks passed
@olimorris olimorris deleted the perf/chat-buffer branch March 28, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant