fix: use available prompt window for document context

[SidheshwarSarangal]

Summary

• Replaces the fixed 15% system-prompt cap in messageArrayCompressor with a budget based on the remaining prompt-window capacity.
• Reserves space for the response, user prompt, and maximum permitted chat history.
• Allows document context to reuse system-instruction capacity that was not needed.
• Adds a focused regression test for large document context.

Problem

When prompt compression was triggered, document context was limited to 75% of the system allowance. Since the system allowance was 15% of the model context window, document context could use only 11.25% of the available window—even when substantial capacity remained unused.

This caused large attached or pinned documents to be truncated much more aggressively than necessary.

Implementation

The message-array compressor now calculates the system and document-context allowance from:

• The model's prompt-window limit
• The 600-token response buffer
• The current user-prompt size
• The maximum permitted chat-history allowance

The existing user-prompt and history priorities remain unchanged.

This change intentionally does not modify the tokenizer fallback behavior, which is the other concern discussed in the related issue.

Validation

• Added a regression test for dynamic document-context budgeting
• Focused compressor test passes
• Generic OpenAI provider tests pass
• 10 relevant tests pass
• Production file passes ESLint
• Both changed files pass Prettier
• No whitespace errors

Connects #6142

[SidheshwarSarangal]

Hi maintainers, just following up on this PR when you have a chance. It addresses the document-context budgeting concern from #6142 and includes a focused regression test. Please let me know if any changes are needed. Thanks.

[SidheshwarSarangal]

Hi @timothycarambat, gentle ping when you have a chance. This PR addresses the document-context budgeting concern from #6142 and includes a regression test. I’d appreciate any feedback or requested changes.