[Bug]: Memory's flushing behavior in _manage_queue may end up with an assistant message as the first message

[GinkREAL]

Bug Description

If Memory's flushing limit is set low enough such that one conversation turn is enough to make it overflow and trigger the flushing behavior, it is possible to end up with a list of active messages that begin with a assistant message instead of a user message.

Some providers like Amazon Bedrock do not allow this and return an error, since they require that the first message be a user message.

Version

0.12.52.post1

Steps to Reproduce

1. Create FunctionTool such that the token count returned by the tool triggers Memory's flushing behavior
2. Create any chat
3. Observe that the remaining message in the active queue is only the latest assistant reply

Relevant Logs/Tracbacks

[dosubot]

You’re right—there’s a corner case in the Memory class’s _manage_queue logic where, if the token limit and flush size are set low enough, the active queue can end up with only an assistant message remaining. The code does try to maintain conversation integrity by keeping the first message as a user message, but if only one message fits after flushing, and it’s an assistant reply, there’s no safeguard to restore a preceding user message. This can cause errors with providers like Amazon Bedrock that require the first message to be from the user. You can see the relevant logic and its limitations in the source code here.

A related issue discusses that translating flushed messages into a single string (as done in the vector memory block) could help increase compatibility across providers that have strict requirements on message order—see the discussion here.

For now, if you need to avoid this, you may want to ensure your token limits and flush sizes are set so that at least one full user→assistant turn remains in the queue, or consider patching the flushing logic to always preserve a user message as the first active message.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Prozac614]

You’re right—there’s a corner case in the Memory class’s _manage_queue logic where, if the token limit and flush size are set low enough, the active queue can end up with only an assistant message remaining. The code does try to maintain conversation integrity by keeping the first message as a user message, but if only one message fits after flushing, and it’s an assistant reply, there’s no safeguard to restore a preceding user message. This can cause errors with providers like Amazon Bedrock that require the first message to be from the user. You can see the relevant logic and its limitations in the source code here.

A related issue discusses that translating flushed messages into a single string (as done in the vector memory block) could help increase compatibility across providers that have strict requirements on message order—see the discussion here.

For now, if you need to avoid this, you may want to ensure your token limits and flush sizes are set so that at least one full user→assistant turn remains in the queue, or consider patching the flushing logic to always preserve a user message as the first active message.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other    

Hi @dosu, thanks for the clear analysis!

I’d be interested in looking into a fix for this edge case in the Memory._manage_queue logic.

Rough idea:

• Ensure that after flushing, the active queue always starts with a user (or possibly a system) message, even when only a single message fits within the token limit.
• If the only remaining message would be an assistant reply, either:
  • restore a preceding user message (even if it slightly exceeds the token limit), or
  • prepend a summarized user/system message representing the flushed history.

Before I start prototyping, does this direction align with how you’d like Memory to behave across providers (especially Amazon Bedrock)?

If so, I’d be happy to open a PR and add tests for this scenario.