Question about the KV Cache Design between PaliGemma and Gemma Expert

[Xubbbb]

Body:

Hi,

I have a question regarding the design of the KV cache in your model. Specifically, I noticed that in your implementation, the KV cache from PaliGemma (which processes both images and text) is directly concatenated in front of the KV cache for the Gemma Expert before computing attention.

However, the weight matrices Wq, Wk, Wv in PaliGemma are different from those in the Gemma Expert. Given that they are not the same, what is the theoretical basis for placing their KV caches together in the attention computation?

I would appreciate any insights you could provide regarding the reasoning behind this design choice.

Thanks!