Skip to content

[CPU][PA] Align bidirectional image attention and sliding window interaction with reference#35446

Merged
p-wysocki merged 4 commits intoopenvinotoolkit:masterfrom
p-wysocki:cpu_bidir_attn
May 4, 2026
Merged

[CPU][PA] Align bidirectional image attention and sliding window interaction with reference#35446
p-wysocki merged 4 commits intoopenvinotoolkit:masterfrom
p-wysocki:cpu_bidir_attn

Conversation

@p-wysocki
Copy link
Copy Markdown
Contributor

@p-wysocki p-wysocki commented Apr 21, 2026

Details:

  • When both bidirectional image attention and a sliding window were active, the CPU plugin computed the attention start index as:
    start_idx = image_group_end - sliding_window
    Whereas the correct way to do it (implemented by transformers and GPU Plugin) is to not clip the image attention, but instead pass it as a whole block, no matter what the sliding window size is.

  • An example

Let's assume 6 image tokens and a sliding window of size 5.

Before the fix:
Attention mask: [1, 6) - first token cut

After the fix:
Attention mask: [0, 6) - full group

If attention mask is being calculated for a text token, the attention is regular causal/sliding window, no matter if previous tokens were image or text, which matches transformers implementation.

Tickets:

AI Assistance:

  • AI assistance used: yes, to write tests and make sure the changes don't impact other aspects/inputs of PA

Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
@p-wysocki p-wysocki added this to the 2026.2 milestone Apr 21, 2026
@p-wysocki p-wysocki requested review from a team as code owners April 21, 2026 12:26
@github-actions github-actions Bot added the category: CPU OpenVINO CPU plugin label Apr 21, 2026
@maxnick
Copy link
Copy Markdown
Contributor

maxnick commented Apr 29, 2026

@zhangYiIntel , could you please review?

@zhangYiIntel zhangYiIntel self-requested a review April 30, 2026 22:32
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
@p-wysocki p-wysocki enabled auto-merge May 4, 2026 08:23
@p-wysocki p-wysocki added this pull request to the merge queue May 4, 2026
Merged via the queue into openvinotoolkit:master with commit f1c2004 May 4, 2026
196 checks passed
@p-wysocki p-wysocki deleted the cpu_bidir_attn branch May 4, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: CPU OpenVINO CPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants