Skip to content

Conversation

@ilya-lavrenov
Copy link
Contributor

@ilya-lavrenov ilya-lavrenov added this to the 2025.2 milestone May 6, 2025
@github-actions github-actions bot added the category: LLM LLM pipeline (stateful, static) label May 6, 2025
@Wovchena Wovchena requested a review from Copilot May 6, 2025 11:42
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enables paged attention (PA) support on ARM64, broadening the scope of architectures that can leverage this functionality.

  • Removed the forward declaration for with_cpu_sve() since its runtime support is no longer conditionally required.
  • Modified preprocessor conditions to enable PA on ARM64 along with x86_64.
  • Updated error messages to consistently reflect that PA is supported on x86_64 and ARM64 platforms only.
Comments suppressed due to low confidence (3)

src/cpp/src/llm/pipeline.cpp:50

  • The updated error message for the continuous batching backend now correctly reflects support on ARM64. Ensure that the relevant documentation is updated accordingly.
OPENVINO_THROW("Continuous batching backend requires PagedAttention operation support, which is available on x86_64 or ARM64 platforms only");

src/cpp/src/llm/pipeline.cpp:57

  • The error message for speculative decoding now correctly indicates support on both x86_64 and ARM64. Confirm that this behavior aligns with the expected backend logic for ARM64.
OPENVINO_THROW("Speculative decoding requires PagedAttention operation support, which is available on x86_64 or ARM64 platforms only");

src/cpp/src/llm/pipeline.cpp:64

  • The error message for prompt lookup decoding is updated to reflect ARM64 support. Verify that there is no implicit dependency on SVE requirements within the ARM64 code path.
OPENVINO_THROW("Prompt lookup decoding requires PagedAttention operation support, which is available on x86_64 or ARM64 platforms only");

@ilya-lavrenov ilya-lavrenov merged commit 60b40bf into openvinotoolkit:master May 6, 2025
73 of 76 checks passed
@ilya-lavrenov ilya-lavrenov deleted the enable-pa-for-arm64 branch May 6, 2025 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: LLM LLM pipeline (stateful, static)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants