GEMM performance: OpenBLAS vs Accelerate

[xwuupb]

The performance gap between OpenBLAS and Accelerate is evident in the measured GEMM results presented here. I believe the main reason for this large difference is that OpenBLAS performs GEMM computations on CPU cores, whereas Accelerate may utilize the SME engine for GEMM. This can be verified through a simple experiment: increase the number of threads in the benchmark code to 2, 3, and 4. For OpenBLAS, I observed a proportional increase in performance. In contrast, the performance of Accelerate remains unchanged as the number of threads increases. Therefore, I think OpenBLAS and Accelerate use different execution units on Apple silicon.

[ViralBShah]

Perhaps the simplest thing to do here is to change the benchmark to Float64.

[xwuupb]

While reviewing the reported benchmark results here, I found myself considering two main questions:

1. Are these benchmarks sufficient to reflect real computational performance?
2. What explains the significant performance gap between OpenBLAS and Apple's Accelerate framework?

Since I have access to an Apple Silicon M4, I conducted additional benchmarks and would like to share my results and opinions.

Test environment:

• Hardware: Mac mini with Apple Silicon M4
• Julia version: 1.13.0-beta3
• AppleAccelerate.jl version: 0.6.0

Focus on Float64 GEMM benchmark results for N = 4096

• The serial code was taken directly from bench_dense.jl without modification.
• The multi-threaded code was modified manually at the following lines:
• Line 15
• Line 105
• Line 106

Results:

number of threads	OpenBLAS (GFLOPS)	OpenBLAS Speedup	AppleAccelerate.jl (GFLOPS)
1	58.3	1.0x	404.3
2	112.3	1.9x	404.2
3	164.1	2.8x	404.0
4	199.6	3.4x	401.3

Observations:

• OpenBLAS performance scales with the number of threads (although not perfectly).
• AppleAccelerate.jl performance remains essentially unchanged with more threads.

I believe it is insufficient, and can be misleading, to merely report single-threaded benchmark results here. Doing so may create the impression that OpenBLAS is poorly optimized for Apple Silicon and that the large performance gap persists regardless of multithreading.

Instead, it would be more informative to also include multi-threaded benchmarks and discuss the possible reasons behind the observed differences.

For Apple Silicon M4, my understanding is that OpenBLAS utilizes P cores (only), whereas Apple's Accelerate framework leverages the P SME Engine. More info are available in this paper, e.g., in Table 1, which explains why single-threaded DGEMM performance for OpenBLAS is around 58 GFLOPS. As threads increase, OpenBLAS performance improves, while Apple's Accelerate performance remains constant. This is actually consistent with the Apple Silicon core microarchitecture described in this document from Apple.

Sorry for this long post, to summarize:

1. Including multi-threaded benchmark results provides a more complete and fair performance comparison between OpenBLAS and Apple's Accelerate framework.
2. Short explanation about the performance differences helps readers and/or users when using OpenBLAS and/or AppleAccelerate.jl for serial and multi-threaded Julia codes.

[ViralBShah]

Right - my intention was originally to benchmark the quality of the kernels for single-threaded single CPU compute, but if Accelerate is using the GPU, that kind of defeats the purpose.

Happy to accept any PRs that make the benchmark and the docs more useful. Even updating the commentary in there could be valuable.

[xwuupb]

Thanks for your reply. However, I'd like to clarify a couple of points:

but if Accelerate is using the GPU, ...

On Apple Silicon M4 the Accelerate framework does not use GPU. Instead it leverages the so-called SME engine(s), co-processor(s) that operate alongside the CPU, one in P core cluster and one in E core cluster (none of them is GPU, details see here). Moreover, Apple's documentation for the Accelerate framework explicitly states that it is CPU-based (rather than GPU-based, see here). Last but not least, current Apple GPUs do not natively support Float64 (see Table 1 in this paper).

Happy to accept any PRs that make the benchmark and the docs more useful. ...

I agree that the benchmarks and documentation would benefit from being reworked to better reflect the computational performance of OpenBLAS and Apple Accelerate framework on Apple Silicon SoC. I'll try to contribute over the next few weeks using my spare time.