feat: add accelerate feature + document acceleration matrix

Add `accelerate` feature flag that enables Apple's Accelerate framework
(AMX hardware) for CPU matmul. Benchmark results on M3 Pro:

| Config                  | tok/s | Notes                           |
|-------------------------|-------|---------------------------------|
| CPU F16 (no features)   | 26.4  | Pure-Rust gemm, best CPU option |
| CPU F32 + accelerate    | 23.3  | 2.7x faster F32, but 2x memory |
| CPU F32 (no accelerate) |  8.5  | Baseline F32                    |
| Metal                   | 42.4  | Best overall on Apple Silicon   |

Key finding: F16 with pure-Rust gemm beats F32 with Accelerate because
the 2x memory bandwidth savings outweigh BLAS optimization. Apple's
Accelerate has no F16 BLAS (cblas_hgemm) — F16 is only available via
Metal MPS.

Also cleaned up CPU fallback path in Qwen3.5 full attention and added
comprehensive acceleration feature documentation to CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: evilsocket

CLAUDE.md

@@ -24,6 +24,22 @@ cargo build --release --features metal
 cargo build --release --features vulkan
 ```
 
+## Acceleration Features
+
+| Feature | Platform | Backend | Best For | Notes |
+|---------|----------|---------|----------|-------|
+| `metal` | macOS (Apple Silicon) | GPU via MPS + custom MSL kernels | Primary inference on Mac | Fastest option on Apple Silicon (~42 tok/s on M3 Pro) |
+| `cuda` | Linux (NVIDIA GPU) | GPU via cuBLAS/cuDNN | Primary inference on Linux | Requires CUDA toolkit matching driver version |
+| `accelerate` | macOS | CPU via Apple Accelerate (AMX) | CPU-only F32 inference on Mac | 2.7x faster than pure-Rust for F32 matmul; no F16 support |
+| `vulkan` | Any (Vulkan 1.3+) | GPU via Vulkan compute shaders | Steam Deck, AMD GPUs | Portable but less optimized than Metal/CUDA |
+| (none) | Any | CPU via pure-Rust `gemm` | Portable CPU fallback | F16 weights stay F16, avoids bandwidth doubling |
+
+**When to use which:**
+- **Apple Silicon (stevie.local):** Use `--features metal`. Metal is 1.6x faster than CPU F16 (42 vs 26 tok/s). The `accelerate` feature doesn't help with Metal and doesn't support F16 matmul, so CPU F16 (default, no features) is actually faster than `accelerate` with F32 (26 vs 23 tok/s).
+- **NVIDIA GPU (blade/bahamut):** Use `--features cuda`. Add `flash-attn` for flash attention support.
+- **CPU-only with F32 models:** Use `--features accelerate` on macOS for 2.7x faster F32 matmul. On Linux, consider linking against MKL or OpenBLAS.
+- **CPU-only with F16 models:** Use no features — pure-Rust `gemm` with F16 avoids the 2x memory bandwidth penalty of converting to F32.
+
 ## Interactive Chat
 
 ```bash

cake-core/Cargo.toml

@@ -70,6 +70,7 @@ base64 = "0.22.1"
 default = ["master", "llama", "qwen2", "qwen3_5", "qwen3", "qwen3_moe", "qwen3_5_moe", "phi4", "mistral", "gemma3", "falcon3", "olmo2", "exaone4", "flux", "vibevoice", "luxtts"]
 
 metal = ["candle-core/metal", "candle-nn/metal", "candle-transformers/metal", "dep:candle-metal-kernels", "dep:objc2-metal"]
+accelerate = ["candle-core/accelerate", "candle-nn/accelerate", "candle-transformers/accelerate"]
 cuda = ["candle-core/cuda", "candle-nn/cuda", "candle-transformers/cuda", "dep:bindgen_cuda"]
 flash-attn = ["cuda", "dep:candle-flash-attn"]
 vulkan = ["dep:ash", "dep:gpu-allocator", "dep:bytemuck"]

cake-core/src/models/qwen3_5/full_attention.rs

@@ -238,18 +238,16 @@ impl Qwen3_5FullAttention {
                     .map_err(|e| anyhow!("att matmul v: {e}"))?;
             }
 
-            // CPU fallback: manual attention with GQA head expansion
+            // CPU: manual attention with GQA head expansion
             let k = self.repeat_kv(k).map_err(|e| anyhow!("repeat_kv k: {e}"))?;
             let v = self.repeat_kv(v).map_err(|e| anyhow!("repeat_kv v: {e}"))?;
             let att = (q.matmul(&k.t()?)? / (self.head_dim as f64).sqrt())?;
-            let att = if seq_len == 1 { att } else {
-                let mask = cache.mask(seq_len, att.device())
-                    .map_err(|e| anyhow!("mask: {e}"))?
-                    .broadcast_as(att.shape())
-                    .map_err(|e| anyhow!("mask broadcast: {e}"))?;
-                masked_fill(&att, &mask, f32::NEG_INFINITY)
-                    .map_err(|e| anyhow!("masked_fill: {e}"))?
-            };
+            let mask = cache.mask(seq_len, att.device())
+                .map_err(|e| anyhow!("mask: {e}"))?
+                .broadcast_as(att.shape())
+                .map_err(|e| anyhow!("mask broadcast: {e}"))?;
+            let att = masked_fill(&att, &mask, f32::NEG_INFINITY)
+                .map_err(|e| anyhow!("masked_fill: {e}"))?;
             let att = self.backend.softmax(&att, att.rank() - 1)?;
             att.matmul(&v.contiguous()?)?
         };