qwen3_5 example config leaves use_sdpa_with_kv_cache off, which costs 2x decode

[john-rocky]

examples/models/qwen3_5/config/qwen3_5_xnnpack_fp32.yaml sets

  use_sdpa_with_kv_cache: False

with no comment saying why, while examples/models/lfm2/config/lfm2_xnnpack_q8da4w.yaml — the same kind of hybrid attention/convolution decoder — sets it True. Turning it on for Qwen3.5 roughly doubles decode.

Measured

Qwen3.5-2B, XNNPACK, 8da4w + 8-bit embedding, the same checkpoint exported twice with only that line different, both measured in one pass on an idle machine (Mac arm64, llm_params/gen_static.py-style token-by-token prefill then greedy decode):

	decode
use_sdpa_with_kv_cache: False	8.20 tok/s
use_sdpa_with_kv_cache: True	16.64 tok/s

Same prompt, same 32 new tokens. (Both files are 1483 MB; the flag does not change the size.)

That it is the same model, not a faster different one

• Four prompts run through both builds: three came out token-identical. The fourth differs where a thinking model decides how long to stay inside <think>, which is a near-tie in the logits either way.
• First-step logits from the two builds, same prompt: corr 0.995, same top-1.
• 338 tokens into the context, same comparison: corr 0.990, same top-1 and top-2.

That is the size of difference two attention kernels give each other, not the size a wrong one gives.

Ask

If it is off deliberately — some interaction with the linear-attention layers I have not hit — a comment in the config would save the next person the experiment. If it is not deliberate, I am happy to send the one-line change.

Measured with the executorch 1.4.0 wheel; both config files are unchanged on main at e4576d0.

[JacobSzwejbka]

@Gasoonjia any idea why its off?