[SYCL] Keep fast kernel cache entries when the kernel name is already registered

[Agusx1211]

KernelProgramCache::saveKernel registers the kernel name in MFastKernelCache with

MFastKernelCache.try_emplace(std::string(KernelName), FastKernelSubcacheWrapper(KernelSubcache, getURContext()));

The wrapper temporary is built before the call and destroyed after it, whether it was inserted or not. When the name is already registered (every fast cache miss after the first one for that kernel) the temporary is destroyed intact, and ~FastKernelSubcacheWrapper erases every entry of the current context from the kernel's subcache.

With a single device this goes unnoticed, the entry that was just erased is added right back. With two or more devices in one context it makes the fast cache useless: a launch on device B erases the entry for device A, the next launch on A misses, erases B, and so on. Every one of those misses goes through the slow path and appends a (std::string kernel name, device) pair to MProgramToFastKernelCacheKeyMap, which is never trimmed while in-memory cache eviction is off (the default). On a llama.cpp server running Qwen3.8-27B tensor-split across two Intel Arc Pro B70s (oneAPI 2026.1, one context, about a thousand launches per decode step) this leaked around 3 MB/s while generating tokens and got the process OOM killed roughly every 2.5 hours.

The fix constructs the wrapper only when it is going to be stored; the lookup and the insert happen under MFastKernelCacheMutex, which saveKernel already holds. A unit test launches the same kernel alternately on two mock devices of one context and checks that the subcache keeps one entry per device and that tryToGetKernelFast hits for both.

How it was validated: on the shipped 2026.1.0 libsycl.so.9 first, by patching the equivalent branch in the compiled saveKernel (the je that skips the inlined destructor when the wrapper was moved-from, turned into a jmp). saveKernel calls during a 200 token generation went from about 4000 (breakpoint count under gdb, one per launch) to 0, RSS growth per 600 token generation went from +90..+280 MiB to within +-10 MiB, and single stream decode went from about 26 to about 31 tokens/s because the launches now hit the fast cache.

Then with this branch built from source (Release, GCC 13, buildbot/configure.py --disable-jit --disable-preview-lib), running KernelAndProgramTests-Non_Preview_Tests --gtest_filter='MultipleDevice*':

• with the fix: 4 tests pass, including the new MultipleDeviceFastCacheTest.EntriesKeptForAllDevices.
• with kernel_program_cache.hpp from the parent commit swapped back in: the new test fails with Subcache.Entries.size() being 1 instead of 2 (only the last device launched is left) and tryToGetKernelFast missing for device 0, which is the thrash described above. The three existing ProgramRetain cases still pass.
• with the fix restored: 4 tests pass again.

[Copilot]

🟢 Approval recommended

The fix is synchronized, narrowly scoped, and covered by a targeted regression test.

Pull request overview

Fixes fast kernel cache thrashing for kernels launched across multiple devices in one context.

Changes:

• Constructs subcache wrappers only when inserting new kernel names.
• Adds multi-device regression coverage for cache retention and hits.

File summaries

File	Description
sycl/source/detail/kernel_program_cache.hpp	Prevents unintended subcache cleanup on duplicate names.
sycl/unittests/kernel-and-program/MultipleDevsCache.cpp	Tests alternating launches across two devices.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.