Sequential embedding requests return invalid/zeroed vectors on RK3588 (NPU)

[advillalba]

When performing sequential requests to the /api/embed endpoint on an RK3588 NPU, only the first request returns a valid vector. Subsequent requests return vectors filled with zeros or consistent garbage values, even though the HTTP response code is 200 OK.

The issue persists until the model is explicitly unloaded from memory.

Steps to Reproduce:

Start the server with an embedding model (e.g., qwen3-embedding:0.6b-rk3588).

Send a POST request to /api/embed.

Result: Valid vector returned at index [0].

Immediately send a second POST request to /api/embed with different text.

Result: The returned vector at index [0] is invalid (contains zeros or static noise artifacts).

Observed Workaround: The issue is resolved if the model is forced to unload between requests. Since passing "keep_alive": 0 directly to the /api/embed endpoint appears to be ignored in this build.

Hardware: Rockchip RK3588
Endpoint: /api/embed
Model: qwen3-embedding (.rkllm)

My container is called ollama, but it's executing rkllama.

Execute again without unload:

[danielferr85]

Fixed.. need to clear cache after each request to delete previos input in last hidden layer....

Wait for pull request merge...

[iiiokojiadbi]

Reproduced on Orange Pi 5 Plus (RK3588), rkllama 0.0.66 / image ghcr.io/notpunchnox/rkllama:main (built 2026-04-14).

Root cause: PR_SET_PDEATHSIG inherited into each forked worker (added in #144) + Flask threaded=True — on Linux this binds the death signal to the request thread, not to the main process. As soon as the HTTP response completes and the thread exits, the kernel SIGTERMs the worker, and the inherited shutdown handler kills it.

Strace confirms a spurious SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=1} lands in the worker 2ms after the 200 OK response.

The keep_alive parameter handling in the embed flow is independent — even without this particular bug, keep_alive is accepted in embeddings_ollama() but never forwarded to EmbedEndpointHandler.handle_complete() (line 940 of server_utils.py). That's a separate small issue worth fixing.

Submitted a one-line fix in #145 — turns _set_parent_death_signal() into a no-op; orphan-worker protection is already handled by _kill_orphaned_workers() at startup.

A/B on a clean image built from the fix branch:

• Without the fix: req 2: 500 57.9s, req 4: 500 58.3s, 3 "died unexpectedly" in logs.
• With the fix: 10/10 sequential embeds return 200 in 115–180ms; 0 "died"/signal lines in logs.