You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug geniex serve (OpenAI-compatible /v1/chat/completions endpoint) ignores the max_tokens request parameter entirely for models running on the qairt plugin. Regardless of the requested value, generation always continues to a fixed ceiling of exactly 2048 completion tokens, at which point the response is truncated with finish_reason:"length".
This is not a "sometimes exceeds the limit" issue — the requested value has zero effect on where generation actually stops. Tested with max_tokens:100 and max_tokens:900; both produced exactly 2048 completion tokens.
Confirmed on three different qairt-plugin models, including qualcomm/Qwen3-8B — a mainline, widely-used baseline model, not an edge case.
To Reproduce
Start the server: geniex serve (default 127.0.0.1:18181)
Pull and use any qairt-plugin model, e.g. geniex pull qualcomm/Qwen3-8B --model-hub aihub
Send a chat completion with a prompt likely to produce a long natural response, and a small max_tokens:
curl -s -X POST http://127.0.0.1:18181/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{ "model": "qualcomm/Qwen3-8B", "messages": [{"role": "user", "content": "Write a long essay about the history of the printing press."}], "max_tokens": 100 }'
Observe: usage.completion_tokens is 2048, finish_reason is "length" — the max_tokens:100 request is silently ignored.
Expected behavior max_tokens:100 should cap the response at ~100 tokens (give or take normal tokenizer-boundary slop), the same way it does on the llama_cpp plugin path, not silently continue to a fixed 2048.
Environment
OS: Windows 11 ARM64 (Snapdragon X2 Elite)
GenieX CLI: reproduced on both v0.3.19 and v0.5.0 (update did not fix it)
QAIRT Runtime: v2.45.0.260326
Additional context
2048 is exactly half of the 4096-token context window (genie_config.json's context.size) on every model tested so far — unclear whether this is a coincidence or a hardcoded ctx/2 generation-buffer split in the qairt plugin. Not yet confirmed whether the cap scales with context size on a higher-context model.
Describe the bug
geniex serve(OpenAI-compatible/v1/chat/completionsendpoint) ignores themax_tokensrequest parameter entirely for models running on theqairtplugin. Regardless of the requested value, generation always continues to a fixed ceiling of exactly 2048 completion tokens, at which point the response is truncated withfinish_reason:"length".This is not a "sometimes exceeds the limit" issue — the requested value has zero effect on where generation actually stops. Tested with
max_tokens:100andmax_tokens:900; both produced exactly 2048 completion tokens.Confirmed on three different qairt-plugin models, including
qualcomm/Qwen3-8B— a mainline, widely-used baseline model, not an edge case.To Reproduce
geniex serve(default127.0.0.1:18181)geniex pull qualcomm/Qwen3-8B --model-hub aihubmax_tokens:usage.completion_tokensis 2048,finish_reasonis"length"— themax_tokens:100request is silently ignored.Confirmed reproductions (all identical: requested max_tokens ignored, output truncated at exactly 2048 tokens, finish_reason:"length")
qualcomm/Qwen3-8Bqualcomm/Qwen3-VL-8B-Instructqualcomm/Qwen3-VL-8B-Instructqualcomm/Qwen3-VL-4B-InstructExpected behavior
max_tokens:100should cap the response at ~100 tokens (give or take normal tokenizer-boundary slop), the same way it does on thellama_cppplugin path, not silently continue to a fixed 2048.Environment
Additional context
genie_config.json'scontext.size) on every model tested so far — unclear whether this is a coincidence or a hardcodedctx/2generation-buffer split in the qairt plugin. Not yet confirmed whether the cap scales with context size on a higher-context model.llama_cppplugin leaking<think>reasoning intomessage.content, a different plugin path and a different symptom (content-field pollution, not a token-count ceiling). This bug is specific to theqairtplugin and affects the generation-length cap itself, not the content field.max_tokens,qairt, and2048before filing — found no existing report of this exact behavior.