Skip to content

geniex serve: max_tokens ignored on the qairt plugin — always generates to a fixed 2048-token cap #1403

Description

Describe the bug
geniex serve (OpenAI-compatible /v1/chat/completions endpoint) ignores the max_tokens request parameter entirely for models running on the qairt plugin. Regardless of the requested value, generation always continues to a fixed ceiling of exactly 2048 completion tokens, at which point the response is truncated with finish_reason:"length".

This is not a "sometimes exceeds the limit" issue — the requested value has zero effect on where generation actually stops. Tested with max_tokens:100 and max_tokens:900; both produced exactly 2048 completion tokens.

Confirmed on three different qairt-plugin models, including qualcomm/Qwen3-8B — a mainline, widely-used baseline model, not an edge case.

To Reproduce

  1. Start the server: geniex serve (default 127.0.0.1:18181)
  2. Pull and use any qairt-plugin model, e.g. geniex pull qualcomm/Qwen3-8B --model-hub aihub
  3. Send a chat completion with a prompt likely to produce a long natural response, and a small max_tokens:
curl -s -X POST http://127.0.0.1:18181/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qualcomm/Qwen3-8B",
    "messages": [{"role": "user", "content": "Write a long essay about the history of the printing press."}],
    "max_tokens": 100
  }'
  1. Observe: usage.completion_tokens is 2048, finish_reason is "length" — the max_tokens:100 request is silently ignored.

Confirmed reproductions (all identical: requested max_tokens ignored, output truncated at exactly 2048 tokens, finish_reason:"length")

Model Type max_tokens requested completion_tokens returned
qualcomm/Qwen3-8B llm 100 2048
qualcomm/Qwen3-VL-8B-Instruct vlm 100 2048
qualcomm/Qwen3-VL-8B-Instruct vlm 900 2048
qualcomm/Qwen3-VL-4B-Instruct vlm 900 2048

Expected behavior
max_tokens:100 should cap the response at ~100 tokens (give or take normal tokenizer-boundary slop), the same way it does on the llama_cpp plugin path, not silently continue to a fixed 2048.

Environment

  • OS: Windows 11 ARM64 (Snapdragon X2 Elite)
  • GenieX CLI: reproduced on both v0.3.19 and v0.5.0 (update did not fix it)
  • QAIRT Runtime: v2.45.0.260326

Additional context

  • 2048 is exactly half of the 4096-token context window (genie_config.json's context.size) on every model tested so far — unclear whether this is a coincidence or a hardcoded ctx/2 generation-buffer split in the qairt plugin. Not yet confirmed whether the cap scales with context size on a higher-context model.
  • This looks distinct from geniex serve: thinking/<think> reasoning always emitted in content — no flag or request param to disable or move it to reasoning_content #1294 (closed) — that issue is about the llama_cpp plugin leaking <think> reasoning into message.content, a different plugin path and a different symptom (content-field pollution, not a token-count ceiling). This bug is specific to the qairt plugin and affects the generation-length cap itself, not the content field.
  • Searched existing issues for max_tokens, qairt, and 2048 before filing — found no existing report of this exact behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions