Skip to content

REM: thinking models return empty generations through the ollama backend — document non-thinking model requirement #712

Description

@tps-flint

Finding (dogfood run, 2026-07-13)

Pointing REM's ollama backend at a thinking/reasoning model (e.g. qwen3-next) produces 502 distillation_failed on every execute-mode run: Ollama routes all output tokens into the response's thinking field (thinking defaults on for capable models), Harper's ollama backend reads only response, so models.generate() returns empty content and validation fails closed (correctly — zero rows staged).

Verified by direct probe with the exact request shape the backend sends: think: false → valid schema-conformant JSON; think absent/default → empty response, everything in thinking. Non-thinking models (qwen3-coder-next, llama3.1) work perfectly — the full stage → review → promote loop passed 14/14 checks against a live model.

Actions

  • docs/rem.md warning: pick a non-thinking model for the ollama backend until the upstream behavior changes (PR follows).
  • Upstream fix belongs in Harper's ollama backend (default think: false for non-streaming generate, or expose it on GenerateOpts, or error on empty-content-with-thinking). Upstream filing in progress.

REM's own posture held: fail-closed, zero partial candidates, distinct 502 with no leakage. The failure mode is availability (no candidates), not correctness.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions