[llmapi] NemotronV3ReasoningParser returns empty content when enable_thinking=False

[zentradev-rabih]

System Info

• TensorRT-LLM version: 1.3.0rc9
• Model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 (revision 4f0cf9d)
• Reasoning parser: --reasoning_parser nano-v3
• Tool parser: --tool_parser qwen3_coder
• Backend: pytorch
• GPU: NVIDIA RTX PRO 6000 Blackwell (96 GB, SM120)
• Inference cmd: trtllm-serve serve <model> --backend pytorch --reasoning_parser nano-v3 --tool_parser qwen3_coder --enable_chunked_prefill --trust_remote_code ...

Description

NemotronV3ReasoningParser (registered as nano-v3 in tensorrt_llm/llmapi/reasoning_parser.py) returns empty content when a request is sent with chat_template_kwargs: {"enable_thinking": false} AND the model leaks tokens into the reasoning stream OR fails to emit a closing </think> tag.

The parser inherits DeepSeekR1Parser's default behavior of routing tokens to reasoning_content until a </think> is seen. In thinking-disabled mode the model shouldn't be producing reasoning at all — but in practice it occasionally does (model behavior, not parser behavior). The current parser then returns:

• content: empty string
• reasoning_content: the actual answer

OpenAI-compatible chat clients render only content, so the user sees a blank response.

Why this is a bug, not a feature

NemotronV3ReasoningParser already has a force_nonempty_content mechanism that does exactly the right thing — but it requires the caller to opt in via chat_template_kwargs.force_nonempty_content=True. That's a fine power-user toggle, but it doesn't match the documented intent of enable_thinking=False.

For comparison, the HuggingFace model card for nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 ships a vLLM reasoning parser (super_v3_reasoning_parser.py, revision 4f0cf9d) that triggers the same swap when EITHER force_nonempty_content=True OR enable_thinking=False. So the TRT-LLM parser is out of sync with the model's documented inference behavior.

Reproduction (occurs ~5-15% of the time in our environment)

# Spin up trtllm-serve with Nemotron-3-Super-120B-NVFP4 + --reasoning_parser nano-v3
for i in $(seq 1 20); do
  curl -ksf https://localhost:8443/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"nemotron-super-120b-nvfp4",
         "messages":[{"role":"user","content":"What is 17 * 23?"}],
         "max_tokens":80,
         "chat_template_kwargs":{"enable_thinking":false}}' \
    | jq -r '.choices[0].message.content' &
done
wait
# Count empty responses

Without the fix, 1-3 of 20 responses come back with empty content while reasoning_content has the answer. With the proposed fix, 20/20 return content.

Proposed fix

Extend the swap-gate in NemotronV3ReasoningParser so it ALSO fires when chat_template_kwargs.enable_thinking is False. Same semantics as force_nonempty_content=True, just gated on an additional condition.

Four touch points, all inside the same class in tensorrt_llm/llmapi/reasoning_parser.py:

1. __init__: read enable_thinking flag, store as self._enable_thinking_is_false
2. _maybe_swap_content: extend gate to (self._force_nonempty_content or self._enable_thinking_is_false)
3. finish: same gate extension on the missing-closing-tag branch
4. parse_delta: same gate extension on the accumulator

Behavior when thinking is ENABLED — unchanged.
Behavior when force_nonempty_content=True — unchanged.
The new branch only fires when enable_thinking=False AND the parser would otherwise return empty content.

47-line unified diff (no incidental edits — strictly within NemotronV3ReasoningParser) ready to submit. Happy to open a PR with unit tests as soon as this issue is approved.

Worth considering during review

• Should the fix also extend to KimiK2ReasoningParser (also inherits DeepSeekR1Parser)? Kimi K2 doesn't expose an enable_thinking chat-template flag today, so unclear if it has the equivalent problem — happy to investigate as a follow-up.
• Alternative API: instead of adding a new instance var, the constructor could just default self._force_nonempty_content = True when enable_thinking is False. Less code, same effect, but force_nonempty_content then becomes overloaded with two meanings. Open to either direction.

Environment evidence

Patched our local install of tensorrt_llm/llmapi/reasoning_parser.py with the proposed fix on 2026-05-21. Smoke verified: PING/PONG and a 17×23 prompt both return non-empty content with enable_thinking=False. Behavior with enable_thinking=True unchanged (clean <think>...</think> framing, correct math answer). No regression in force_nonempty_content=True path.

[zentradev-rabih]

Since the fix is small, posting the unified diff inline for review. Applies cleanly against tensorrt_llm/llmapi/reasoning_parser.py at v1.3.0rc9. No incidental changes — all four hunks are inside NemotronV3ReasoningParser.

--- a/tensorrt_llm/llmapi/reasoning_parser.py
+++ b/tensorrt_llm/llmapi/reasoning_parser.py
@@ -288,11 +288,16 @@
                  reasoning_at_start: bool = True,
                  chat_template_kwargs: Optional[dict[str, Any]] = None) -> None:
         self._force_nonempty_content = False
+        # When the caller explicitly disabled thinking, treat unexpected
+        # reasoning_content as content (matches HF SuperV3ReasoningParser).
+        self._enable_thinking_is_false = False
         if isinstance(chat_template_kwargs, dict):
             reasoning_at_start = chat_template_kwargs.get(
                 "enable_thinking", reasoning_at_start)
             self._force_nonempty_content = chat_template_kwargs.get(
                 "force_nonempty_content", False) is True
+            self._enable_thinking_is_false = (
+                chat_template_kwargs.get("enable_thinking") is False
+            )
         super().__init__(reasoning_at_start=reasoning_at_start,
                          chat_template_kwargs=chat_template_kwargs)
         # Workaround: the model sometimes does not send closing think tags
@@ -306,7 +311,7 @@
             self, result: ReasoningParserResult) -> ReasoningParserResult:
         """When force_nonempty_content is set and content is empty, move
         reasoning_content into content so the response always has content."""
-        if self._force_nonempty_content and not result.content and result.reasoning_content:
+        if (self._force_nonempty_content or self._enable_thinking_is_false) and not result.content and result.reasoning_content:
             return ReasoningParserResult(content=result.reasoning_content,
                                          reasoning_content="")
         return result
@@ -318,7 +323,7 @@
         is cleared to free memory."""
         was_in_reasoning = self.in_reasoning
         result = super().parse_delta(delta_text)
-        if self._force_nonempty_content:
+        if self._force_nonempty_content or self._enable_thinking_is_false:
             if result.reasoning_content:
                 self._accumulated_reasoning += result.reasoning_content
             if was_in_reasoning and not self.in_reasoning:
@@ -340,7 +345,7 @@
         if self.in_reasoning and not self._found_closing_tag:
             remaining = self._buffer
             self._buffer = ""
-            if self._force_nonempty_content:
+            if self._force_nonempty_content or self._enable_thinking_is_false:
                 all_content = self._accumulated_reasoning + remaining
                 self._accumulated_reasoning = ""
                 self.in_reasoning = False

Verified in our environment:

• PING / PONG with enable_thinking=False → content populated
• 17×23 prompt with enable_thinking=False → streams real content (was sometimes empty pre-patch)
• 17×23 with enable_thinking=True → unchanged: clean <think>...</think> framing + correct math answer
• force_nonempty_content=True path unchanged (regression check)

Ready to open a PR with this diff plus matching unit tests as soon as a maintainer approves the approach.

[zentradev-rabih]

Quick rebase check — bug is still present on v1.3.0rc15 (released 2026-05-21).

Two related fixes shipped between rc9 and rc15:

• NVBug 6060281 — _maybe_swap_content now does not content.strip() so whitespace-only content (e.g. a trailing newline after </think>) doesn't block the swap. Good fix, doesn't apply here because the enable_thinking=False case can leave content truly empty (not whitespace).
• NVBug 6082303 — parse_delta treats <tool_call> as an implicit end-of-reasoning marker so the tool parser still sees the call when the model omits </think>. Different code path; doesn't help when there's no tool call.

Neither addresses the enable_thinking=False empty-content case.

The proposed patch rebases cleanly onto rc15 — same four touch points inside NemotronV3ReasoningParser, just adjusted to keep the .strip() from NVBug 6060281. Updated diff against rc15:

--- a/tensorrt_llm/llmapi/reasoning_parser.py
+++ b/tensorrt_llm/llmapi/reasoning_parser.py
@@ -309,11 +309,16 @@
                  reasoning_at_start: bool = True,
                  chat_template_kwargs: Optional[dict[str, Any]] = None) -> None:
         self._force_nonempty_content = False
+        # When the caller explicitly disabled thinking, treat unexpected
+        # reasoning_content as content (matches HF SuperV3ReasoningParser).
+        self._enable_thinking_is_false = False
         if isinstance(chat_template_kwargs, dict):
             reasoning_at_start = chat_template_kwargs.get(
                 "enable_thinking", reasoning_at_start)
             self._force_nonempty_content = chat_template_kwargs.get(
                 "force_nonempty_content", False) is True
+            self._enable_thinking_is_false = (
+                chat_template_kwargs.get("enable_thinking") is False
+            )
         super().__init__(reasoning_at_start=reasoning_at_start,
                          chat_template_kwargs=chat_template_kwargs)
         self._tool_call_start = "<tool_call>"
@@ -331,7 +336,7 @@
         Whitespace-only content (e.g. a newline after the closing think tag) is
         treated as empty so the swap still runs (NVBug 6060281)."""
         content = result.content or ""
-        if self._force_nonempty_content and not content.strip(
+        if (self._force_nonempty_content or self._enable_thinking_is_false) and not content.strip(
         ) and result.reasoning_content:
             return ReasoningParserResult(content=result.reasoning_content,
                                          reasoning_content="")
@@ -360,7 +365,7 @@
                 self._accumulated_reasoning = ""
             return ReasoningParserResult(content=content,
                                          reasoning_content=reasoning)
 
         was_in_reasoning = self.in_reasoning
         result = super().parse_delta(delta_text)
-        if self._force_nonempty_content:
+        if self._force_nonempty_content or self._enable_thinking_is_false:
             if result.reasoning_content:
                 self._accumulated_reasoning += result.reasoning_content
             if was_in_reasoning and not self.in_reasoning:
@@ -383,7 +388,7 @@
         if self.in_reasoning and not self._found_closing_tag:
             remaining = self._buffer
             self._buffer = ""
-            if self._force_nonempty_content:
+            if self._force_nonempty_content or self._enable_thinking_is_false:
                 all_content = self._accumulated_reasoning + remaining
                 self._accumulated_reasoning = ""
                 self.in_reasoning = False

Same semantics as the previous diff — just shifted to rc15 line numbers and aligned with the .strip() refinement from NVBug 6060281.

Happy to base the PR on main (or a release branch — whichever the reviewer prefers) once approved.