[BUG] Structured output fails silently in Godot

[MaxKablaam]

Describe the bug
When I use set_sampler_preset_json() in my script to set the model to use structured output and run the program, it immediately quits once the scene is loaded, with no error message.

To Reproduce
Steps to reproduce the behavior:

1. Create a new scene
2. Use the simple chat template from the docs, put set_sampler_preset_json() right before the ask()
3. Run the scene
4. It immediately quits with no error

Expected behavior
The runtime should not quit, and the model should give structured output.

Environment:

• OS: MacOS
• Godot Version: 4.6.1
• Nobodywho Version: 8.1.0
• LLM Model: Tested with...
  • Phi-4-mini-instruct-Q4_K_M.gguf
  • Qwen_Qwen3-0.6B-Q4_K_M.gguf
  • Qwen_Qwen3-4B-Q4_K_M.gguf
  • gemma-3-4b-it-q4_0.gguf

Additional context
Here is my script:

extends NobodyWhoChat

func _ready():
    # self.response_updated.connect(_on_response_updated)
    self.response_finished.connect(_on_response_finished)

    start_worker()

    set_sampler_preset_json()
    ask("How are you?")

func _on_response_updated(token):
    # this will print every time a new token is generated
    print(token)

func _on_response_finished(response):
    # this will print when the entire response is finished
    print(response)

Here is the output:

2026-03-12T20:46:42.583325Z  INFO get_model: nobodywho::llm: Loading model use_gpu=true gpu_layers=4294967295 model_path="/Users/eric/Workspace/llm-test/models/gemma-3-4b-it-q4_0.gguf" use_gpu_if_available=true
2026-03-12T20:46:44.239808Z  INFO get_model:model_load: nobodywho::llm: Model loaded successfully model_path="/Users/eric/Workspace/llm-test/models/gemma-3-4b-it-q4_0.gguf" use_gpu_if_available=true path="/Users/eric/Workspace/llm-test/models/gemma-3-4b-it-q4_0.gguf"
2026-03-12T20:46:44.240963Z  INFO nobodywho::llm: Initializing worker
2026-03-12T20:46:44.474019Z  INFO nobodywho::chat: Worker processing: msg=SetSamplerConfig { sampler_config: SamplerConfig { steps: [TopK { top_k: 20 }, TopP { min_keep: 1, top_p: 0.95 }, Temperature { temperature: 0.6 }, Grammar { trigger_on: None, root: "root", grammar: "# this default gbnf grammar forces valid json output\nroot   ::= object\nvalue  ::= object | array | string | number | (\"true\" | \"false\" | \"null\") ws\n\nobject ::=\n\"{\" ws (\n            string \":\" ws value\n    (\",\" ws string \":\" ws value)*\n)? \"}\" ws\n\narray  ::=\n\"[\" ws (\n            value\n    (\",\" ws value)*\n)? \"]\" ws\n\nstring ::=\n\"\\\"\" (\n    [^\"\\\\\\x7F\\x00-\\x1F] |\n    \"\\\\\" ([\"\\\\bfnrt] | \"u\" [0-9a-fA-F]{4}) # escapes\n)* \"\\\"\" ws\n\nnumber ::= (\"-\"? ([0-9] | [1-9] [0-9]{0,15})) (\".\" [0-9]+)? ([eE] [-+]? [0-9] [1-9]{0,15})? ws\n\n# Optional space: by convention, applied in this grammar after literal chars when allowed\nws ::= | \" \" | \"\\n\" [ \\t]{0,20}" }], sample_step: Some(Dist), seed: 1234 } }
2026-03-12T20:46:44.476651Z  INFO nobodywho::chat: Worker processing: msg=Ask { text: "How are you?" }
2026-03-12T20:46:44.492472Z  INFO nobodywho::chat: Worker writing until done
--- Debugging process stopped ---

[bruno-pmakers]

This seems to be an issue with nobodywho godot version 8.1.0

I've tested with nobodywho godot version 8.0.3 and it works fine.

I have this log when the crash happens in v8.1.0.

2026-03-16T21:13:34.833047Z  INFO nobodywho::llm: Initializing worker
2026-03-16T21:13:38.604555Z  INFO nobodywho::chat: Worker processing: msg=SetSamplerConfig { sampler_config: SamplerConfig { steps: [TopK { top_k: 20 }, TopP { min_keep: 1, top_p: 0.95 }, Temperature { temperature: 0.6 }, Grammar { trigger_on: None, root: "root", grammar: "root ::= \"foo\" | \"bar\" | \"baz\"" }], sample_step: Some(Dist), seed: 1234 } }
2026-03-16T21:13:38.604802Z  INFO nobodywho::chat: Worker processing: msg=ResetChat { system_prompt: Some("You are a trash talker."), tools: "[0 tools]" }
2026-03-16T21:13:38.624461Z  INFO nobodywho::chat: Worker processing: msg=Ask { text: "Hello, how are you?" }
2026-03-16T21:13:41.102991Z  INFO nobodywho::chat: Worker writing until done
fatal runtime error: Rust cannot catch foreign exceptions, aborting

[BDB-David]

Confirming this is still present in 9.1.0 on Windows 10. Did a a test on Llama-3.2-1B-Instruct-Q4_K_M.gguf, Qwen3-0.6B-Q4_K_M.gguf, gemma-2-2b-it-Q4_K_M.gguf and they all silent crash on each.

Environment: Windows 10, NVIDIA RTX 3090, Godot 4.6.2.stable.official, NobodyWho 9.1.0.

Minimal repro (8 lines):

extends NobodyWhoChat

func _ready() -> void:
	response_finished.connect(func(r): print("REPLY: ", r))
	worker_started.connect(_after_ready)
	start_worker()

func _after_ready() -> void:
	set_sampler_preset_json()
	ask('Return {"hello": "world"}.')