Skip to content

[BUG] Crashes when constrained sampling #487

@marek-hradil

Description

@marek-hradil

Investigate why this crash happens.

➜  git:(main) ✗ uv run python
Python 3.13.5 (main, Jun 12 2025, 12:22:43) [Clang 20.1.4 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import nobodywho
>>> sampler = nobodywho.SamplerPresets.grammar("root ::= [0-9]+")
>>> chat = nobodywho.Chat('../models/gemma-4.gguf', sampler=sampler)
>>> chat.ask("Tell me just a number, nothing else").completed()
'42'
>>> chat.ask("Tell me anything").completed()
Traceback (most recent call last):
  File "<python-input-9>", line 1, in <module>
    chat.ask("Tell me anything").completed()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
RuntimeError: Worker thread terminated before completing the response. This usually indicates an error occurred during token generation (e.g., context shift failure, sampling error, or token decoding issue).
>>> fatal runtime error: Rust cannot catch foreign exceptions, aborting

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions