Skip to content

fix: add chat_template property to Model for setting chat templates#1455

Open
octo-patch wants to merge 1 commit into
guidance-ai:mainfrom
octo-patch:fix/issue-1196-chat-template-property
Open

fix: add chat_template property to Model for setting chat templates#1455
octo-patch wants to merge 1 commit into
guidance-ai:mainfrom
octo-patch:fix/issue-1196-chat-template-property

Conversation

@octo-patch

Copy link
Copy Markdown

Fixes #1196

Problem

The chat template smoke tests in tests/need_credentials/test_chat_templates.py were disabled with @pytest.mark.skip because the tests set lm.chat_template = ... on Mock model objects, but the Model base class had no chat_template property. As a result, setting lm.chat_template only set a plain attribute on the model instance, rather than updating the interpreter's chat template that is actually used for role-based formatting.

Solution

Add a chat_template property (getter and setter) to the Model base class in guidance/models/_base/_model.py:

  • Getter: returns self._interpreter.chat_template if the interpreter supports it (via getattr with None as default for interpreters that do not have this attribute)
  • Setter: sets self._interpreter.chat_template = value, which is then preserved across deepcopy when the model is copied during __add__ operations

This allows tests (and users) to configure the chat template on a model object directly:

lm = guidance.models.Mock("")
lm.chat_template = CHAT_TEMPLATE_CACHE[model_chat_template]()
# Now role blocks use the specified chat template
with guidance.user():
    lm += "Hello!"

Re-enable the two previously-skipped smoke tests: test_chat_format_smoke and test_chat_format_smoke_with_system.

Testing

The tests require HF_TOKEN and download actual HuggingFace models, so they run in the need_credentials suite. The fix is mechanically verified: lm.chat_template = value now properly sets lm._interpreter.chat_template = value, and subsequent deepcopy of the interpreter (triggered by model copy during __add__) preserves the new template.

…lates

Add getter/setter property for chat_template on the Model base class that
delegates to the underlying interpreter. This allows users to set the chat
template on model objects directly (e.g. Mock models in tests), enabling
the previously-disabled chat template smoke tests to run correctly.

Fixes guidance-ai#1196
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Chat Template tests disabled.

1 participant