Skip to content

feat(chat): add friendly name aliases for chat templates (fixes #976)#1460

Open
octo-patch wants to merge 1 commit into
guidance-ai:mainfrom
octo-patch:fix/chat-template-name-lookup
Open

feat(chat): add friendly name aliases for chat templates (fixes #976)#1460
octo-patch wants to merge 1 commit into
guidance-ai:mainfrom
octo-patch:fix/chat-template-name-lookup

Conversation

@octo-patch

Copy link
Copy Markdown

Summary

Closes #976.

Users currently cannot pass friendly names like "llama2" or "ChatML" to the chat_template parameter — they must supply the full Jinja2 template string. This PR implements the alias system that was explicitly called out in two TODO comments in chat.py:

TODO [HN]: Add an alias system so we can instantiate with other simple keys (e.g. "llama2" instead of the full template string)

Changes

  • guidance/chat.py — updates load_template_class() to check a new CHAT_TEMPLATE_NAME_MAP dictionary before falling through to the Jinja2 template string cache. Lookup is case-insensitive and strips spaces, dashes, underscores, and dots, so all of these work:

    model = Transformers("meta-llama/Llama-2-7b-chat-hf", chat_template="llama2")
    model = Transformers("meta-llama/Llama-2-7b-chat-hf", chat_template="Llama2")
    model = Transformers("meta-llama/Llama-2-7b-chat-hf", chat_template="llama-2")
    model = Transformers("meta-llama/Llama-3.2-1B-Instruct", chat_template="Llama 3.2")
    model = Transformers("Qwen/Qwen3-8B", chat_template="qwen3")
  • CHAT_TEMPLATE_NAME_MAP — a new module-level dict mapping normalized alias keys to template classes. Covers all templates currently defined in the module: ChatML, Llama 2/3/3.2, Phi-3 mini/small/medium, Phi-4 mini, Mistral 7B Instruct, Gemma 2 9B, Qwen 2.5, and Qwen 3.

  • tests/unit/test_chat.py (new) — unit tests covering all alias variants, case-insensitivity, separator stripping, None fallback, unknown-name warning, and map-key normalization invariant.

Backwards compatibility

Existing code that passes full Jinja2 template strings continues to work unchanged — the name map check only runs first; if no alias matches, the code falls through to CHAT_TEMPLATE_CACHE exactly as before.

…nce-ai#976)

Users can now pass short names like "llama2", "ChatML", "mistral", or
"qwen3" to any chat_template parameter instead of requiring the full
Jinja2 template string. Lookup is case-insensitive and strips spaces,
dashes, underscores, and dots (e.g. "Llama 3.2", "llama-3.2", and
"llama32" all resolve to Llama3dot2ChatTemplate).

Resolves the TODO at guidance/chat.py that asked for an alias system.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CHAT_TEMPLATE_CACHE keys are not what I expect

1 participant