Skip to content

fix: update adding_new_models tutorial to use current API#1450

Open
octo-patch wants to merge 1 commit into
guidance-ai:mainfrom
octo-patch:fix/update-adding-new-models-notebook-v2
Open

fix: update adding_new_models tutorial to use current API#1450
octo-patch wants to merge 1 commit into
guidance-ai:mainfrom
octo-patch:fix/update-adding-new-models-notebook-v2

Conversation

@octo-patch

Copy link
Copy Markdown

Fixes #1402

Problem

The adding_new_models.ipynb tutorial uses the outdated API where custom model support was implemented by subclassing TransformersChat. This API no longer exists in v0.3.x.

The old approach:

class OrcaChat(models.transformers.TransformersChat):
    def get_role_start(self, role_name, **kwargs): ...
    def get_role_end(self, role_name=None): ...

orca = OrcaChat('pankajmathur/orca_mini_3b', ...)

Solution

Update the notebook to use the current ChatTemplate API:

  • Replace OrcaChat(TransformersChat) with OrcaChatTemplate(ChatTemplate)
  • Use models.Transformers(..., chat_template=OrcaChatTemplate) to load with a custom template
  • Import UnsupportedRoleException for proper error handling on unsupported roles
  • Remove stale cell outputs and update explanations to reflect the new architecture

Testing

The updated code is self-contained and follows the same pattern used in the existing guidance documentation and codebase (e.g., guidance/chat.py).

Update the adding_new_models tutorial to use the current API:
- Replace OrcaChat (TransformersChat subclass) with OrcaChatTemplate (ChatTemplate subclass)
- Use models.Transformers with chat_template parameter instead of subclassing
- Remove stale outputs and update explanations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No example work with 0.3.0

1 participant