Skip to content

smolagents AnyLLM client should be created only once and reused for completions #824

@HareeshBahuleyan

Description

@HareeshBahuleyan

Originally based on the comment:

We should try to use the API described in https://github.com/mozilla-ai/any-llm?tab=readme-ov-file#basic-usage where we can create an instance here and reuse it on each call

Originally posted by @daavoo in #817

When attempting to use a single client throws the following error:
Event loop is closed
Behaviour: first call to the LLM works alright, the error is thrown when making the second call.

It can be reproduced by applying the following git patch to the existing file:
https://github.com/mozilla-ai/any-agent/blob/main/src/any_agent/frameworks/smolagents.py

smolagent_reuse_anyllm_client.patch

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions