Using a custom LLM model in autogen 0.5.5

[syclong]

We can use a custom LLM model in autogen 0.2. The method has been writen in this blog https://microsoft.github.io/autogen/0.2/blog/2024/01/26/Custom-Models.
Then, I want to know how to use custom models in autogen 0.5.5.
Thanks!

[SongChiYoung]

You want this?
https://medium.com/@songchiyoung/the-truth-about-autogen-model-support-dd00638a32ac

what kind of custom model do you want?

[syclong]

Thanks for your reply!
I use model form my company. And this model dose not surpport openai client. If I want to connect with this model, I only use the HTTP request with "verify=false".
So I want to know weather the autogen 0.5.5 have a method to use a custom model like the autogen 0.2.

[SongChiYoung]

Why not?
I think Ollama client and Replay client is simple to you learn how could to build custom client.
Ollama : https://github.com/microsoft/autogen/blob/main/python/packages/autogen-ext/src/autogen_ext/models/ollama/_ollama_client.py
Replay : https://github.com/microsoft/autogen/blob/main/python/packages/autogen-ext/src/autogen_ext/models/replay/_replay_chat_completion_client.py

Or you could using SemanticKernelClient to (If you could know about the SK well)
https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.models.semantic_kernel.html#autogen_ext.models.semantic_kernel.SKChatCompletionAdapter

[syclong]

Thank you!
I will modify the Replay client to use the custom LLM model in autogen 0.5.5.

[syclong]

And could you please give me an example code about using custom LLM model in autogen 0.5.5?
Thank you!

[SongChiYoung]

Hey, I actually tried to find an example for using a custom LLM model in AutoGen 0.5.5, but couldn’t find one either.

I was hoping to help, but it seems like there’s no direct example in the docs or repo.

Maybe starting from the Ollama or Replay clients could help — they’re simpler and show how to integrate a different model.

If I find something more specific, I’ll share it with you for sure!