AutoGen with Azure AI Foundry CodeLlama managed endpoint

[antonslutskyms]

Hello dear community! I'm trying to use AutoGen with a CodeLlama-70b-Instruct-hf model deployed via Azure AI Foundry managed endpoint and getting the following error message:
APIStatusError: Error code: 424 - {'detail': 'Not Found'}

Looking online, this seems to be related to an issue discussed in:
Azure/azure-sdk-for-python#39025

where adding /scope to the end of the base_url resulted in issues.

I tried to remove /scope from the base_url parameter of the LLM config block but getting the same error.
Tried both 0.2 and 0.4 versions of AugoGen -- same error in both.

My 0.2 version looks like this:
llm_config = { 'config_list': [{
'model': 'codellama',
'api_key': 'my api key',
'base_url': 'https://my endpoint name.my region.inference.ml.azure.com/score', # or "https://my endpoint name.my region.inference.ml.azure.com/" with same result
'api_type': 'azure',
}] }

assistant = AssistantAgent('assistant', llm_config=llm_config,
code_execution_config=False,
max_consecutive_auto_reply=30,
human_input_mode='NEVER'
)

0.4 version is:
az_model_client = OpenAIChatCompletionClient(
model= "codellama",
api_key="my key",
base_url="https://my endpoint name.my region.inference.ml.azure.com/score", # or without /score
model_capabilities={
"vision": False, # Replace with True if the model has vision capabilities.
"function_calling": False, # Replace with True if the model has function calling capabilities.
"json_output": False, # Replace with True if the model has JSON output capabilities.
},
max_tokens=128000
)

agent = AssistantAgent("assistant", az_model_client)

Would anyone have any suggestions?

TIA!

[ekzhu]

You can't use the OpenAIChatCompletionClient with Azure AI Inference endpoint at the moment -- they are not the same.

AI Foundry client will be available next release, follow #5153