Skip to content

Azure Inference Sdk does not work I have copy pasted the code from Chat Playground #40019

Open
@ManbirRakhra

Description

@ManbirRakhra
  • azure.ai.interence:
  • latest:
  • Windows 11:
  • 3.10.11:

Describe the bug
`# Install the following dependencies: azure.identity and azure-ai-inference
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

endpoint = os.getenv("AZURE_INFERENCE_SDK_ENDPOINT", "https://ai-manbirrdhubwu3151263134242.services.ai.azure.com/models")
model_name = os.getenv("DEPLOYMENT_NAME", "gpt-4o")
key = os.getenv("AZURE_INFERENCE_SDK_KEY", "xxx")
client = ChatCompletionsClient(endpoint=endpoint, credential=AzureKeyCredential(key))

response = client.complete(
messages=[
SystemMessage(content="You are a helpful assistant."),
UserMessage(content="What are 3 things to visit in Seattle?")
],
model = model_name,
max_tokens=1000
)

print(response)`

To Reproduce
Steps to reproduce the behavior:

  1. just run the code after installing all the imports

Expected behavior
This should work like Azure Open Ai Service. Azure AI Interfence crashes

Screenshots

Image

Image

Additional context
Add any other context about the problem here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

AI Model InferenceIssues related to the client library for Azure AI Model Inference (\sdk\ai\azure-ai-inference)Service AttentionWorkflow: This issue is responsible by Azure service team.customer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-author-feedbackWorkflow: More information is needed from author to address the issue.questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions