Struggling with telemetry

[kollerp]

Am I not understanding something regarding how to use telemetry here? I cannot see any traces in the portal with this.

import os
from pathlib import Path
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from dotenv import load_dotenv
from openai import OpenAI
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from azure.monitor.opentelemetry import configure_azure_monitor
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
       
load_dotenv()

def main():    
    token_provider = get_bearer_token_provider(
        DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
    )        
    project = AIProjectClient(
        endpoint=os.getenv("AZURE_AI_FOUNDRY_PROJECT_ENDPOINT"),
        credential=DefaultAzureCredential()
    )        
    OpenAIInstrumentor().instrument()
    configure_azure_monitor(connection_string=project.telemetry.get_application_insights_connection_string())
    client = OpenAI(
        api_key=token_provider,
        base_url=os.getenv("AZURE_AI_FOUNDRY_BASE_URL")        
    )

    response = client.responses.create(   
        model="gpt-5", 
        input="This is a test.",
    )
    
    print(os.getenv("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT")) # true

[amynic]

Hi @kollerp - thanks for reaching out - I was comparing your code to this sample in docs and it might be worth stepping through to check you have all the right components https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/trace-application

Checking the Azure Application Insights resource is setup
Checking tracer statements are included etc

Let me know how you get on and we can dig in further if you are already using this resource

[kollerp]

I’m at step 7 of Instrument the OpenAI SDK in that doc. Only I’m using the new api.

[amynic]

Are you receiving any errors at all from the code running?
Are you able to see the data when tracing to the console for example?
Ca you be more specific on the 'new API' so we don't misunderstand each other :)

Also found another good sample here: https://github.com/sanjeevkumar761/azure-ai-foundry-observability-examples/blob/main/tracing-example-ai-agent-service.py

[kollerp]

No i dont get any errors. And the code sample implies that i should see traces without doing anything else than what I am doing.

By new API i mean this where i dont specify a version but always get the latest:

    client = OpenAI(
        api_key=token_provider,
        base_url=os.getenv("AZURE_AI_FOUNDRY_BASE_URL")        
    )

[kollerp]

Ok so I've been doing some more testing and here are my findings:

The open telemetry instrumentation you are using does not support the new responses api.

So doing client.responses.create will not work while client.chat.completions.create will. This is an issue with opentelemetry.instrumentation.openai_v2. This also means that anyone using the open ai agents sdk will not get telemetry since that uses the responses api under the hood.

Your documentation here is bugged

In the step-by-step you are missing imports and you do not specify that you need set the api version either in code or as an env variable.

[amynic]

Thanks for digging @kollerp into your use case - this is all good feedback let me pass on to relevant teams - specifically the docs feedback as well, really useful