Skip to content

ai evaluations python sdk - https response messages still referencing "ai studio" as product name #41140

Open
@yogitasrivastava

Description

@yogitasrivastava
Image
  • script: |
    pip install openai pandas
    pip install azure.identity
    pip install azure-ai-evaluation # Install the correct Azure AI Evaluation library
    pip install azure-ai-projects # Install the missing dependency
    pip install azure-ai-inference # Install the required Azure OpenAI library
    displayName: "Install Dependencies"

Describe the bug
I was using azure ai evaluations sdk to evaluate model deployment and inference endpoint. Used below piece of code to publish results in ai foundry project. code ran successfully but the message in the logs confused customer/us because it gave aistudio link for results whereas customer was using ai foundry..

result = evaluate( evaluation_name="adversarial_model_evaluation", data=jsonl_path, # Replace with the actual file path target=None, # No need for a target since responses are already captured evaluators=evaluators, evaluator_config=evaluator_config, azure_ai_project=azure_ai_project

], "metrics": { "indirect_attack.xpia_manipulated_content": 0.03636363636363636, "indirect_attack.xpia_intrusion": 0.12727272727272726, "indirect_attack.xpia_information_gathering": 0.01818181818181818, "protected_material.protected_material_defect_rate": 0.0, "indirect_attack.xpia_defect_rate": 0.18 }, "studio_url": "[https://ai.azure.com/build/evaluation/b39f20ab-8846-44ca-8f9c-3dbb3ffde4a2?wsid=/subscriptions/e69c5bc8-9ce2-43da-ab1e-9b5851243390/resourceGroups/test_aifoundryrbac/providers/Microsoft.MachineLearningServices/workspaces/yoaifoundry-4251"](https://ai.azure.com/build/evaluation/b39f20ab-8846-44ca-8f9c-3dbb3ffde4a2?wsid=/subscriptions/e69c5bc8-9ce2-43da-ab1e-9b5851243390/resourceGroups/test_aifoundryrbac/providers/Microsoft.MachineLearningServices/workspaces/yoaifoundry-4251%22) }

To Reproduce
Steps to reproduce the behavior:
1.

Expected behavior
https response should not contain references to old product names it confuses customers

Screenshots

Image

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

AIClientThis issue points to a problem in the data-plane of the library.EvaluationIssues related to the client library for Azure AI EvaluationService AttentionWorkflow: This issue is responsible by Azure service team.customer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-team-attentionWorkflow: This issue needs attention from Azure service team or SDK teamquestionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions