Skip to content

add more built-in tools #40697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

add more built-in tools #40697

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_converters/_ai_services.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@
_CODE_INTERPRETER = "code_interpreter"
_BING_GROUNDING = "bing_grounding"
_FILE_SEARCH = "file_search"
_AZURE_AI_SEARCH = "azure_ai_search"
_FABRIC_DATAAGENT = "fabric_dataagent"

# Built-in tool descriptions and parameters are hidden, but we include basic descriptions
# for evaluation purposes.
Expand All @@ -45,6 +47,8 @@
+ "up to 20 files.",
_BING_GROUNDING: "Enhance model output with web data.",
_FILE_SEARCH: "Search for data across uploaded files.",
_AZURE_AI_SEARCH: "Search an Azure AI Search index for relevant data.",
_FABRIC_DATAAGENT: "Connect to Microsoft Fabric data agents to retrieve data across different data sources."
}

# Built-in tool parameters are hidden, but we include basic parameters for evaluation purposes.
Expand All @@ -70,6 +74,14 @@
}
},
},
_AZURE_AI_SEARCH: {
"type": "object",
"properties": {"input": {"type": "string", "description": "Search terms to use."}},
},
_FABRIC_DATAAGENT: {
"type": "object",
"properties": {"input": {"type": "string", "description": "Search terms to use."}},
}
}

@experimental
Expand Down
8 changes: 8 additions & 0 deletions sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_converters/_models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,10 @@ def break_tool_call_into_messages(tool_call: ToolCall, run_id: str) -> List[Mess
arguments = {
"ranking_options": {"ranker": options["ranker"], "score_threshold": options["score_threshold"]}
}
elif tool_call.details["type"] == "azure_ai_search":
Copy link
Preview

Copilot AI Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an inconsistent access pattern when retrieving the tool type value. In this branch, the code uses subscript access (tool_call.details["type"]) while later branches use attribute access (tool_call.details.type). Standardizing the access method would enhance clarity and reduce the risk of runtime issues.

Copilot uses AI. Check for mistakes.

arguments = {"input": tool_call.details["azure_ai_search"]["input"]}
elif tool_call.details["type"] == "fabric_dataagent":
arguments = {"input": tool_call.details["fabric_dataagent"]["input"]}
else:
# unsupported tool type, skip
return messages
Expand Down Expand Up @@ -231,6 +235,10 @@ def break_tool_call_into_messages(tool_call: ToolCall, run_id: str) -> List[Mess
}
for result in tool_call.details.file_search.results
]
elif tool_call.details.type == "azure_ai_search":
output = tool_call.details.azure_ai_search["output"]
elif tool_call.details.type == "fabric_dataagent":
Comment on lines +238 to +240
Copy link
Preview

Copilot AI Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of attribute access for obtaining the tool type here contrasts with the subscript access used earlier. Consider unifying this pattern across all branches for consistency in handling tool details.

Suggested change
elif tool_call.details.type == "azure_ai_search":
output = tool_call.details.azure_ai_search["output"]
elif tool_call.details.type == "fabric_dataagent":
elif tool_call.details["type"] == "azure_ai_search":
output = tool_call.details.azure_ai_search["output"]
elif tool_call.details["type"] == "fabric_dataagent":

Copilot uses AI. Check for mistakes.

output = tool_call.details.fabric_dataagent["output"]
except:
return messages

Expand Down