Skip to content

adding tracing for list_run_steps #40166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
15 changes: 15 additions & 0 deletions sdk/ai/azure-ai-projects/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# Release History

## 1.0.0b8 (Unreleased)

### Features added

* Added support for tracing run steps when agents.list_run_steps is called. This can be used to trace service-side tool calls.

### Sample updates

### Bugs Fixed

* Fix for a bug in agent tracing causing event handler return values to not be returned when tracing is enabled.
* Fix for a bug in agent tracing causing tool calls not to be recorded in traces.

### Breaking Changes

## 1.0.0b7 (2025-03-06)

### Features added
Expand Down
39 changes: 38 additions & 1 deletion sdk/ai/azure-ai-projects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ for conn in conn_list:
print(conn_id)

# Initialize agent AI search tool and add the search index connection id
ai_search = AzureAISearchTool(index_connection_id=conn_id, index_name="myindexname")
ai_search = AzureAISearchTool(index_connection_id=conn_id, index_name="sample_index")

# Create agent with AI search tool and process assistant run
with project_client:
Expand All @@ -480,6 +480,34 @@ with project_client:

<!-- END SNIPPET -->

If the agent has found the relevant information in the index, the reference
and annotation will be provided in the message response. In the example above, we replace
the reference placeholder by the actual reference and url. Please note, that to
get sensible result, the index needs to have fields "title" and "url".

<!-- SNIPPET:sample_agents_azure_ai_search.populate_references_agent_with_azure_ai_search_tool -->

```python
# Fetch and log all messages
messages = project_client.agents.list_messages(thread_id=thread.id, order=ListSortOrder.ASCENDING)
for message in messages.data:
if message.role == MessageRole.AGENT and message.url_citation_annotations:
placeholder_annotations = {
annotation.text: f" [see {annotation.url_citation.title}] ({annotation.url_citation.url})"
for annotation in message.url_citation_annotations
}
for message_text in message.text_messages:
message_str = message_text.text.value
for k, v in placeholder_annotations.items():
message_str = message_str.replace(k, v)
print(f"{message.role}: {message_str}")
else:
for message_text in message.text_messages:
print(f"{message.role}: {message_text.text.value}")
```

<!-- END SNIPPET -->

#### Create Agent with Function Call

You can enhance your Agents by defining callback functions as function tools. These can be provided to `create_agent` via either the `toolset` parameter or the combination of `tools` and `tool_resources`. Here are the distinctions:
Expand Down Expand Up @@ -1305,6 +1333,15 @@ In addition, you might find helpful to see the tracing logs in console. You can
```python
project_client.telemetry.enable(destination=sys.stdout)
```

#### How to get traces for service-side tool calls

Additional traces related to available run steps, which include service-side tool calls, will be recorded when tracing is enabled and the run steps are retrieved in your code:
```python
steps = project_client.agents.list_run_steps(thread_id=thread.id, run_id=run.id)
```
You do not have to iterate over the steps in your code to get them traced. Content recording has to be enabled for the tool call details to be included in the traces.

#### How to trace your own functions

The decorator `trace_function` is provided for tracing your own function calls using OpenTelemetry. By default the function name is used as the name for the span. Alternatively you can provide the name for the span as a parameter to the decorator.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,7 @@ def from_dict(
:param function key_extractors: A key extractor function.
:param str content_type: JSON by default, set application/xml if XML.
:returns: An instance of this model
:raises: DeserializationError if something went wrong
:raises DeserializationError: if something went wrong
:rtype: Self
"""
deserializer = Deserializer(cls._infer_class_models())
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/azure/ai/projects/_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

VERSION = "1.0.0b7"
VERSION = "1.0.0b8"
Original file line number Diff line number Diff line change
Expand Up @@ -5144,8 +5144,8 @@ async def _list_connections(
"""List the details of all the connections (not including their credentials).

:keyword category: Category of the workspace connection. Known values are: "AzureOpenAI",
"Serverless", "AzureBlob", "AIServices", "CognitiveSearch", and "API Key". Default value is
None.
"Serverless", "AzureBlob", "AIServices", "CognitiveSearch", "ApiKey", "CustomKeys", and
"CognitiveService". Default value is None.
:paramtype category: str or ~azure.ai.projects.models.ConnectionType
:keyword include_all: Indicates whether to list datastores. Service default: do not list
datastores. Default value is None.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@


from ._models import ( # type: ignore
AOAIModelConfig,
Agent,
AgentDeletionStatus,
AgentThread,
Expand All @@ -34,6 +35,7 @@
Dataset,
Evaluation,
EvaluationSchedule,
EvaluationTarget,
EvaluatorConfiguration,
FileDeletionStatus,
FileListResponse,
Expand All @@ -48,6 +50,7 @@
IncompleteRunDetails,
IndexResource,
InputData,
MAASModelConfig,
MessageAttachment,
MessageContent,
MessageDelta,
Expand Down Expand Up @@ -145,6 +148,7 @@
SubmitToolOutputsAction,
SubmitToolOutputsDetails,
SystemData,
TargetModelConfig,
ThreadDeletionStatus,
ThreadMessage,
ThreadMessageOptions,
Expand Down Expand Up @@ -224,6 +228,7 @@
from ._patch import patch_sdk as _patch_sdk

__all__ = [
"AOAIModelConfig",
"Agent",
"AgentDeletionStatus",
"AgentThread",
Expand All @@ -244,6 +249,7 @@
"Dataset",
"Evaluation",
"EvaluationSchedule",
"EvaluationTarget",
"EvaluatorConfiguration",
"FileDeletionStatus",
"FileListResponse",
Expand All @@ -258,6 +264,7 @@
"IncompleteRunDetails",
"IndexResource",
"InputData",
"MAASModelConfig",
"MessageAttachment",
"MessageContent",
"MessageDelta",
Expand Down Expand Up @@ -355,6 +362,7 @@
"SubmitToolOutputsAction",
"SubmitToolOutputsDetails",
"SystemData",
"TargetModelConfig",
"ThreadDeletionStatus",
"ThreadMessage",
"ThreadMessageOptions",
Expand Down
6 changes: 6 additions & 0 deletions sdk/ai/azure-ai-projects/azure/ai/projects/models/_enums.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,8 @@ class AuthenticationType(str, Enum, metaclass=CaseInsensitiveEnumMeta):
"""Entra ID authentication (formerly known as AAD)"""
SAS = "SAS"
"""Shared Access Signature (SAS) authentication"""
CUSTOM = "CustomKeys"
"""Custom authentication"""
NONE = "None"
"""No authentication"""

Expand All @@ -153,6 +155,10 @@ class ConnectionType(str, Enum, metaclass=CaseInsensitiveEnumMeta):
"""Azure AI Search"""
API_KEY = "ApiKey"
"""Generic connection that uses API Key authentication"""
CUSTOM = "CustomKeys"
"""Generic connection that uses Custom authentication"""
COGNITIVE_SERVICE = "CognitiveService"
"""Cognitive Service"""


class DoneEvent(str, Enum, metaclass=CaseInsensitiveEnumMeta):
Expand Down
Loading