Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,7 @@ The easiest way to log traces is to use one of our direct integrations. Opik sup
| --------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| ADK | Log traces for Google Agent Development Kit (ADK) | [Documentation](https://www.comet.com/docs/opik/integrations/adk?utm_source=opik&utm_medium=github&utm_content=google_adk_link&utm_campaign=opik) |
| AG2 | Log traces for AG2 LLM calls | [Documentation](https://www.comet.com/docs/opik/integrations/ag2?utm_source=opik&utm_medium=github&utm_content=ag2_link&utm_campaign=opik) |
| Agent Spec | Log traces for Agent Spec calls | [Documentation](https://www.comet.com/docs/opik/integrations/agentspec?utm_source=opik&utm_medium=github&utm_content=agentspec_link&utm_campaign=opik) |
| AIsuite | Log traces for aisuite LLM calls | [Documentation](https://www.comet.com/docs/opik/integrations/aisuite?utm_source=opik&utm_medium=github&utm_content=aisuite_link&utm_campaign=opik) |
| Agno | Log traces for Agno agent orchestration framework calls | [Documentation](https://www.comet.com/docs/opik/integrations/agno?utm_source=opik&utm_medium=github&utm_content=agno_link&utm_campaign=opik) |
| Anthropic | Log traces for Anthropic LLM calls | [Documentation](https://www.comet.com/docs/opik/integrations/anthropic?utm_source=opik&utm_medium=github&utm_content=anthropic_link&utm_campaign=opik) |
Expand Down
198 changes: 198 additions & 0 deletions apps/opik-documentation/documentation/docs/cookbook/agentspec.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using Opik with Agent Spec\n",
"\n",
"[Agent Spec](https://oracle.github.io/agent-spec/development/agentspec/index.html) is a portable configuration language for defining agentic systems (agents, tools, and structured workflows).\n",
"\n",
"In this notebook, we will build a simple Agent Spec agent and use Opik's `AgentSpecInstrumentor` to capture a trace of the agent's tool and LLM execution."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creating an account on Comet.com\n",
"\n",
"[Comet](https://www.comet.com/site?from=llm&utm_source=opik&utm_medium=colab&utm_content=agentspec&utm_campaign=opik) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm&utm_source=opik&utm_medium=colab&utm_content=agentspec&utm_campaign=opik) and grab your API Key.\n",
"\n",
"> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/overview/?from=llm&utm_source=opik&utm_medium=colab&utm_content=agentspec&utm_campaign=opik) for more information."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "%pip install --upgrade opik \"pyagentspec[langgraph]\" opentelemetry-sdk opentelemetry-instrumentation"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import opik\n",
"\n",
"opik.configure(use_local=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preparing our environment\n",
"\n",
"This demo uses OpenAI as the LLM provider. Set your OpenAI API key as an environment variable:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import getpass\n",
"\n",
"if \"OPENAI_API_KEY\" not in os.environ:\n",
" os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define an Agent Spec agent\n",
"\n",
"We'll define a small calculator agent with a couple of tools:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pyagentspec.agent import Agent\n",
"from pyagentspec.llms import OpenAiConfig\n",
"from pyagentspec.property import FloatProperty\n",
"from pyagentspec.tools import ServerTool\n",
"\n",
"\n",
"def build_agentspec_agent() -> Agent:\n",
" tools = [\n",
" ServerTool(\n",
" name=\"sum\",\n",
" description=\"Sum two numbers\",\n",
" inputs=[FloatProperty(title=\"a\"), FloatProperty(title=\"b\")],\n",
" outputs=[FloatProperty(title=\"result\")],\n",
" ),\n",
" ServerTool(\n",
" name=\"subtract\",\n",
" description=\"Subtract two numbers\",\n",
" inputs=[FloatProperty(title=\"a\"), FloatProperty(title=\"b\")],\n",
" outputs=[FloatProperty(title=\"result\")],\n",
" ),\n",
" ]\n",
"\n",
" return Agent(\n",
" name=\"calculator_agent\",\n",
" description=\"An agent that provides assistance with tool use.\",\n",
" llm_config=OpenAiConfig(name=\"openai-gpt-5-mini\", model_id=\"gpt-5-mini\"),\n",
" system_prompt=(\n",
" \"You are a helpful calculator agent.\\n\"\n",
" \"Your duty is to compute the result of the given operation using tools, \"\n",
" \"and to output the result.\\n\"\n",
" \"It's important that you reply with the result only.\\n\"\n",
" ),\n",
" tools=tools,\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run the agent with Opik tracing enabled\n",
"\n",
"Wrap the agent execution in `AgentSpecInstrumentor().instrument_context(...)` to capture traces in Opik.\n",
"\n",
"> Agent traces can include prompts, tool inputs/outputs, and messages. If you need to avoid logging sensitive information, set `mask_sensitive_information=True`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from opik.integrations.agentspec import AgentSpecInstrumentor\n",
"from pyagentspec.adapters.langgraph import AgentSpecLoader\n",
"\n",
"agent = build_agentspec_agent()\n",
"\n",
"tool_registry = {\n",
" \"sum\": lambda a, b: a + b,\n",
" \"subtract\": lambda a, b: a - b,\n",
"}\n",
"\n",
"langgraph_agent = AgentSpecLoader(tool_registry=tool_registry).load_component(agent)\n",
"\n",
"with AgentSpecInstrumentor().instrument_context(\n",
" project_name=\"agentspec-demo\",\n",
" mask_sensitive_information=False,\n",
"):\n",
" messages = []\n",
"\n",
" messages.append({\"role\": \"user\", \"content\": \"Compute 13.5 + 2.25 using the sum tool.\"})\n",
" response = langgraph_agent.invoke(\n",
" input={\"messages\": messages},\n",
" config={\"configurable\": {\"thread_id\": \"1\"}},\n",
" )\n",
" agent_answer = response[\"messages\"][-1].content.strip()\n",
" print(\"AGENT >>>\", agent_answer)\n",
" messages.append({\"role\": \"assistant\", \"content\": agent_answer})\n",
"\n",
" messages.append({\"role\": \"user\", \"content\": \"Now compute 10 - 3.5 using the subtract tool.\"})\n",
" response = langgraph_agent.invoke(\n",
" input={\"messages\": messages},\n",
" config={\"configurable\": {\"thread_id\": \"1\"}},\n",
" )\n",
" agent_answer = response[\"messages\"][-1].content.strip()\n",
" print(\"AGENT >>>\", agent_answer)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After running the cell above, open Opik and navigate to the `agentspec-demo` project to inspect the trace tree and debug tool usage and LLM generations."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "py312_llm_eval",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
9 changes: 9 additions & 0 deletions apps/opik-documentation/documentation/fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,9 @@ navigation:
- page: AG2
path: docs/tracing/integrations/ag2.mdx
slug: ag2
- page: Agent Spec
path: docs/tracing/integrations/agentspec.mdx
slug: agentspec
- page: Agno
path: docs/tracing/integrations/agno.mdx
slug: agno
Expand Down Expand Up @@ -1152,6 +1155,9 @@ redirects:
- source: "/docs/opik/tracing/integrations/aisuite"
destination: "/docs/opik/integrations/aisuite"
permanent: true
- source: "/docs/opik/tracing/integrations/agentspec"
destination: "/docs/opik/integrations/agentspec"
permanent: true
- source: "/docs/opik/tracing/integrations/agno"
destination: "/docs/opik/integrations/agno"
permanent: true
Expand Down Expand Up @@ -1352,6 +1358,9 @@ redirects:
- source: "/docs/opik/cookbook/anthropic"
destination: "/docs/opik/integrations/anthropic"
permanent: true
- source: "/docs/opik/cookbook/agentspec"
destination: "/docs/opik/integrations/agentspec"
permanent: true
- source: "/docs/opik/cookbook/bedrock"
destination: "/docs/opik/integrations/bedrock"
permanent: true
Expand Down
2 changes: 1 addition & 1 deletion apps/opik-documentation/documentation/fern/docs/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ Gemini, Groq, Mistral AI, Novita AI, Ollama, OpenAI (Python & JS/TS), Predibase,
WatsonX, xAI Grok

**Frameworks:**
AG2, Agno, Autogen, CrewAI, DSPy, Haystack, Instructor, LangChain (Python & JS/TS), LangGraph,
AG2, Agent Spec, Agno, Autogen, CrewAI, DSPy, Haystack, Instructor, LangChain (Python & JS/TS), LangGraph,
LlamaIndex, Mastra, Pydantic AI, Semantic Kernel, Smolagents, Spring AI, Strands Agents, VoltAgent,
OpenAI Agents, Google Agent Development Kit, LiveKit Agents, BeeAI

Expand Down
Loading
Loading