Skip to content

feat(agents): Support for OpenAI Agents#508

Open
Murike wants to merge 10 commits intomainfrom
feature/parity-openai-agents
Open

feat(agents): Support for OpenAI Agents#508
Murike wants to merge 10 commits intomainfrom
feature/parity-openai-agents

Conversation

@Murike
Copy link
Contributor

@Murike Murike commented Feb 27, 2026

User description

(TS SDK Parity Effort) Missing - OpenAI Agent support - sc-34519

Description

  • Support for OpenAI Agents
    • Dedicated tracing processor
    • Node data structure for better support to Agents workflow
  • Tests and documentation

Generated description

Below is a concise technical summary of the changes proposed in this PR:

graph LR
registerGalileoTraceProcessor_("registerGalileoTraceProcessor"):::added
GalileoTracingProcessor_("GalileoTracingProcessor"):::added
OPENAI_AGENTS_CORE_("OPENAI_AGENTS_CORE"):::added
mapSpanType_("mapSpanType"):::added
mapSpanName_("mapSpanName"):::added
createNode_("createNode"):::added
extractLlmData_("extractLlmData"):::added
createGalileoCustomSpanData_("createGalileoCustomSpanData"):::added
registerGalileoTraceProcessor_ -- "Registers and returns a new tracing processor instance." --> GalileoTracingProcessor_
registerGalileoTraceProcessor_ -- "Calls addTraceProcessor from @openai/agents-core to register processor." --> OPENAI_AGENTS_CORE_
GalileoTracingProcessor_ -- "Determines node type including galileo_custom sentinel handling." --> mapSpanType_
GalileoTracingProcessor_ -- "Derives human-readable span names for logging." --> mapSpanName_
GalileoTracingProcessor_ -- "Creates node entries with empty children stored in _nodes." --> createNode_
GalileoTracingProcessor_ -- "Normalizes LLM span payloads, tokens, responses for logging." --> extractLlmData_
GalileoTracingProcessor_ -- "Wraps Galileo span for injection into agent tracing." --> createGalileoCustomSpanData_
classDef added stroke:#15AA7A
classDef removed stroke:#CD5270
classDef modified stroke:#EDAC4C
linkStyle default stroke:#CBD5E1,font-size:13px
Loading

Enable OpenAI Agents support by adding the GalileoTracingProcessor/GalileoCustomSpan suite plus node/span utilities so agent, LLM, tool, workflow, and Galileo custom spans are enriched, mapped, and emitted through GalileoLogger with status, metadata, embedded tool, and input/output handling. Harden the workflow by exporting the new processors, updating logger typings, and wiring dependency/test coverage so the agent tracing flow works when the optional @openai/agents peer dependencies are present.

TopicDetails
Agent Test & Meta Bundle dependency metadata and tests so the optional @openai/agents/openai peer deps are advertised while verifying the new tracing helpers across unit and integration scenarios.
Modified files (9)
  • package-lock.json
  • package.json
  • tests/handlers/openai-agents/custom-span.test.ts
  • tests/handlers/openai-agents/data-extraction.test.ts
  • tests/handlers/openai-agents/embedded-tool.test.ts
  • tests/handlers/openai-agents/integration.test.ts
  • tests/handlers/openai-agents/node.test.ts
  • tests/handlers/openai-agents/span-mapping.test.ts
  • tests/handlers/openai-agents/tracing-processor.test.ts
Latest Contributors(2)
UserCommitDate
Murikefeat-logging-Updated-e...February 27, 2026
vanessafsoares@hotmail...ci-coverage-Add-Codeco...February 03, 2026
Agent Tracing Flow Expand the agent tracing flow by adding GalileoTracingProcessor, node/span utilities, span mappers, Galileo custom span helpers, embedded tool extraction, and logger hooks so OpenAI Agent traces normalize inputs/outputs/statuses and flow through GalileoLogger with metadata and embedded tool awareness.
Modified files (9)
  • src/handlers/openai-agents/custom-span.ts
  • src/handlers/openai-agents/data-extraction.ts
  • src/handlers/openai-agents/embedded-tools.ts
  • src/handlers/openai-agents/index.ts
  • src/handlers/openai-agents/node.ts
  • src/handlers/openai-agents/span-mapping.ts
  • src/index.ts
  • src/types/logging/logger.types.ts
  • src/utils/galileo-logger.ts
Latest Contributors(0)
UserCommitDate
This pull request is reviewed by Baz. Review like a pro on (Baz).

@Murike Murike changed the title Feature/parity OpenAI agents feat(agents) : Support for OpenAI Agents Feb 27, 2026
@Murike Murike force-pushed the feature/parity-openai-agents branch from 820b376 to 411618a Compare February 27, 2026 18:52
@Murike Murike changed the title feat(agents) : Support for OpenAI Agents feat(agents): Support for OpenAI Agents Feb 27, 2026
@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

❌ Patch coverage is 95.69472% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.65%. Comparing base (2bbc9bb) to head (e261754).

Files with missing lines Patch % Lines
src/handlers/openai-agents/index.ts 93.83% 13 Missing ⚠️
src/handlers/openai-agents/embedded-tools.ts 94.04% 5 Missing ⚠️
src/handlers/openai-agents/data-extraction.ts 98.56% 2 Missing ⚠️
src/handlers/openai-agents/span-mapping.ts 96.77% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #508      +/-   ##
==========================================
+ Coverage   75.21%   76.65%   +1.44%     
==========================================
  Files          75       81       +6     
  Lines        6741     7252     +511     
  Branches     1860     2125     +265     
==========================================
+ Hits         5070     5559     +489     
- Misses       1660     1682      +22     
  Partials       11       11              
Flag Coverage Δ
unittests 76.65% <95.69%> (+1.44%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/handlers/openai-agents/custom-span.ts 100.00% <100.00%> (ø)
src/handlers/openai-agents/node.ts 100.00% <100.00%> (ø)
src/index.ts 100.00% <100.00%> (ø)
src/utils/galileo-logger.ts 76.91% <ø> (ø)
src/handlers/openai-agents/data-extraction.ts 98.56% <98.56%> (ø)
src/handlers/openai-agents/span-mapping.ts 96.77% <96.77%> (ø)
src/handlers/openai-agents/embedded-tools.ts 94.04% <94.04%> (ø)
src/handlers/openai-agents/index.ts 93.83% <93.83%> (ø)
Files with missing lines Coverage Δ
src/handlers/openai-agents/custom-span.ts 100.00% <100.00%> (ø)
src/handlers/openai-agents/node.ts 100.00% <100.00%> (ø)
src/index.ts 100.00% <100.00%> (ø)
src/utils/galileo-logger.ts 76.91% <ø> (ø)
src/handlers/openai-agents/data-extraction.ts 98.56% <98.56%> (ø)
src/handlers/openai-agents/span-mapping.ts 96.77% <96.77%> (ø)
src/handlers/openai-agents/embedded-tools.ts 94.04% <94.04%> (ø)
src/handlers/openai-agents/index.ts 93.83% <93.83%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

@fernandocorreia-galileo fernandocorreia-galileo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Murike please check if these auto-generated notes are valid.

Issues Found

  • [Major] Agent spans misclassified as workflow spans

    • File: src/handlers/openai-agents/span-mapping.ts (line 36-39) and src/handlers/openai-agents/index.ts (lines 370-379)
    • mapSpanType() maps OpenAI's 'agent' type to 'workflow', and _logNodeTree() catches all non-llm/non-tool nodes in the else branch calling addWorkflowSpan(). GalileoLogger has a distinct addAgentSpan() method (with agentType, stepNumber parameters) that should be used for agent spans to preserve the Trace→Agent hierarchy in the Galileo platform.
    • Suggestion: In mapSpanType(), map 'agent' to 'agent' instead of 'workflow'. In _logNodeTree(), add a dedicated branch for node.nodeType === 'agent' that calls this._galileoLogger.addAgentSpan(). Update the test at tracing-processor.test.ts:142 accordingly.
  • [Major] _firstInput is never populated

    • File: src/handlers/openai-agents/index.ts (lines 79, 304-305)
    • _firstInput is declared as null, reset to null in onTraceEnd, and checked in _logNodeTree — but never assigned a value anywhere. The trace-level input always falls back to the root node's input, which for agent traces is typically empty (since agent spans have input: '').
    • Suggestion: In onSpanEnd(), after extracting span data, add: if (this._firstInput === null && node.spanParams.input) { this._firstInput = node.spanParams.input; }. This captures the first non-empty input (typically from the first LLM or tool span).
  • [Major] parseUsage misses standard OpenAI token detail paths

    • File: src/handlers/openai-agents/data-extraction.ts (lines 44-53)
    • The function looks for reasoning/cached tokens under usage.details.reasoning_tokens and usage.details.cached_tokens. However, the standard OpenAI API (and Agents SDK) uses usage.output_tokens_details.reasoning_tokens and usage.input_tokens_details.cached_tokens. The current details fallback is not the correct path, so reasoning and cached token counts will be zero for all spans.
    • Suggestion: Add lookups for output_tokens_details.reasoning_tokens and input_tokens_details.cached_tokens as the primary paths, falling back to the existing details and top-level fields.
  • [Minor] GalileoCustomSpanData's galileoSpan reference is never used

    • File: src/handlers/openai-agents/index.ts (lines 158-159) and src/handlers/openai-agents/custom-span.ts
    • The createGalileoCustomSpanData() function and GalileoCustomSpanData interface provide API to embed a Galileo span reference, but onSpanStart() maps GALILEO_CUSTOM_TYPE to 'workflow' and discards the galileoSpan reference. The custom span becomes a generic workflow span with data from extractWorkflowData(). Either wire the galileoSpan into the logging path or simplify the API (remove galileoSpan if it's not needed yet).
    • Suggestion: If this is intended for future use, add a TODO comment. If it should work now, extract spanData.data.galileoSpan in onSpanStart and use it when logging the node.
  • [Minor] Orphan spans silently dropped

    • File: src/handlers/openai-agents/index.ts (lines 187-192)
    • If parentId points to a node not yet in _nodes, the new span is stored but never linked as a child. When _commitTrace walks the tree from root, orphaned spans are never visited and never logged. In normal SDK operation spans arrive in order, but out-of-order delivery (e.g., async agent flows) could cause silent data loss.
    • Suggestion: Add a fallback — if parentNode is not found, try linking to the trace root node. Add a sdkLogger.warn() when this happens.
  • [Nit] void _removed pattern for unused destructured variable

    • File: src/handlers/openai-agents/index.ts (lines 225-226)
    • const { _responseObject: _removed, ...rest } = finalData; void _removed; — the underscore-prefixed variable plus void is unusual. A cleaner pattern is just const { _responseObject: _, ...rest } = finalData; or delete the key from the object.
  • [Nit] Dynamic import warning fires eagerly at module load

    • File: src/handlers/openai-agents/index.ts (lines 25-29)
    • The top-level import('@openai/agents-core' as string).catch(...) runs as soon as the module is loaded, even if the consumer never intends to use GalileoTracingProcessor. This causes a spurious warning for users who import other Galileo features but don't use OpenAI Agents.
    • Suggestion: Move the availability check into the constructor or registerGalileoTraceProcessor() so it only fires when actually used.
  • [Nit] Trace metadata — consider defensive filtering

    • File: src/handlers/openai-agents/index.ts (lines 101-108)
    • While trace metadata is user-provided (not SDK-injected credentials), it would be a good defensive practice to at least warn or document that all metadata is forwarded verbatim to Galileo.

Questions

  • Is the GalileoCustomSpanData / galileoSpan reference intended to be functional in this PR, or is it scaffolding for a follow-up? The current API is exposed publicly but does nothing with the embedded span.
  • Should the agent type in mapSpanType map to 'agent' (using addAgentSpan()) or is the 'workflow' mapping intentional for some Galileo backend reason?
  • The _firstInput field — was there a plan to populate it that got lost during development, or is the fallback-to-root-input behavior intentional?

@Murike
Copy link
Contributor Author

Murike commented Mar 5, 2026

@Murike please check if these auto-generated notes are valid.

Issues Found

* **[Major] Agent spans misclassified as workflow spans**
  
  * File: `src/handlers/openai-agents/span-mapping.ts` (line 36-39) and `src/handlers/openai-agents/index.ts` (lines 370-379)
  * `mapSpanType()` maps OpenAI's `'agent'` type to `'workflow'`, and `_logNodeTree()` catches all non-llm/non-tool nodes in the `else` branch calling `addWorkflowSpan()`. GalileoLogger has a distinct `addAgentSpan()` method (with `agentType`, `stepNumber` parameters) that should be used for agent spans to preserve the Trace→Agent hierarchy in the Galileo platform.
  * Suggestion: In `mapSpanType()`, map `'agent'` to `'agent'` instead of `'workflow'`. In `_logNodeTree()`, add a dedicated branch for `node.nodeType === 'agent'` that calls `this._galileoLogger.addAgentSpan()`. Update the test at `tracing-processor.test.ts:142` accordingly.

* **[Major] `_firstInput` is never populated**
  
  * File: `src/handlers/openai-agents/index.ts` (lines 79, 304-305)
  * `_firstInput` is declared as `null`, reset to `null` in `onTraceEnd`, and checked in `_logNodeTree` — but never assigned a value anywhere. The trace-level input always falls back to the root node's input, which for agent traces is typically empty (since agent spans have `input: ''`).
  * Suggestion: In `onSpanEnd()`, after extracting span data, add: `if (this._firstInput === null && node.spanParams.input) { this._firstInput = node.spanParams.input; }`. This captures the first non-empty input (typically from the first LLM or tool span).

* **[Major] `parseUsage` misses standard OpenAI token detail paths**
  
  * File: `src/handlers/openai-agents/data-extraction.ts` (lines 44-53)
  * The function looks for reasoning/cached tokens under `usage.details.reasoning_tokens` and `usage.details.cached_tokens`. However, the standard OpenAI API (and Agents SDK) uses `usage.output_tokens_details.reasoning_tokens` and `usage.input_tokens_details.cached_tokens`. The current `details` fallback is not the correct path, so reasoning and cached token counts will be zero for all spans.
  * Suggestion: Add lookups for `output_tokens_details.reasoning_tokens` and `input_tokens_details.cached_tokens` as the primary paths, falling back to the existing `details` and top-level fields.

* **[Minor] GalileoCustomSpanData's `galileoSpan` reference is never used**
  
  * File: `src/handlers/openai-agents/index.ts` (lines 158-159) and `src/handlers/openai-agents/custom-span.ts`
  * The `createGalileoCustomSpanData()` function and `GalileoCustomSpanData` interface provide API to embed a Galileo span reference, but `onSpanStart()` maps `GALILEO_CUSTOM_TYPE` to `'workflow'` and discards the `galileoSpan` reference. The custom span becomes a generic workflow span with data from `extractWorkflowData()`. Either wire the `galileoSpan` into the logging path or simplify the API (remove `galileoSpan` if it's not needed yet).
  * Suggestion: If this is intended for future use, add a TODO comment. If it should work now, extract `spanData.data.galileoSpan` in `onSpanStart` and use it when logging the node.

* **[Minor] Orphan spans silently dropped**
  
  * File: `src/handlers/openai-agents/index.ts` (lines 187-192)
  * If `parentId` points to a node not yet in `_nodes`, the new span is stored but never linked as a child. When `_commitTrace` walks the tree from root, orphaned spans are never visited and never logged. In normal SDK operation spans arrive in order, but out-of-order delivery (e.g., async agent flows) could cause silent data loss.
  * Suggestion: Add a fallback — if `parentNode` is not found, try linking to the trace root node. Add a `sdkLogger.warn()` when this happens.

* **[Nit] `void _removed` pattern for unused destructured variable**
  
  * File: `src/handlers/openai-agents/index.ts` (lines 225-226)
  * `const { _responseObject: _removed, ...rest } = finalData; void _removed;` — the underscore-prefixed variable plus `void` is unusual. A cleaner pattern is just `const { _responseObject: _, ...rest } = finalData;` or delete the key from the object.

* **[Nit] Dynamic import warning fires eagerly at module load**
  
  * File: `src/handlers/openai-agents/index.ts` (lines 25-29)
  * The top-level `import('@openai/agents-core' as string).catch(...)` runs as soon as the module is loaded, even if the consumer never intends to use `GalileoTracingProcessor`. This causes a spurious warning for users who import other Galileo features but don't use OpenAI Agents.
  * Suggestion: Move the availability check into the constructor or `registerGalileoTraceProcessor()` so it only fires when actually used.

* **[Nit] Trace metadata — consider defensive filtering**
  
  * File: `src/handlers/openai-agents/index.ts` (lines 101-108)
  * While trace metadata is user-provided (not SDK-injected credentials), it would be a good defensive practice to at least warn or document that all metadata is forwarded verbatim to Galileo.

Questions

* Is the GalileoCustomSpanData / `galileoSpan` reference intended to be functional in this PR, or is it scaffolding for a follow-up? The current API is exposed publicly but does nothing with the embedded span.

* Should the `agent` type in `mapSpanType` map to `'agent'` (using `addAgentSpan()`) or is the `'workflow'` mapping intentional for some Galileo backend reason?

* The `_firstInput` field — was there a plan to populate it that got lost during development, or is the fallback-to-root-input behavior intentional?

FIXED - [Major] Agent spans misclassified as workflow spans

FIXED - [Major] _firstInput is never populated

FIXED - [Major] parseUsage misses standard OpenAI token detail paths

FIXED - [Minor] GalileoCustomSpanData's galileoSpan reference is never used

FIXED - [Minor] Orphan spans silently dropped

ANSWERED - [Nit] void _removed pattern for unused destructured variable
A: Suggestion doesn't compile (underscore still read as unused var), voiding is the less verbose solution here.

FIXED - [Nit] Dynamic import warning fires eagerly at module load

ANSWERED - [Nit] Trace metadata — consider defensive filtering
A: This is part of a bigger subject on how we are handling sensitive data on the SDK side. Won't be implemented here.

Questions

FIXED - Is the GalileoCustomSpanData / galileoSpan reference intended to be functional in this PR, or is it scaffolding for a follow-up? The current API is exposed publicly but does nothing with the embedded span.
FIXED - Should the agent type in mapSpanType map to 'agent' (using addAgentSpan()) or is the 'workflow' mapping intentional for some Galileo backend reason?
FIXED - The _firstInput field — was there a plan to populate it that got lost during development, or is the fallback-to-root-input behavior intentional?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants