Skip to content

Commit 094b94e

Browse files
joaomdmouraclaude
andauthored
fix(core): scope span export to our own tracer provider (#6954)
* fix(telemetry): stop exporting third-party spans to the collector set_tracer() installed CrewAI's TracerProvider as the global one, so every OTel-instrumented library in the host process - HTTP servers, Redis clients, ORMs - resolved trace.get_tracer() to our provider and exported to CrewAI's endpoint. A 20M-row sample of the telemetry table found 18,866 distinct operation names under our serviceName; CrewAI emits 21. The same wiring lost data in the other direction: when an application had already installed its own provider, our spans were created by theirs and went to their collector, so CrewAI received nothing from instrumented processes. Spans are now created from the private provider in both packages. Deletes _attach_common_attributes and its WeakSet/lock, whose multi-provider dedupe guarded a state that can no longer occur. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(telemetry): keep process context on crewai-core spans Isolating each package to its own TracerProvider removed an accident the CLI spans depended on: crewai_core.telemetry had no CommonAttributesSpanProcessor, so its spans only ever carried coding_agent/runtime_context/project_id by riding the global provider that crewai installed at import. A differential capture of every span reaching the exporter showed 8 of 54 spans losing those attributes - Feature Usage (cli_usage:*), Start Deployment, Template Installed, Create Crew Deployment, Get Crew Logs, Remove Crew, Deploy Signup Error and Flow Creation. Moves the marker tables and the detect_* helpers to crewai_core.runtime_env and the processor plus common_span_attributes() to crewai_core.telemetry, so both implementations share one source of truth. crewai.telemetry.utils and crewai.utilities.constants re-export the moved names, so their import paths are unchanged. Also fixes a gap that predates the isolation change: a CLI-only process never imports crewai, so it never reported either attribute. It does now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(core): type test helpers and stop tests reaching the collector mypy runs over lib/crewai-core/tests (the one test tree not excluded), so the new test file needed full annotations and a narrowed span.attributes. Also patches SafeOTLPSpanExporter before Telemetry is constructed and shuts the provider down afterwards: __init__ wires a BatchSpanProcessor around the real OTLP exporter, so each test was attempting a live export and leaving its batch worker thread running. Corrects the marker-precedence docstring, which named Cursor third when the table checks it last so that assistants running inside its terminal are not masked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * style(core): use one import form per module in telemetry tests Both test modules imported their telemetry module twice - once aliased for the monkeypatch target and once via from-import for the names. Dropping the alias in favour of monkeypatch's dotted-string target leaves a single import form and removes the need to qualify every reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * docs(core): drop comments that restate the code The TRACER_NAME constants were annotated with what their name and set_tracer()'s docstring already say, and the test fixtures narrated provider.shutdown() and the exporter patch at more length than either needed. Keeps the ones carrying something the code cannot: the resource-attribute ingestion quirk, why the marker tables moved packages, and the two ordering traps the fixtures exist to avoid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 17f107c commit 094b94e

17 files changed

Lines changed: 970 additions & 583 deletions

File tree

docs/edge/ar/telemetry.mdx

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,20 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true'
3434
os.environ['OTEL_SDK_DISABLED'] = 'true'
3535
```
3636

37+
### العزل عن إعداد OpenTelemetry الخاص بك
38+
39+
يعمل القياس عن بُعد الخاص بـ CrewAI على `TracerProvider` خاص به ولا يسجل نفسه
40+
أبدًا كمزوّد عام. هذا يفصل الاتجاهين:
41+
42+
- لا تُرسَل أبدًا إلى CrewAI الامتدادات (spans) الصادرة عن المكتبات الأخرى
43+
المزوّدة بأدوات القياس في عمليتك — أطر الويب، وعملاء قواعد البيانات،
44+
وعملاء HTTP.
45+
- لا تُرسَل امتدادات القياس عن بُعد الخاصة بـ CrewAI إلى نظام المراقبة لديك، لذا
46+
لن تظهر في Langfuse أو Braintrust أو Phoenix أو أي مُجمِّع آخر تقوم بإعداده.
47+
48+
لا تتأثر تكاملات المراقبة: فهي تقيس CrewAI عبر مزوّد التتبع الخاص بها، وهو
49+
مستقل عن المزوّد الموصوف هنا.
50+
3751
### شرح البيانات:
3852
| افتراضي | البيانات | السبب والتفاصيل |
3953
|:----------|:------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|

docs/edge/en/telemetry.mdx

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,20 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true'
3434
os.environ['OTEL_SDK_DISABLED'] = 'true'
3535
```
3636

37+
### Isolation from your own OpenTelemetry setup
38+
39+
CrewAI's telemetry runs on its own private `TracerProvider` and never registers
40+
itself as the global one. This keeps the two directions separate:
41+
42+
- Spans from other instrumented libraries in your process — web frameworks,
43+
database clients, HTTP clients — are never sent to CrewAI.
44+
- CrewAI's telemetry spans are never sent to your observability backend, so they
45+
will not appear in Langfuse, Braintrust, Phoenix, or any other collector you
46+
configure.
47+
48+
Observability integrations are unaffected: they instrument CrewAI through their
49+
own tracer provider, which is independent of the one described here.
50+
3751
### Data Explanation:
3852
| Defaulted | Data | Reason and Specifics |
3953
|:----------|:------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|

docs/edge/ko/telemetry.mdx

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,19 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true'
3333
os.environ['OTEL_SDK_DISABLED'] = 'true'
3434
```
3535

36+
### 사용자 OpenTelemetry 설정과의 격리
37+
38+
CrewAI의 telemetry는 자체 전용 `TracerProvider`에서 실행되며 자신을 전역
39+
provider로 등록하지 않습니다. 이를 통해 양방향이 분리됩니다:
40+
41+
- 프로세스 내 다른 계측된 라이브러리(웹 프레임워크, 데이터베이스 클라이언트,
42+
HTTP 클라이언트)의 span은 CrewAI로 전송되지 않습니다.
43+
- CrewAI의 telemetry span은 사용자의 관측 가능성 백엔드로 전송되지 않으므로
44+
Langfuse, Braintrust, Phoenix 또는 구성한 다른 수집기에 나타나지 않습니다.
45+
46+
관측 가능성 통합은 영향을 받지 않습니다: 해당 통합은 여기서 설명한 provider와
47+
독립적인 자체 tracer provider를 통해 CrewAI를 계측합니다.
48+
3649
### 데이터 설명:
3750
| 기본값 | 데이터 | 사유 및 세부 사항 |
3851
|:--------|:-------------------------------------------|:----------------------------------------------------------------------------------------------------------------------|

docs/edge/pt-BR/telemetry.mdx

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,20 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true'
3434
os.environ['OTEL_SDK_DISABLED'] = 'true'
3535
```
3636

37+
### Isolamento da sua própria configuração do OpenTelemetry
38+
39+
A telemetria do CrewAI roda em seu próprio `TracerProvider` privado e nunca se
40+
registra como o provider global. Isso mantém as duas direções separadas:
41+
42+
- Spans de outras bibliotecas instrumentadas no seu processo — frameworks web,
43+
clientes de banco de dados, clientes HTTP — nunca são enviados ao CrewAI.
44+
- Os spans de telemetria do CrewAI nunca são enviados ao seu backend de
45+
observabilidade, portanto não aparecerão no Langfuse, Braintrust, Phoenix ou
46+
em qualquer outro coletor que você configurar.
47+
48+
As integrações de observabilidade não são afetadas: elas instrumentam o CrewAI
49+
por meio do próprio tracer provider, que é independente do descrito aqui.
50+
3751
### Explicação dos Dados:
3852
| Padrão | Dados | Razão e Especificidades |
3953
|--------|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
"""Detection of the coding assistant and runtime a process is running under.
2+
3+
Lives in ``crewai_core`` rather than ``crewai`` because both telemetry
4+
implementations need it: the CLI emits deployment, template and flow-creation
5+
spans through ``crewai_core.telemetry`` without importing ``crewai`` at all.
6+
7+
``crewai.utilities.constants`` and ``crewai.telemetry.utils`` re-export from
8+
here, so the original import paths keep working.
9+
"""
10+
11+
from __future__ import annotations
12+
13+
import os
14+
import sys
15+
from typing import Final
16+
17+
18+
CC_ENV_VAR: Final[str] = "CLAUDECODE"
19+
CODEX_ENV_VARS: Final[tuple[str, ...]] = (
20+
"CODEX_CI",
21+
"CODEX_MANAGED_BY_NPM",
22+
"CODEX_SANDBOX",
23+
"CODEX_SANDBOX_NETWORK_DISABLED",
24+
"CODEX_THREAD_ID",
25+
)
26+
CC_ENV_VARS: Final[tuple[str, ...]] = (CC_ENV_VAR, "CLAUDE_CODE")
27+
CURSOR_ENV_VARS: Final[tuple[str, ...]] = (
28+
"CURSOR_AGENT",
29+
"CURSOR_EXTENSION_HOST_ROLE",
30+
"CURSOR_SANDBOX",
31+
"CURSOR_TRACE_ID",
32+
"CURSOR_WORKSPACE_LABEL",
33+
)
34+
ANTIGRAVITY_ENV_VARS: Final[tuple[str, ...]] = (
35+
"ANTIGRAVITY_AGENT",
36+
"ANTIGRAVITY_CLI_ALIAS",
37+
)
38+
AUGMENT_ENV_VARS: Final[tuple[str, ...]] = ("AUGMENT_AGENT",)
39+
CLINE_ENV_VARS: Final[tuple[str, ...]] = ("CLINE_ACTIVE",)
40+
GEMINI_CLI_ENV_VARS: Final[tuple[str, ...]] = ("GEMINI_CLI",)
41+
JUNIE_ENV_VARS: Final[tuple[str, ...]] = ("JUNIE_DATA", "JUNIE_SHIM_PATH")
42+
OPENCODE_ENV_VARS: Final[tuple[str, ...]] = ("OPENCODE", "OPENCODE_CLIENT")
43+
44+
# Proposed cross-vendor marker (agentsmd/agents.md#136). Checked last and
45+
# reported as "other": it says an assistant is present without naming one, and
46+
# reading its value to find out would put an arbitrary string in telemetry.
47+
GENERIC_AGENT_ENV_VARS: Final[tuple[str, ...]] = ("AI_AGENT",)
48+
49+
# Ordered (name, env vars) pairs for identifying the AI coding assistant a
50+
# process is running under. Reuses the sets above and keeps the same precedence
51+
# as ``get_env_context()``, so the env-context events and telemetry never
52+
# disagree about which assistant is present.
53+
#
54+
# Deliberately limited to assistants whose markers are verified. Guessing a
55+
# variable name is worse than omitting the assistant: a wrong name never
56+
# matches, so that assistant is silently counted as "unknown" while the table
57+
# implies it is covered.
58+
#
59+
# Two rules for adding an entry:
60+
# 1. Confirm the variable the tool actually sets - do not infer it from the
61+
# product name.
62+
# 2. Use only *session*-scoped variables the assistant sets for processes it
63+
# spawns. Persistent user configuration (an ``AIDER_MODEL`` in a committed
64+
# ``.env``, say) is unusable: crewai loads dotenv files on normal runs, so
65+
# a leftover config value would mislabel ordinary human executions.
66+
#
67+
# Extend the shared sets above rather than adding a parallel tuple here, so both
68+
# detection paths pick the new markers up together: ``get_env_context()`` walks
69+
# this same table for its precedence and emits ``DefaultEnvEvent`` for the
70+
# assistants that have no event class of their own.
71+
#
72+
# Markers below the first three were taken from the published detection matrix
73+
# at vercel/detect-agent (agents.json), cross-checked against the proposal in
74+
# agentsmd/agents.md#136 and microsoft/vscode#311734. Rule 2 excluded several
75+
# entries those sources list: Goose's ``GOOSE_PROVIDER`` and Copilot's
76+
# ``COPILOT_MODEL`` / ``COPILOT_GITHUB_TOKEN`` are user configuration, and a
77+
# committed ``.env`` carrying one would relabel every ordinary run. Replit's
78+
# ``REPL_ID`` is a hosted environment rather than an assistant, so it stays in
79+
# HOSTED_IDE_ENV_VARS.
80+
#
81+
# The assistants that spawn inside another editor's terminal are ordered ahead
82+
# of Cursor for the same reason Codex is: CURSOR_* is set for every integrated
83+
# terminal, so checking Cursor first would mask anything running inside it.
84+
CODING_AGENT_ENV_MARKERS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = (
85+
("claude_code", CC_ENV_VARS),
86+
("codex", CODEX_ENV_VARS),
87+
("cline", CLINE_ENV_VARS),
88+
("gemini_cli", GEMINI_CLI_ENV_VARS),
89+
("augment", AUGMENT_ENV_VARS),
90+
("opencode", OPENCODE_ENV_VARS),
91+
("antigravity", ANTIGRAVITY_ENV_VARS),
92+
("junie", JUNIE_ENV_VARS),
93+
("cursor", CURSOR_ENV_VARS),
94+
)
95+
96+
# Markers for *where* a process runs, kept separate from which assistant is
97+
# driving it. The two answer different questions: a scheduled container run has
98+
# no assistant to detect, and folding it into the assistant field made "no
99+
# marker found" and "no assistant possible" indistinguishable.
100+
#
101+
# These are published platform contracts rather than per-tool observations, so
102+
# they do not need the case-by-case verification the assistant table requires.
103+
# Presence is all that is checked; no value is ever read.
104+
CI_ENV_VARS: Final[tuple[str, ...]] = (
105+
"APPVEYOR",
106+
"BITBUCKET_BUILD_NUMBER",
107+
"BUILDKITE",
108+
"CI",
109+
"CIRCLECI",
110+
"DRONE",
111+
"GITHUB_ACTIONS",
112+
"GITLAB_CI",
113+
"JENKINS_URL",
114+
"TEAMCITY_VERSION",
115+
"TF_BUILD",
116+
"TRAVIS",
117+
)
118+
SERVERLESS_ENV_VARS: Final[tuple[str, ...]] = (
119+
"AWS_LAMBDA_FUNCTION_NAME",
120+
"FUNCTIONS_EXTENSION_VERSION",
121+
"FUNCTIONS_WORKER_RUNTIME",
122+
"FUNCTION_TARGET",
123+
"K_SERVICE",
124+
"VERCEL",
125+
)
126+
# Managed application platforms, kept apart from serverless: their markers are
127+
# set for long-lived containers rather than per-invocation functions, and
128+
# checking them under "serverless" would have claimed every Heroku dyno and
129+
# Azure App Service instance before the container check could see them.
130+
#
131+
# Azure Functions run on the App Service host and inherit WEBSITE_INSTANCE_ID,
132+
# so they would land here despite being serverless. The FUNCTIONS_* markers
133+
# above are checked first to keep them out.
134+
PAAS_ENV_VARS: Final[tuple[str, ...]] = (
135+
"DYNO",
136+
"WEBSITE_INSTANCE_ID",
137+
)
138+
HOSTED_IDE_ENV_VARS: Final[tuple[str, ...]] = (
139+
"CODESPACES",
140+
"GITPOD_WORKSPACE_ID",
141+
"REPL_ID",
142+
)
143+
NOTEBOOK_ENV_VARS: Final[tuple[str, ...]] = (
144+
"COLAB_RELEASE_TAG",
145+
"JPY_PARENT_PID",
146+
)
147+
CONTAINER_ENV_VARS: Final[tuple[str, ...]] = ("KUBERNETES_SERVICE_HOST",)
148+
149+
# Ordered most specific first. CI jobs and hosted IDEs usually run inside
150+
# containers, so a bare container match is only meaningful once the others have
151+
# been ruled out.
152+
RUNTIME_CONTEXT_ENV_MARKERS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = (
153+
("ci", CI_ENV_VARS),
154+
("serverless", SERVERLESS_ENV_VARS),
155+
("paas", PAAS_ENV_VARS),
156+
("hosted_ide", HOSTED_IDE_ENV_VARS),
157+
("notebook", NOTEBOOK_ENV_VARS),
158+
("container", CONTAINER_ENV_VARS),
159+
)
160+
161+
162+
# Editors whose integrated terminal a process can be launched from. Matched on
163+
# an exact value rather than presence, since these variables name the terminal.
164+
_EDITOR_TERM_MARKERS: Final[tuple[tuple[str, str, str], ...]] = (
165+
("TERM_PROGRAM", "vscode", "vscode_terminal"),
166+
("TERMINAL_EMULATOR", "JetBrains-JediTerm", "jetbrains_terminal"),
167+
)
168+
169+
# Marks a process running inside a container when no more specific runtime
170+
# marker is present.
171+
_DOCKER_ENV_PATH: Final[str] = "/.dockerenv"
172+
173+
_UNKNOWN: Final[str] = "unknown"
174+
_OTHER: Final[str] = "other"
175+
176+
# The complete set of values detect_coding_agent() can ever return. Every value
177+
# is a literal from CODING_AGENT_ENV_MARKERS or this module, which is what makes
178+
# the function structurally incapable of emitting PII: no environment value,
179+
# path, hostname, or user-supplied string can reach the return value.
180+
KNOWN_CODING_AGENTS: Final[frozenset[str]] = frozenset(
181+
[name for name, _ in CODING_AGENT_ENV_MARKERS] + [_OTHER, _UNKNOWN]
182+
)
183+
184+
# The same guarantee for detect_runtime_context(): a closed set of literals.
185+
KNOWN_RUNTIME_CONTEXTS: Final[frozenset[str]] = frozenset(
186+
[name for name, _ in RUNTIME_CONTEXT_ENV_MARKERS]
187+
+ [name for _, _, name in _EDITOR_TERM_MARKERS]
188+
+ ["interactive", "non_interactive", _UNKNOWN]
189+
)
190+
191+
192+
def detect_coding_agent() -> str:
193+
"""Best-effort detection of the AI coding assistant running this process.
194+
195+
Uses the shared ``CODING_AGENT_ENV_MARKERS`` table, so this agrees with the
196+
env-context events emitted by ``get_env_context()`` rather than maintaining
197+
a second, narrower set of markers. Precedence follows that table, which ends
198+
with Cursor: ``CURSOR_*`` is set for every integrated terminal, so checking
199+
it earlier would mask any assistant running inside one.
200+
201+
Only the assistant's normalized name is returned - environment variable
202+
values are never read into the return value or recorded anywhere.
203+
204+
Two limits worth knowing. This is heuristic: markers change as tools
205+
evolve, so "unknown" means "no known marker present", not "no agent". And
206+
some markers (the Cursor set in particular) are set by the editor for any
207+
integrated terminal, so a result names the environment the process is
208+
running *under*, not proof that an agent authored the code.
209+
210+
Answers only *which assistant*. Where the process runs is
211+
:func:`detect_runtime_context`, so an "unknown" here is a genuine gap in
212+
the marker table rather than a run that never had an assistant to find.
213+
214+
Returns:
215+
A normalized assistant name (e.g. "claude_code", "cursor", "codex"), or
216+
"unknown" when no marker matches. The result is always a member of
217+
KNOWN_CODING_AGENTS.
218+
"""
219+
for agent_name, env_vars in CODING_AGENT_ENV_MARKERS:
220+
if any(os.environ.get(env_var) for env_var in env_vars):
221+
return agent_name
222+
223+
# Checked last and by presence: the cross-vendor marker establishes that an
224+
# assistant is present without naming one, and an empty value still says so.
225+
if any(env_var in os.environ for env_var in GENERIC_AGENT_ENV_VARS):
226+
return _OTHER
227+
228+
return _UNKNOWN
229+
230+
231+
def detect_runtime_context() -> str:
232+
"""Best-effort detection of where this process is running.
233+
234+
Separate from :func:`detect_coding_agent` because the two answer different
235+
questions. Automated runs -- CI, containers, serverless -- have no
236+
assistant to detect, and reporting them in the assistant field made an
237+
unrecognized assistant indistinguishable from a run that could never have
238+
had one.
239+
240+
Precedence runs most specific first: CI and hosted IDEs typically run
241+
inside containers, so a bare container match only applies once the more
242+
specific markers have been ruled out.
243+
244+
Returns:
245+
One of the runtime names (e.g. "ci", "serverless", "container",
246+
"notebook", "hosted_ide"), an editor terminal (e.g. "vscode_terminal"),
247+
"interactive" or "non_interactive" when only a TTY check applies, or
248+
"unknown" when that check cannot be made. The result is always a member
249+
of KNOWN_RUNTIME_CONTEXTS.
250+
"""
251+
# Presence, not truthiness: a platform that exports an empty CI= is still
252+
# CI, unlike the assistant markers where an empty value means the tool set
253+
# a placeholder rather than claiming the session.
254+
for context_name, env_vars in RUNTIME_CONTEXT_ENV_MARKERS:
255+
if any(env_var in os.environ for env_var in env_vars):
256+
return context_name
257+
258+
for env_var, expected, context_name in _EDITOR_TERM_MARKERS:
259+
if os.environ.get(env_var) == expected:
260+
return context_name
261+
262+
if os.path.exists(_DOCKER_ENV_PATH):
263+
return "container"
264+
265+
try:
266+
return "interactive" if sys.stdout.isatty() else "non_interactive"
267+
except (AttributeError, ValueError, OSError):
268+
return _UNKNOWN

0 commit comments

Comments
 (0)