Releases: DataDog/dd-trace-py
4.8.8
Bug Fixes
- Fixed an issue that could have caused some timers, like the one responsible for Symbol Database uploads, to fire repeatedly after the first execution.
- internal: This fix resolves a memory leak where reference cycles through
PeriodicThreadcallbacks were invisible to Python's cyclic garbage collector and could accumulate when threads used bound methods as targets.
- profiling: Fixes a memory leak in native frame tracking caused by unbounded native call-site metadata growth.
- SCA: This fix resolves an issue where unresolved runtime reachability targets could accumulate across Software Composition Analysis updates, causing resident memory usage to grow over time.
- sqlalchemy: Fixes duplicate SQLAlchemy event listeners when
trace_engine()is called repeatedly for the same engine.
- dynamic instrumentation: fixes an issue where the Symbol Database uploader sends empty payloads on a recurring timer.
4.10.2
Bug Fixes
- LLM Observability: This fix resolves an issue in the Claude Agent SDK integration where a span's error message showed an uncategorized
unknownerror category from the upstream Claude Agent SDK instead of a descriptive API error. The integration now surfaces the detailed error message from the assistant message content.
- tracing: Fixes a race condition where extra service names could be silently dropped from Remote Configuration
/v0.7/configpayloads in multi-threaded applications (e.g. uWSGI).
- code origin: fixed an issue that could have caused pytest to crash internally when inspecting the call stack from an exception thrown by a view function when Code Origin is enabled.
- LLM Observability: Resolves an issue where non-string tag values passed to
LLMObs.annotate(tags=...)could cause spans to be dropped during ingestion.
- LLM Observability: Fixes provider mis-attribution on
openaispans when anOpenAI(orAsyncOpenAI) client and anAzureOpenAI(orAsyncAzureOpenAI) client are instantiated at the same time. Provider is now determined per-call rather than from the most recently constructed client.
4.11.0rc1
Upgrade Notes
- flask: Requests served under a non-empty WSGI
SCRIPT_NAMEnow expose the client-hit resource on a newflask.resource.fulltag (e.g.GET /api/v2/users). The spanresourceandflask.url_ruletag are unchanged. The tag is only set when its value would differ fromspan.resource.
- flask: API endpoint discovery now reports every HTTP method the framework serves — including Werkzeug's auto-added
HEADfor anyGETroute and Flask's auto-handledOPTIONSfor every route — not just the methods listed inmethods=[...].
New Features
- tracing: collect the
x-datadog-endpoint-scanandx-datadog-security-testHTTP request headers on service entry spans unconditionally ashttp.request.headers.x-datadog-endpoint-scanandhttp.request.headers.x-datadog-security-testtags. These markers identify Datadog-originated endpoint scans and security tests so the API inventory pipeline can distinguish scan/test traffic from real user traffic. The headers are tagged regardless ofDD_TRACE_HEADER_TAGSconfiguration or AppSec enablement, and are not propagated to downstream services.
- Database Monitoring (DBM) propagation supports
dynamic_serviceas a newDD_DBM_PROPAGATION_MODEvalue. SetDD_DBM_PROPAGATION_MODE=dynamic_serviceto inject DBM service metadata and the SQL base hash without injecting trace context.
- ai_guard: add AI Guard evaluation support to the Anthropic SDK Messages instrumentation. Both non-streaming and streaming requests and non-streaming responses are evaluated through the configured AI Guard client (covering
Messages.create/Messages.streamand their async and Beta variants), and evaluation is automatically skipped when a framework integration (LangChain) is already evaluating the same call.
- aiokafka: Adds
kafka.partitionandkafka.message_offsettags to the producer span once the broker acknowledges the send. The partition reflects the partition the broker actually assigned to the message, which may differ from the partition the caller requested. Mirrors the behavior added to the Java tracer in DataDog/dd-trace-java#11107.
- ASM: This introduces the
_dd.appsec.normalized_routespan tag for FastAPI and Starlette request spans when API Security is enabled. The tag follows RFC-1103 and provides a per-request, framework-agnostic representation of the matched route — converter types are stripped, multi-parameter URL segments are combined with+,pathcatch-all parameters are emitted as a single tail element, and trailing slashes are preserved as declared. Mount-prefixed sub-application routes are reported with their full assembled path.
- ASM: Extends
_dd.appsec.normalized_routespan tag support to Flask request spans when API Security is enabled. Flask / Werkzeug<converter:name>route syntax is normalized following RFC-1103: converter types are stripped, multi-parameter URL segments (e.g./<first>.<last>/) are combined with+,<path:name>catch-all parameters are emitted as a single tail element, and trailing slashes are preserved as declared. Routes served throughDispatcherMiddlewaresub-apps are reported with their full assembled path (mount prefix included).
- LLM Observability: The
claude_agent_sdkintegration now emits span links between step, LLM, and tool spans so multi-step traces render the sequencing between LLM calls and tool calls.
- aws_durable_execution_sdk_python: Adds distributed tracing for durable workflows across suspend/resume cycles, so a workflow that pauses and resumes in a later invocation appears as a single connected trace. See
aws_durable_execution_sdk_pythonfor details and opt-out configuration.
- AAP: API endpoint discovery now covers Flask sub-applications mounted via
werkzeug.middleware.dispatcher.DispatcherMiddleware. Sub-app routes are reported with their full mounted path.
- google_cloud_pubsub: This introduces Data Streams Monitoring (DSM) context propagation for the Google Cloud Pub/Sub integration. Producer publish operations inject the DSM pathway context into message attributes, and subscriber callbacks extract it and record a consume checkpoint. To enable, set
DD_DATA_STREAMS_ENABLED=true.
- LLM Observability: automatically tags spans and experiments with
git.commit.shaandgit.repository_url. Values come fromDD_GIT_COMMIT_SHA/DD_GIT_REPOSITORY_URLor the main package'sProject-URLmetadata, falling back to runninggitagainst the current working directory. HonorsDD_TRACE_GIT_METADATA_ENABLEDand user-supplied tags with the same keys.
- LLM Observability: experiment evaluators and summary evaluators can now return a
MultiEvaluatorResultto emit multiple named evaluation metrics from a single evaluator call. Each sub-value may itself be anEvaluatorResultcarrying its own reasoning, assessment, metadata, and tags. By default emitted metric labels are prefixed with the evaluator's name ("<evaluator_name>-<key>"); passprefix=Falseto emit raw keys.
- LLM Observability: Add
LLMObs.pull_experiment(experiment_id)to fetch a previously-run experiment from the Datadog backend by UUID. The returnedSyncExperimenthas its.resultpopulated and is ready for downstream inspection (e.g..as_dataframe()) without re-executing the original task.
- LLM Observability:
taskanddatasetare now optional when creating an experiment viaLLMObs.experiment()/LLMObs.async_experiment(), supporting pull-based workflows. Callingrun()without providingtaskanddatasetraises aValueError.
- LLM Observability: When a tool definition with a
versionfield is set on a parent LLM span viatool_definitions, the resolved tool version is now also written onto manually-started child tool spans (created viaLLMObs.tool()) asmeta.tool.version. Previously, this propagation only occurred for tool spans produced by auto-instrumented integrations (OpenAI, LangChain, OpenAI Agents, ReAct) through theLinkTrackerdispatch mechanism.
- LLM Observability: When a tool definition with a
versionfield is set on a parent LLM span viatool_definitions, the resolved tool version is now also written onto the corresponding child tool span asmeta.tool.version. This allows the LLM Observability UI to aggregate tool version counts and filter tool spans by version without correlating to the parent LLM span.
- profiling: The fast stack profiler optimisation is now enabled by default.
- Profiling: The heap profiler can now track allocations made through
PYMEM_DOMAIN_MEM(PyMem_Malloc/Calloc/Realloc) in addition to the existingPYMEM_DOMAIN_OBJcoverage whenDD_PROFILING_MEMORY_MEM_DOMAIN_ENABLED=trueis set. This support is effective on Python 3.12 and newer, making some previously invisible allocations visible in heap profiles in those environments.
- ray: Added
DD_ML_JOB_ENV— a single opt-in env var for forwarding configuration into Ray job workers. Set it on the dashboard process as a semicolon-separated list ofKEY:VALUEpairs, e.g.DD_SERVICE:my-svc;DD_AGENT_HOST:10.0.0.1, and workers receive them verbatim asDD_SERVICE=my-svc,DD_AGENT_HOST=10.0.0.1, etc.
-
CI Visibility: Adds
ddtrace.testing.logs.DDTestLogsHandler, a publiclogging.Handlerfor shipping log records to the Datadog logs intake correlated with CI Visibility test traces, intended for use in subprocesses that execute tests outside of the main pytest process. Also addsddtrace.testing.logs.CorrelationFilter, a base class for stampingdd.trace_id/dd.span_idonto log records, and a ready-madeddtrace.testing.logs.ThreadLocalCorrelationFilterfor the common thread-local case. SubclassCorrelationFilterto plug in other concurrency models (asyncioContextVar, global state, etc.).Example:
from pydantic_evals.evaluators import EqualsExpected import logging
from ddtrace.testing.logs import DDTestLogsHandler from ddtrace.testing.logs import ThreadLocalCorrelationFilter
handler = DDTestLogsHandler(service="my-service") correlation = ThreadLocalCorrelationFilter() handler.addFilter(correlation) logging.getLogger().addHandler(handler)
while True:
job = queue.get() correlation.set_context(trace_id=job.trace_id, span_id=job.span_id) run_test(job.item)handler.close()
Bug Fixes
- AAP: This fix resolves an issue where the AppSec body-parsing hook consumed the
websocket.connectASGI message, causing ASGI/FastAPI WebSocket connections to fail with HTTP 500 when AppSec was enabled.
- aiokafka: Collects the Kafka cluster ID when tracing aiokafka producers and consumers and forwards it to Data Streams Monitoring, matching the behavior of the
confluent_kafkaintegration.
- aap: Adds instrumentation telemetry distributions for WAF and RASP execution durations, including
waf.duration,waf.duration_ext,rasp.duration, andrasp.duration_ext.
- azure_cosmos: This fix resolves an issue where
cosmosdb.queryspan resource names included per-item identifiers (such as document, user, and permission ids) from the request URI, which caused unbounded resource cardinality. The integration now redacts those ids by replacing them with?while keeping the database (dbs) and collection (colls) names intact. For example,Read /dbs/myDb/colls/myColl/docs/item1is now reported asRead /dbs/myDb/colls/myColl/docs/?.
- CI Visibility: fix the default HTTP timeout for backend requests from 15 seconds to 30 seconds, and add the
DD_CIVISIBILITY_BACKEND_API_TIMEOUT_MILLISenvironment variable (previously missing) to override it. The value is expressed in m...
4.10.1
Bug Fixes
- internal: Fixed an issue that could have caused some timers, like the one responsible for Symbol Database uploads, to fire repeatedly after the first execution.
- internal: This fix resolves a memory leak where reference cycles through
PeriodicThreadcallbacks were invisible to Python's cyclic garbage collector and could accumulate when threads used bound methods as targets.
- profiling: Fixes a memory leak in native frame tracking caused by unbounded native call-site metadata growth.
- SCA: This fix resolves an issue where unresolved runtime reachability targets could accumulate across Software Composition Analysis updates, causing resident memory usage to grow over time.
4.10.0
Bug Fixes
- AAP: This fix resolves an issue where the AppSec body-parsing hook consumed the
websocket.connectASGI message, causing ASGI/FastAPI WebSocket connections to fail with HTTP 500 when AppSec was enabled.
- LLM Observability: Fixes an issue where
reasoning_contentwas missing from streamed chat completions in the OpenAI and LiteLLM integrations when an OpenAI-compatible reasoning provider (e.g. DeepSeek, Qwen) emitteddelta.reasoning_contentchunks. The aggregated message now captures reasoning text in the output message, matching non-streaming behavior.
- dynamic instrumentation: fixes an issue where the Symbol Database uploader sends empty payloads on a recurring timer.
Other Changes
- LLM Observability: when LLMObs is enabled in agentless mode (Datadog Agent not reachable or with
DD_LLMOBS_AGENTLESS_ENABLED=1), APM traces are now exported agentlessly to Datadog's intake. This should not change user-facing behavior: both APM and LLMObs spans remain visible in the UI; LLMObs spans are simply no longer shipped separately for agentless users. Note that settingDD_APM_TRACING_ENABLED=falsetakes higher precedence and will result in LLMObs span events shipping separately as existing behavior.
4.10.0rc1
Other Changes
- LLM Observability: when LLMObs is enabled in agentless mode (Datadog Agent not reachable or with
DD_LLMOBS_AGENTLESS_ENABLED=1), APM traces are now exported agentlessly to Datadog's intake. This should not change user-facing behavior: both APM and LLMObs spans remain visible in the UI; LLMObs spans are simply no longer shipped separately for agentless users. Note that settingDD_APM_TRACING_ENABLED=falsetakes higher precedence and will result in LLMObs span events shipping separately as existing behavior.
4.9.0
Upgrade Notes
- AI Guard:
ddtrace.appsec.ai_guard.AIGuardAbortErrornow derives fromddtrace.internal._exceptions.DDBlockException(aBaseExceptionsubclass) instead ofException. This brings AI Guard block decisions in line with how ASM blocks are surfaced and prevents a genericexcept Exception:in user code from silently swallowing a block.
- settings: Legacy environment variable names registered as aliases in the configuration registry now also work when set via local or fleet stable config files, not just shell environment variables. #17958
- ray: Adds
DD_TRACE_RAY_IGNORED_ACTORSconfiguration to exclude specific Ray actor methods from instrumentation. SetDD_TRACE_RAY_IGNORED_ACTORS='{"ActorA": ["method1"], "ActorB": "*"}'to leave matching methods or actors uninstrumented while continuing to trace other Ray actor methods. Matching is based on actor class name only.
Deprecation Notes
- Tracing:
DD_TRACE_INFERRED_SPANS_ENABLEDis deprecated and will be removed in 5.0.0. UseDD_TRACE_INFERRED_PROXY_SERVICES_ENABLEDinstead. The old environment variable continues to work but emits aDDTraceDeprecationWarningwhen set.
New Features
- aws_durable_execution_sdk_python: Add tracing support for the
aws-durable-execution-sdk-pythonlibrary. Instruments@durable_executionworkflows andDurableContextoperations (step,invoke,wait,wait_for_condition,wait_for_callback,create_callback,map,parallel,run_in_child_context) to generate spans.
- LLM Observability: Adds
stepspans to the Claude Agent SDK integration. Each inference cycle is now represented by astepcontainer span with anllmchild span for the model call andtoolchild spans for any tool invocations.
- tracing: Adds a centralized
supported-configurations.jsonregistry of all supportedDD_*andOTEL_*environment variables, following the same schema used by other Datadog tracing libraries. Accesses to unregistered environment variables now produce a debug log to help identify typos or unsupported configuration options.
- AI Guard: Copies anomaly-detection attributes from the local root (service-entry) span onto every
ai_guardspan:ai_guard.http.useragent,ai_guard.http.client_ip,ai_guard.network.client.ip,ai_guard.usr.idandai_guard.usr.session_id.
- AI Guard: When
DD_AI_GUARD_ENABLED=trueis set and anai_guardspan is created during a request, the tracer now populateshttp.client_ipandnetwork.client.ipon the service-entry (local root) span, mirroring the behavior used for Application Security. If AI Guard does not run during the request, no client IP tags are added.DD_TRACE_CLIENT_IP_ENABLEDis ignored once AI Guard reports, andDD_TRACE_CLIENT_IP_HEADERcontinues to override header resolution.
- ai_guard: add AI Guard evaluation support to the OpenAI SDK chat completions instrumentation. Both non-streaming and streaming requests and non-streaming responses are evaluated through the configured AI Guard client, and evaluation is automatically skipped when a framework integration (LangChain, Strands Agents) is already evaluating the same call.
- code origin for spans: The code origin for spans feature has been enabled by default.
- code origin: attach code origin information to the first span generated by a function wrapped with tracer.wrap.
- openfeature: This introduces a configurable initialization timeout for
DataDogProvider. The timeout controls how longinitialize()waits for configuration before returning, and defaults to 10 seconds. Set it via theDD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MSenvironment variable or theinit_timeoutconstructor parameter.
- CI Visibility: This introduces Jenkins custom parent ID propagation, which enables Datadog to correlate tests run from Jenkins with their Jenkins jobs and pipelines.
- LLM Observability: Adds an optional
cost_tagsargument toLLMObs.annotate()andLLMObs.annotation_context(). Pass a list of tag keys (already set viatagsor annotated previously on the same span) to have them attached to the cost and token metrics generated from LLM and embedding spans, which can help breaking down spend by team, project, org, or any custom dimension.
- LLM Observability: Adds support for an optional
version(string) field on each tool definition dictionary passed toLLMObs.annotate()via thetool_definitionsparameter.
- profiling: Add
DD_PROFILING_LOCK_EXCLUDE_MODULESconfig to skip lock profiling for framework-internal locks. Excluded locks remain native with zero profiling overhead. Set it to a comma-separated list of module prefixes (e.g.,django.db,sqlalchemy.pool,urllib3).
- LLM Observability: Bedrock Agent orchestration step events (model invocations, tool/action group calls, knowledge base lookups, guardrails, rationales) are now emitted as APM child spans of the
Bedrock Agent <agent_id>span when LLM Observability is enabled, with the same LLMObs payload shape as before.
Bug Fixes
-
tracing: Exclude
wrapt==2.2.0from the supported dependency range to avoid a regression that breaks wrapped C descriptors. -
ai_guard: This fix resolves a conflict between
ddtrace.autoandstrandswhen imported in the same file, which left Strands hooks silently disabled. The Strands integration now loads lazily on first attribute access so its event class identities match those the agent dispatches.
- appsec: Adds telemetry metrics
instrum.user_auth.missing_user_loginandinstrum.user_auth.missing_user_idwhen Django auth events cannot resolve the expected identity fields, enabling detection of misconfigured user model field mappings.
- AAP: This fix resolves an issue where the
usr.session_idtag was missing from the entry span of authenticated follow-up Django requests when automatic user instrumentation was enabled. They now also carryusr.session_id, matching other authenticated user-tagging paths.
- azure_cosmos: This change removes the http.status_code tags from Azure CosmosDB spans and replaces them with the use of the db.response.status_code metric. For customers using ddtrace v4.8.0 and relying on the http.status_code tag of cosmosdb.query spans, this is a breaking change.
- CI Visibility: Fixes an issue in the pytest plugin where a malformed log call emitted a
--- Logging error ---traceback to stderr during Attempt to Fix retries, polluting pytest output and contributing to spurious test failures.
- CI Visibility: Fixes an
IndexErrorin retry bookkeeping that occurred when a test's teardown phase failed. The error produced--- Logging error ---tracebacks in stderr, which could pollute test output and cause spurious test failures during retries. #17863
- CI Visibility: Fixes a regression where setting
DD_TEST_MANAGEMENT_ENABLED=0was not honored by the new pytest plugin, causing Test Management features such as quarantining, disabling tests, and Attempt to Fix to remain enabled.
- CI Visibility: Fixes code coverage instrumentation on Python 3.13, 3.14, and 3.15. Resolves lost per-test line data caused by:
sys.monitoringcallbacks running in a snapshot context whereContextVarchanges are not visible (Python 3.14+); empty modules emitting noLINEevents (Python 3.13+); andProcessPoolExecutorchild coverage not being propagated to the parent context. Also fixes a stale-data bug where child process executable lines could inflate coverage denominators afterstop_coverage()was called beforejoin().
- datastreams: Demotes the
retry limit exceeded submitting pathway statslog fromERRORtoWARNINGand removes the multi-line traceback from the record. This message fires when the processor cannot reach the agent within its 1-second timeout; the dropped 10 seconds of DSM data is auto-recovered on the next flush.
- LLM Observability: Fixes a concurrency bug in the Bedrock Agent integration where concurrent
invoke_agentcalls could orphan or cross-attribute spans due to shared class-level state. Per-invocation state is now used.
- LLM Observability: This fix resolves an issue where text wrapped in Bedrock Converse
guardContentcontent blocks was rendered as[Unsupported content type: guardContent]in traces, dropping the user's input.
- Fixed an issue that could have caused some instrumented code to fail to execute correctly when the original function had keyword arguments passed in as a cell variable.
- CI Visibility: Fixes an issue where tests marked as attempt-to-fix could have failures hidden when they were also quarantined or disabled.
- django: Stop tagging async view and middleware spans as errored on routine ASGI cancellations (e.g. client disconnects on streaming responses), a regression introduced in 4.8.0rc4. Cancellation still propagates; the span just finishes without
error.type='asyncio.exceptions.CancelledError'.
- django: Fixes
DD_DJANGO_DATABASE_SERVICEandDD_DJANGO_DATABASE_SERVICE_NAME, which were previously generated asDD_DJANGO-DATABASE_SERVICEandDD_DJANGO-DATABASE_SERVICE_NAME. The hyphenated names were invalid POSIX identifiers and unusable from most shells. Hyphens in integration names are now normalized to underscores when building env var names. The old hyphenated names are preserved as aliases for backward compatibility. #17952
- django: API endpoint discovery now supports Django sub-applications mounted with
django.urls.include(...). Endpoints are reported with their full URL path i...
4.8.7
4.8.6
Estimated end-of-life date, accurate to within three months: 05-2027
See the support level definitions for more information.
Bug Fixes
- LLM Observability: Users auto instrumenting Langchain and using Bedrock inference profiles would have their spans' be associated with the inference profile's ARN instead of the underlying LLM model. We now resolve the correct underlying model if users pass
base_model_idto theirChatBedrockConverseinstantiation. #18151
Full Changelog: v4.8.5...v4.8.6
4.9.0rc2
Estimated end-of-life date, accurate to within three months: 10-2027
See the support level definitions for more information.
Upgrade Notes
- AI Guard:
ddtrace.appsec.ai_guard.AIGuardAbortErrornow derives fromddtrace.internal._exceptions.DDBlockException(aBaseExceptionsubclass) instead ofException. This brings AI Guard block decisions in line with how ASM blocks are surfaced and prevents a genericexcept Exception:in user code from silently swallowing a block.
- settings: Legacy environment variable names registered as aliases in the configuration registry now also work when set via local or fleet stable config files, not just shell environment variables. #17958
Deprecation Notes
- Tracing:
DD_TRACE_INFERRED_SPANS_ENABLEDis deprecated and will be removed in 5.0.0. UseDD_TRACE_INFERRED_PROXY_SERVICES_ENABLEDinstead. The old environment variable continues to work but emits aDDTraceDeprecationWarningwhen set.
New Features
- aws_durable_execution_sdk_python: Add tracing support for the
aws-durable-execution-sdk-pythonlibrary. Instruments@durable_executionworkflows andDurableContextoperations (step,invoke,wait,wait_for_condition,wait_for_callback,create_callback,map,parallel,run_in_child_context) to generate spans.
- LLM Observability: Adds
stepspans to the Claude Agent SDK integration. Each inference cycle is now represented by astepcontainer span with anllmchild span for the model call andtoolchild spans for any tool invocations.
- tracing: Adds a centralized
supported-configurations.jsonregistry of all supportedDD_*andOTEL_*environment variables, following the same schema used by other Datadog tracing libraries. Accesses to unregistered environment variables now produce a debug log to help identify typos or unsupported configuration options.
- AI Guard: Copies anomaly-detection attributes from the local root (service-entry) span onto every
ai_guardspan:ai_guard.http.useragent,ai_guard.http.client_ip,ai_guard.network.client.ip,ai_guard.usr.idandai_guard.usr.session_id.
- ai_guard: add AI Guard evaluation support to the OpenAI SDK chat completions instrumentation. Both non-streaming and streaming requests and non-streaming responses are evaluated through the configured AI Guard client, and evaluation is automatically skipped when a framework integration (LangChain, Strands Agents) is already evaluating the same call.
- code origin for spans: The code origin for spans feature has been enabled by default.
- code origin: attach code origin information to the first span generated by a function wrapped with tracer.wrap.
- openfeature: This introduces a configurable initialization timeout for
DataDogProvider. The timeout controls how longinitialize()waits for configuration before returning, and defaults to 30 seconds. Set it via theDD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MSenvironment variable or theinit_timeoutconstructor parameter.
- CI Visibility: This introduces Jenkins custom parent ID propagation, which enables Datadog to correlate tests run from Jenkins with their Jenkins jobs and pipelines.
- LLM Observability: Adds an optional
cost_tagsargument toLLMObs.annotate()andLLMObs.annotation_context(). Pass a list of tag keys (already set viatagsor annotated previously on the same span) to have them attached to the cost and token metrics generated from LLM and embedding spans, which can help breaking down spend by team, project, org, or any custom dimension.
- LLM Observability: Adds support for an optional
version(string) field on each tool definition dictionary passed toLLMObs.annotate()via thetool_definitionsparameter.
- profiling: Add
DD_PROFILING_LOCK_EXCLUDE_MODULESconfig to skip lock profiling for framework-internal locks. Excluded locks remain native with zero profiling overhead. Set it to a comma-separated list of module prefixes (e.g.,django.db,sqlalchemy.pool,urllib3).
- LLM Observability: Bedrock Agent orchestration step events (model invocations, tool/action group calls, knowledge base lookups, guardrails, rationales) are now emitted as APM child spans of the
Bedrock Agent <agent_id>span when LLM Observability is enabled, with the same LLMObs payload shape as before.
Bug Fixes
- ai_guard: This fix resolves a conflict between
ddtrace.autoandstrandswhen imported in the same file, which left Strands hooks silently disabled. The Strands integration now loads lazily on first attribute access so its event class identities match those the agent dispatches.
- appsec: Adds telemetry metrics
instrum.user_auth.missing_user_loginandinstrum.user_auth.missing_user_idwhen Django auth events cannot resolve the expected identity fields, enabling detection of misconfigured user model field mappings.
- AAP: This fix resolves an issue where the
usr.session_idtag was missing from the entry span of authenticated follow-up Django requests when automatic user instrumentation was enabled. They now also carryusr.session_id, matching other authenticated user-tagging paths.
- azure_cosmos: This change removes the http.status_code tags from Azure CosmosDB spans and replaces them with the use of the db.response.status_code metric. For customers using ddtrace v4.8.0 and relying on the http.status_code tag of cosmosdb.query spans, this is a breaking change.
- CI Visibility: Fixes an issue in the pytest plugin where a malformed log call emitted a
--- Logging error ---traceback to stderr during Attempt to Fix retries, polluting pytest output and contributing to spurious test failures.
- CI Visibility: Fixes an
IndexErrorin retry bookkeeping that occurred when a test's teardown phase failed. The error produced--- Logging error ---tracebacks in stderr, which could pollute test output and cause spurious test failures during retries. #17863
- CI Visibility: Fixes a regression where setting
DD_TEST_MANAGEMENT_ENABLED=0was not honored by the new pytest plugin, causing Test Management features such as quarantining, disabling tests, and Attempt to Fix to remain enabled.
- datastreams: Demotes the
retry limit exceeded submitting pathway statslog fromERRORtoWARNINGand removes the multi-line traceback from the record. This message fires when the processor cannot reach the agent within its 1-second timeout; the dropped 10 seconds of DSM data is auto-recovered on the next flush.
- LLM Observability: Fixes a concurrency bug in the Bedrock Agent integration where concurrent
invoke_agentcalls could orphan or cross-attribute spans due to shared class-level state. Per-invocation state is now used.
- LLM Observability: This fix resolves an issue where text wrapped in Bedrock Converse
guardContentcontent blocks was rendered as[Unsupported content type: guardContent]in traces, dropping the user's input.
- CI Visibility: Fixes an issue where tests marked as attempt-to-fix could have failures hidden when they were also quarantined or disabled.
- django: Stop tagging async view and middleware spans as errored on routine ASGI cancellations (e.g. client disconnects on streaming responses), a regression introduced in 4.8.0rc4. Cancellation still propagates; the span just finishes without
error.type='asyncio.exceptions.CancelledError'.
- django: Fixes
DD_DJANGO_DATABASE_SERVICEandDD_DJANGO_DATABASE_SERVICE_NAME, which were previously generated asDD_DJANGO-DATABASE_SERVICEandDD_DJANGO-DATABASE_SERVICE_NAME. The hyphenated names were invalid POSIX identifiers and unusable from most shells. Hyphens in integration names are now normalized to underscores when building env var names. The old hyphenated names are preserved as aliases for backward compatibility. #17952
- telemetry: tolerate malformed installed distribution metadata so a single bad dist-info entry no longer floods stderr with repeated tracebacks.
- profiling: Fixes an issue where the lock profiler silently stopped capturing lock events when running under
ddtrace-runwith gevent installed.
- LLM Observability: The OpenAI integration now preserves assistant message content when
tool_callsare present on the same message. #17760
- openfeature: This fix resolves an issue where
DataDogProvider.initialize()returned before configuration was received, causing the OpenFeature SDK to mark the provider as ready to serve evaluations too early and flag evaluations to silently return default values. The provider now waits for configuration before returning.
- openfeature: Fixes targeting key handling in the OpenFeature provider.
Nonetargeting key is now correctly passed to the native evaluator instead of being coerced to empty string. Flags that don't require a targeting key (static, rule-based) now evaluate successfully without one, matching the Datadog provider spec. Additionally, the Rust binding now correctly mapsTargetingKeyMissingerrors from libdatadog instead of returning a generic error code.
- tracing: Fixes an issue where the
svc.autoprocess tag produced garbled values such aspython_-m_unittestwhen a process was launched with the full command as a singlesys.argv[0]string (e.g. from a DockerENTRYPOINT, a process manager, or a subprocess call with an unsplit command). The correct module or script nam...