Skip to content

Releases: DataDog/dd-trace-py

4.8.8

05 Jun 10:05
517dc80

Choose a tag to compare

Bug Fixes

  • Fixed an issue that could have caused some timers, like the one responsible for Symbol Database uploads, to fire repeatedly after the first execution.
  • internal: This fix resolves a memory leak where reference cycles through PeriodicThread callbacks were invisible to Python's cyclic garbage collector and could accumulate when threads used bound methods as targets.
  • profiling: Fixes a memory leak in native frame tracking caused by unbounded native call-site metadata growth.
  • SCA: This fix resolves an issue where unresolved runtime reachability targets could accumulate across Software Composition Analysis updates, causing resident memory usage to grow over time.
  • sqlalchemy: Fixes duplicate SQLAlchemy event listeners when trace_engine() is called repeatedly for the same engine.
  • dynamic instrumentation: fixes an issue where the Symbol Database uploader sends empty payloads on a recurring timer.

4.10.2

04 Jun 14:19
c693939

Choose a tag to compare

Bug Fixes

  • LLM Observability: This fix resolves an issue in the Claude Agent SDK integration where a span's error message showed an uncategorized unknown error category from the upstream Claude Agent SDK instead of a descriptive API error. The integration now surfaces the detailed error message from the assistant message content.
  • tracing: Fixes a race condition where extra service names could be silently dropped from Remote Configuration /v0.7/config payloads in multi-threaded applications (e.g. uWSGI).
  • code origin: fixed an issue that could have caused pytest to crash internally when inspecting the call stack from an exception thrown by a view function when Code Origin is enabled.
  • LLM Observability: Resolves an issue where non-string tag values passed to LLMObs.annotate(tags=...) could cause spans to be dropped during ingestion.
  • LLM Observability: Fixes provider mis-attribution on openai spans when an OpenAI (or AsyncOpenAI) client and an AzureOpenAI (or AsyncAzureOpenAI) client are instantiated at the same time. Provider is now determined per-call rather than from the most recently constructed client.

4.11.0rc1

03 Jun 14:29
8bbc564

Choose a tag to compare

4.11.0rc1 Pre-release
Pre-release

Upgrade Notes

  • flask: Requests served under a non-empty WSGI SCRIPT_NAME now expose the client-hit resource on a new flask.resource.full tag (e.g. GET /api/v2/users). The span resource and flask.url_rule tag are unchanged. The tag is only set when its value would differ from span.resource.
  • flask: API endpoint discovery now reports every HTTP method the framework serves — including Werkzeug's auto-added HEAD for any GET route and Flask's auto-handled OPTIONS for every route — not just the methods listed in methods=[...].

New Features

  • tracing: collect the x-datadog-endpoint-scan and x-datadog-security-test HTTP request headers on service entry spans unconditionally as http.request.headers.x-datadog-endpoint-scan and http.request.headers.x-datadog-security-test tags. These markers identify Datadog-originated endpoint scans and security tests so the API inventory pipeline can distinguish scan/test traffic from real user traffic. The headers are tagged regardless of DD_TRACE_HEADER_TAGS configuration or AppSec enablement, and are not propagated to downstream services.
  • Database Monitoring (DBM) propagation supports dynamic_service as a new DD_DBM_PROPAGATION_MODE value. Set DD_DBM_PROPAGATION_MODE=dynamic_service to inject DBM service metadata and the SQL base hash without injecting trace context.
  • ai_guard: add AI Guard evaluation support to the Anthropic SDK Messages instrumentation. Both non-streaming and streaming requests and non-streaming responses are evaluated through the configured AI Guard client (covering Messages.create / Messages.stream and their async and Beta variants), and evaluation is automatically skipped when a framework integration (LangChain) is already evaluating the same call.
  • aiokafka: Adds kafka.partition and kafka.message_offset tags to the producer span once the broker acknowledges the send. The partition reflects the partition the broker actually assigned to the message, which may differ from the partition the caller requested. Mirrors the behavior added to the Java tracer in DataDog/dd-trace-java#11107.
  • ASM: This introduces the _dd.appsec.normalized_route span tag for FastAPI and Starlette request spans when API Security is enabled. The tag follows RFC-1103 and provides a per-request, framework-agnostic representation of the matched route — converter types are stripped, multi-parameter URL segments are combined with +, path catch-all parameters are emitted as a single tail element, and trailing slashes are preserved as declared. Mount-prefixed sub-application routes are reported with their full assembled path.
  • ASM: Extends _dd.appsec.normalized_route span tag support to Flask request spans when API Security is enabled. Flask / Werkzeug <converter:name> route syntax is normalized following RFC-1103: converter types are stripped, multi-parameter URL segments (e.g. /<first>.<last>/) are combined with +, <path:name> catch-all parameters are emitted as a single tail element, and trailing slashes are preserved as declared. Routes served through DispatcherMiddleware sub-apps are reported with their full assembled path (mount prefix included).
  • LLM Observability: The claude_agent_sdk integration now emits span links between step, LLM, and tool spans so multi-step traces render the sequencing between LLM calls and tool calls.
  • aws_durable_execution_sdk_python: Adds distributed tracing for durable workflows across suspend/resume cycles, so a workflow that pauses and resumes in a later invocation appears as a single connected trace. See aws_durable_execution_sdk_python for details and opt-out configuration.
  • AAP: API endpoint discovery now covers Flask sub-applications mounted via werkzeug.middleware.dispatcher.DispatcherMiddleware. Sub-app routes are reported with their full mounted path.
  • google_cloud_pubsub: This introduces Data Streams Monitoring (DSM) context propagation for the Google Cloud Pub/Sub integration. Producer publish operations inject the DSM pathway context into message attributes, and subscriber callbacks extract it and record a consume checkpoint. To enable, set DD_DATA_STREAMS_ENABLED=true.
  • LLM Observability: automatically tags spans and experiments with git.commit.sha and git.repository_url. Values come from DD_GIT_COMMIT_SHA / DD_GIT_REPOSITORY_URL or the main package's Project-URL metadata, falling back to running git against the current working directory. Honors DD_TRACE_GIT_METADATA_ENABLED and user-supplied tags with the same keys.
  • LLM Observability: experiment evaluators and summary evaluators can now return a MultiEvaluatorResult to emit multiple named evaluation metrics from a single evaluator call. Each sub-value may itself be an EvaluatorResult carrying its own reasoning, assessment, metadata, and tags. By default emitted metric labels are prefixed with the evaluator's name ("<evaluator_name>-<key>"); pass prefix=False to emit raw keys.
  • LLM Observability: Add LLMObs.pull_experiment(experiment_id) to fetch a previously-run experiment from the Datadog backend by UUID. The returned SyncExperiment has its .result populated and is ready for downstream inspection (e.g. .as_dataframe()) without re-executing the original task.
  • LLM Observability: task and dataset are now optional when creating an experiment via LLMObs.experiment() / LLMObs.async_experiment(), supporting pull-based workflows. Calling run() without providing task and dataset raises a ValueError.
  • LLM Observability: When a tool definition with a version field is set on a parent LLM span via tool_definitions, the resolved tool version is now also written onto manually-started child tool spans (created via LLMObs.tool()) as meta.tool.version. Previously, this propagation only occurred for tool spans produced by auto-instrumented integrations (OpenAI, LangChain, OpenAI Agents, ReAct) through the LinkTracker dispatch mechanism.
  • LLM Observability: When a tool definition with a version field is set on a parent LLM span via tool_definitions, the resolved tool version is now also written onto the corresponding child tool span as meta.tool.version. This allows the LLM Observability UI to aggregate tool version counts and filter tool spans by version without correlating to the parent LLM span.
  • profiling: The fast stack profiler optimisation is now enabled by default.
  • Profiling: The heap profiler can now track allocations made through PYMEM_DOMAIN_MEM (PyMem_Malloc/Calloc/Realloc) in addition to the existing PYMEM_DOMAIN_OBJ coverage when DD_PROFILING_MEMORY_MEM_DOMAIN_ENABLED=true is set. This support is effective on Python 3.12 and newer, making some previously invisible allocations visible in heap profiles in those environments.
  • ray: Added DD_ML_JOB_ENV — a single opt-in env var for forwarding configuration into Ray job workers. Set it on the dashboard process as a semicolon-separated list of KEY:VALUE pairs, e.g. DD_SERVICE:my-svc;DD_AGENT_HOST:10.0.0.1, and workers receive them verbatim as DD_SERVICE=my-svc, DD_AGENT_HOST=10.0.0.1, etc.
  • CI Visibility: Adds ddtrace.testing.logs.DDTestLogsHandler, a public logging.Handler for shipping log records to the Datadog logs intake correlated with CI Visibility test traces, intended for use in subprocesses that execute tests outside of the main pytest process. Also adds ddtrace.testing.logs.CorrelationFilter, a base class for stamping dd.trace_id / dd.span_id onto log records, and a ready-made ddtrace.testing.logs.ThreadLocalCorrelationFilter for the common thread-local case. Subclass CorrelationFilter to plug in other concurrency models (asyncio ContextVar, global state, etc.).

    Example:

    from pydantic_evals.evaluators import EqualsExpected import logging

    from ddtrace.testing.logs import DDTestLogsHandler from ddtrace.testing.logs import ThreadLocalCorrelationFilter

    handler = DDTestLogsHandler(service="my-service") correlation = ThreadLocalCorrelationFilter() handler.addFilter(correlation) logging.getLogger().addHandler(handler)

    while True:
    job = queue.get() correlation.set_context(trace_id=job.trace_id, span_id=job.span_id) run_test(job.item)

    handler.close()

Bug Fixes

  • AAP: This fix resolves an issue where the AppSec body-parsing hook consumed the websocket.connect ASGI message, causing ASGI/FastAPI WebSocket connections to fail with HTTP 500 when AppSec was enabled.
  • aiokafka: Collects the Kafka cluster ID when tracing aiokafka producers and consumers and forwards it to Data Streams Monitoring, matching the behavior of the confluent_kafka integration.
  • aap: Adds instrumentation telemetry distributions for WAF and RASP execution durations, including waf.duration, waf.duration_ext, rasp.duration, and rasp.duration_ext.
  • azure_cosmos: This fix resolves an issue where cosmosdb.query span resource names included per-item identifiers (such as document, user, and permission ids) from the request URI, which caused unbounded resource cardinality. The integration now redacts those ids by replacing them with ? while keeping the database (dbs) and collection (colls) names intact. For example, Read /dbs/myDb/colls/myColl/docs/item1 is now reported as Read /dbs/myDb/colls/myColl/docs/?.
  • CI Visibility: fix the default HTTP timeout for backend requests from 15 seconds to 30 seconds, and add the DD_CIVISIBILITY_BACKEND_API_TIMEOUT_MILLIS environment variable (previously missing) to override it. The value is expressed in m...
Read more

4.10.1

01 Jun 15:52
8d50296

Choose a tag to compare

Bug Fixes

  • internal: Fixed an issue that could have caused some timers, like the one responsible for Symbol Database uploads, to fire repeatedly after the first execution.
  • internal: This fix resolves a memory leak where reference cycles through PeriodicThread callbacks were invisible to Python's cyclic garbage collector and could accumulate when threads used bound methods as targets.
  • profiling: Fixes a memory leak in native frame tracking caused by unbounded native call-site metadata growth.
  • SCA: This fix resolves an issue where unresolved runtime reachability targets could accumulate across Software Composition Analysis updates, causing resident memory usage to grow over time.

4.10.0

29 May 15:14
84ae72d

Choose a tag to compare

Bug Fixes

  • AAP: This fix resolves an issue where the AppSec body-parsing hook consumed the websocket.connect ASGI message, causing ASGI/FastAPI WebSocket connections to fail with HTTP 500 when AppSec was enabled.
  • LLM Observability: Fixes an issue where reasoning_content was missing from streamed chat completions in the OpenAI and LiteLLM integrations when an OpenAI-compatible reasoning provider (e.g. DeepSeek, Qwen) emitted delta.reasoning_content chunks. The aggregated message now captures reasoning text in the output message, matching non-streaming behavior.
  • dynamic instrumentation: fixes an issue where the Symbol Database uploader sends empty payloads on a recurring timer.

Other Changes

  • LLM Observability: when LLMObs is enabled in agentless mode (Datadog Agent not reachable or with DD_LLMOBS_AGENTLESS_ENABLED=1), APM traces are now exported agentlessly to Datadog's intake. This should not change user-facing behavior: both APM and LLMObs spans remain visible in the UI; LLMObs spans are simply no longer shipped separately for agentless users. Note that setting DD_APM_TRACING_ENABLED=false takes higher precedence and will result in LLMObs span events shipping separately as existing behavior.

4.10.0rc1

27 May 18:09
db5e3da

Choose a tag to compare

4.10.0rc1 Pre-release
Pre-release

Other Changes

  • LLM Observability: when LLMObs is enabled in agentless mode (Datadog Agent not reachable or with DD_LLMOBS_AGENTLESS_ENABLED=1), APM traces are now exported agentlessly to Datadog's intake. This should not change user-facing behavior: both APM and LLMObs spans remain visible in the UI; LLMObs spans are simply no longer shipped separately for agentless users. Note that setting DD_APM_TRACING_ENABLED=false takes higher precedence and will result in LLMObs span events shipping separately as existing behavior.

4.9.0

26 May 13:41
3173c73

Choose a tag to compare

Upgrade Notes

  • AI Guard: ddtrace.appsec.ai_guard.AIGuardAbortError now derives from ddtrace.internal._exceptions.DDBlockException (a BaseException subclass) instead of Exception. This brings AI Guard block decisions in line with how ASM blocks are surfaced and prevents a generic except Exception: in user code from silently swallowing a block.
  • settings: Legacy environment variable names registered as aliases in the configuration registry now also work when set via local or fleet stable config files, not just shell environment variables. #17958
  • ray: Adds DD_TRACE_RAY_IGNORED_ACTORS configuration to exclude specific Ray actor methods from instrumentation. Set DD_TRACE_RAY_IGNORED_ACTORS='{"ActorA": ["method1"], "ActorB": "*"}' to leave matching methods or actors uninstrumented while continuing to trace other Ray actor methods. Matching is based on actor class name only.

Deprecation Notes

  • Tracing: DD_TRACE_INFERRED_SPANS_ENABLED is deprecated and will be removed in 5.0.0. Use DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED instead. The old environment variable continues to work but emits a DDTraceDeprecationWarning when set.

New Features

  • aws_durable_execution_sdk_python: Add tracing support for the aws-durable-execution-sdk-python library. Instruments @durable_execution workflows and DurableContext operations (step, invoke, wait, wait_for_condition, wait_for_callback, create_callback, map, parallel, run_in_child_context) to generate spans.
  • LLM Observability: Adds step spans to the Claude Agent SDK integration. Each inference cycle is now represented by a step container span with an llm child span for the model call and tool child spans for any tool invocations.
  • tracing: Adds a centralized supported-configurations.json registry of all supported DD_* and OTEL_* environment variables, following the same schema used by other Datadog tracing libraries. Accesses to unregistered environment variables now produce a debug log to help identify typos or unsupported configuration options.
  • AI Guard: Copies anomaly-detection attributes from the local root (service-entry) span onto every ai_guard span: ai_guard.http.useragent, ai_guard.http.client_ip, ai_guard.network.client.ip, ai_guard.usr.id and ai_guard.usr.session_id.
  • AI Guard: When DD_AI_GUARD_ENABLED=true is set and an ai_guard span is created during a request, the tracer now populates http.client_ip and network.client.ip on the service-entry (local root) span, mirroring the behavior used for Application Security. If AI Guard does not run during the request, no client IP tags are added. DD_TRACE_CLIENT_IP_ENABLED is ignored once AI Guard reports, and DD_TRACE_CLIENT_IP_HEADER continues to override header resolution.
  • ai_guard: add AI Guard evaluation support to the OpenAI SDK chat completions instrumentation. Both non-streaming and streaming requests and non-streaming responses are evaluated through the configured AI Guard client, and evaluation is automatically skipped when a framework integration (LangChain, Strands Agents) is already evaluating the same call.
  • code origin for spans: The code origin for spans feature has been enabled by default.
  • code origin: attach code origin information to the first span generated by a function wrapped with tracer.wrap.
  • openfeature: This introduces a configurable initialization timeout for DataDogProvider. The timeout controls how long initialize() waits for configuration before returning, and defaults to 10 seconds. Set it via the DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS environment variable or the init_timeout constructor parameter.
  • CI Visibility: This introduces Jenkins custom parent ID propagation, which enables Datadog to correlate tests run from Jenkins with their Jenkins jobs and pipelines.
  • LLM Observability: Adds an optional cost_tags argument to LLMObs.annotate() and LLMObs.annotation_context(). Pass a list of tag keys (already set via tags or annotated previously on the same span) to have them attached to the cost and token metrics generated from LLM and embedding spans, which can help breaking down spend by team, project, org, or any custom dimension.
  • LLM Observability: Adds support for an optional version (string) field on each tool definition dictionary passed to LLMObs.annotate() via the tool_definitions parameter.
  • profiling: Add DD_PROFILING_LOCK_EXCLUDE_MODULES config to skip lock profiling for framework-internal locks. Excluded locks remain native with zero profiling overhead. Set it to a comma-separated list of module prefixes (e.g., django.db,sqlalchemy.pool,urllib3).
  • LLM Observability: Bedrock Agent orchestration step events (model invocations, tool/action group calls, knowledge base lookups, guardrails, rationales) are now emitted as APM child spans of the Bedrock Agent <agent_id> span when LLM Observability is enabled, with the same LLMObs payload shape as before.

Bug Fixes

  • tracing: Exclude wrapt==2.2.0 from the supported dependency range to avoid a regression that breaks wrapped C descriptors.

  • ai_guard: This fix resolves a conflict between ddtrace.auto and strands when imported in the same file, which left Strands hooks silently disabled. The Strands integration now loads lazily on first attribute access so its event class identities match those the agent dispatches.

  • appsec: Adds telemetry metrics instrum.user_auth.missing_user_login and instrum.user_auth.missing_user_id when Django auth events cannot resolve the expected identity fields, enabling detection of misconfigured user model field mappings.
  • AAP: This fix resolves an issue where the usr.session_id tag was missing from the entry span of authenticated follow-up Django requests when automatic user instrumentation was enabled. They now also carry usr.session_id, matching other authenticated user-tagging paths.
  • azure_cosmos: This change removes the http.status_code tags from Azure CosmosDB spans and replaces them with the use of the db.response.status_code metric. For customers using ddtrace v4.8.0 and relying on the http.status_code tag of cosmosdb.query spans, this is a breaking change.
  • CI Visibility: Fixes an issue in the pytest plugin where a malformed log call emitted a --- Logging error --- traceback to stderr during Attempt to Fix retries, polluting pytest output and contributing to spurious test failures.
  • CI Visibility: Fixes an IndexError in retry bookkeeping that occurred when a test's teardown phase failed. The error produced --- Logging error --- tracebacks in stderr, which could pollute test output and cause spurious test failures during retries. #17863
  • CI Visibility: Fixes a regression where setting DD_TEST_MANAGEMENT_ENABLED=0 was not honored by the new pytest plugin, causing Test Management features such as quarantining, disabling tests, and Attempt to Fix to remain enabled.
  • CI Visibility: Fixes code coverage instrumentation on Python 3.13, 3.14, and 3.15. Resolves lost per-test line data caused by: sys.monitoring callbacks running in a snapshot context where ContextVar changes are not visible (Python 3.14+); empty modules emitting no LINE events (Python 3.13+); and ProcessPoolExecutor child coverage not being propagated to the parent context. Also fixes a stale-data bug where child process executable lines could inflate coverage denominators after stop_coverage() was called before join().
  • datastreams: Demotes the retry limit exceeded submitting pathway stats log from ERROR to WARNING and removes the multi-line traceback from the record. This message fires when the processor cannot reach the agent within its 1-second timeout; the dropped 10 seconds of DSM data is auto-recovered on the next flush.
  • LLM Observability: Fixes a concurrency bug in the Bedrock Agent integration where concurrent invoke_agent calls could orphan or cross-attribute spans due to shared class-level state. Per-invocation state is now used.
  • LLM Observability: This fix resolves an issue where text wrapped in Bedrock Converse guardContent content blocks was rendered as [Unsupported content type: guardContent] in traces, dropping the user's input.
  • Fixed an issue that could have caused some instrumented code to fail to execute correctly when the original function had keyword arguments passed in as a cell variable.
  • CI Visibility: Fixes an issue where tests marked as attempt-to-fix could have failures hidden when they were also quarantined or disabled.
  • django: Stop tagging async view and middleware spans as errored on routine ASGI cancellations (e.g. client disconnects on streaming responses), a regression introduced in 4.8.0rc4. Cancellation still propagates; the span just finishes without error.type='asyncio.exceptions.CancelledError'.
  • django: Fixes DD_DJANGO_DATABASE_SERVICE and DD_DJANGO_DATABASE_SERVICE_NAME, which were previously generated as DD_DJANGO-DATABASE_SERVICE and DD_DJANGO-DATABASE_SERVICE_NAME. The hyphenated names were invalid POSIX identifiers and unusable from most shells. Hyphens in integration names are now normalized to underscores when building env var names. The old hyphenated names are preserved as aliases for backward compatibility. #17952
  • django: API endpoint discovery now supports Django sub-applications mounted with django.urls.include(...). Endpoints are reported with their full URL path i...
Read more

4.8.7

22 May 20:01
c2ae3a9

Choose a tag to compare

Bug Fixes

  • tracing: Exclude wrapt==2.2.0 from the supported dependency range to avoid a regression that breaks wrapped C descriptors.
  • LLM Observability: Agent connection error logs are no longer logged when using agentless and not running a Datadog agent.

4.8.6

19 May 15:52
041f76a

Choose a tag to compare

Estimated end-of-life date, accurate to within three months: 05-2027
See the support level definitions for more information.

Bug Fixes

  • LLM Observability: Users auto instrumenting Langchain and using Bedrock inference profiles would have their spans' be associated with the inference profile's ARN instead of the underlying LLM model. We now resolve the correct underlying model if users pass base_model_id to their ChatBedrockConverse instantiation. #18151

Full Changelog: v4.8.5...v4.8.6

4.9.0rc2

15 May 13:32
279b0ba

Choose a tag to compare

4.9.0rc2 Pre-release
Pre-release

Estimated end-of-life date, accurate to within three months: 10-2027
See the support level definitions for more information.

Upgrade Notes

  • AI Guard: ddtrace.appsec.ai_guard.AIGuardAbortError now derives from ddtrace.internal._exceptions.DDBlockException (a BaseException subclass) instead of Exception. This brings AI Guard block decisions in line with how ASM blocks are surfaced and prevents a generic except Exception: in user code from silently swallowing a block.
  • settings: Legacy environment variable names registered as aliases in the configuration registry now also work when set via local or fleet stable config files, not just shell environment variables. #17958

Deprecation Notes

  • Tracing: DD_TRACE_INFERRED_SPANS_ENABLED is deprecated and will be removed in 5.0.0. Use DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED instead. The old environment variable continues to work but emits a DDTraceDeprecationWarning when set.

New Features

  • aws_durable_execution_sdk_python: Add tracing support for the aws-durable-execution-sdk-python library. Instruments @durable_execution workflows and DurableContext operations (step, invoke, wait, wait_for_condition, wait_for_callback, create_callback, map, parallel, run_in_child_context) to generate spans.
  • LLM Observability: Adds step spans to the Claude Agent SDK integration. Each inference cycle is now represented by a step container span with an llm child span for the model call and tool child spans for any tool invocations.
  • tracing: Adds a centralized supported-configurations.json registry of all supported DD_* and OTEL_* environment variables, following the same schema used by other Datadog tracing libraries. Accesses to unregistered environment variables now produce a debug log to help identify typos or unsupported configuration options.
  • AI Guard: Copies anomaly-detection attributes from the local root (service-entry) span onto every ai_guard span: ai_guard.http.useragent, ai_guard.http.client_ip, ai_guard.network.client.ip, ai_guard.usr.id and ai_guard.usr.session_id.
  • ai_guard: add AI Guard evaluation support to the OpenAI SDK chat completions instrumentation. Both non-streaming and streaming requests and non-streaming responses are evaluated through the configured AI Guard client, and evaluation is automatically skipped when a framework integration (LangChain, Strands Agents) is already evaluating the same call.
  • code origin for spans: The code origin for spans feature has been enabled by default.
  • code origin: attach code origin information to the first span generated by a function wrapped with tracer.wrap.
  • openfeature: This introduces a configurable initialization timeout for DataDogProvider. The timeout controls how long initialize() waits for configuration before returning, and defaults to 30 seconds. Set it via the DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS environment variable or the init_timeout constructor parameter.
  • CI Visibility: This introduces Jenkins custom parent ID propagation, which enables Datadog to correlate tests run from Jenkins with their Jenkins jobs and pipelines.
  • LLM Observability: Adds an optional cost_tags argument to LLMObs.annotate() and LLMObs.annotation_context(). Pass a list of tag keys (already set via tags or annotated previously on the same span) to have them attached to the cost and token metrics generated from LLM and embedding spans, which can help breaking down spend by team, project, org, or any custom dimension.
  • LLM Observability: Adds support for an optional version (string) field on each tool definition dictionary passed to LLMObs.annotate() via the tool_definitions parameter.
  • profiling: Add DD_PROFILING_LOCK_EXCLUDE_MODULES config to skip lock profiling for framework-internal locks. Excluded locks remain native with zero profiling overhead. Set it to a comma-separated list of module prefixes (e.g., django.db,sqlalchemy.pool,urllib3).
  • LLM Observability: Bedrock Agent orchestration step events (model invocations, tool/action group calls, knowledge base lookups, guardrails, rationales) are now emitted as APM child spans of the Bedrock Agent <agent_id> span when LLM Observability is enabled, with the same LLMObs payload shape as before.

Bug Fixes

  • ai_guard: This fix resolves a conflict between ddtrace.auto and strands when imported in the same file, which left Strands hooks silently disabled. The Strands integration now loads lazily on first attribute access so its event class identities match those the agent dispatches.
  • appsec: Adds telemetry metrics instrum.user_auth.missing_user_login and instrum.user_auth.missing_user_id when Django auth events cannot resolve the expected identity fields, enabling detection of misconfigured user model field mappings.
  • AAP: This fix resolves an issue where the usr.session_id tag was missing from the entry span of authenticated follow-up Django requests when automatic user instrumentation was enabled. They now also carry usr.session_id, matching other authenticated user-tagging paths.
  • azure_cosmos: This change removes the http.status_code tags from Azure CosmosDB spans and replaces them with the use of the db.response.status_code metric. For customers using ddtrace v4.8.0 and relying on the http.status_code tag of cosmosdb.query spans, this is a breaking change.
  • CI Visibility: Fixes an issue in the pytest plugin where a malformed log call emitted a --- Logging error --- traceback to stderr during Attempt to Fix retries, polluting pytest output and contributing to spurious test failures.
  • CI Visibility: Fixes an IndexError in retry bookkeeping that occurred when a test's teardown phase failed. The error produced --- Logging error --- tracebacks in stderr, which could pollute test output and cause spurious test failures during retries. #17863
  • CI Visibility: Fixes a regression where setting DD_TEST_MANAGEMENT_ENABLED=0 was not honored by the new pytest plugin, causing Test Management features such as quarantining, disabling tests, and Attempt to Fix to remain enabled.
  • datastreams: Demotes the retry limit exceeded submitting pathway stats log from ERROR to WARNING and removes the multi-line traceback from the record. This message fires when the processor cannot reach the agent within its 1-second timeout; the dropped 10 seconds of DSM data is auto-recovered on the next flush.
  • LLM Observability: Fixes a concurrency bug in the Bedrock Agent integration where concurrent invoke_agent calls could orphan or cross-attribute spans due to shared class-level state. Per-invocation state is now used.
  • LLM Observability: This fix resolves an issue where text wrapped in Bedrock Converse guardContent content blocks was rendered as [Unsupported content type: guardContent] in traces, dropping the user's input.
  • CI Visibility: Fixes an issue where tests marked as attempt-to-fix could have failures hidden when they were also quarantined or disabled.
  • django: Stop tagging async view and middleware spans as errored on routine ASGI cancellations (e.g. client disconnects on streaming responses), a regression introduced in 4.8.0rc4. Cancellation still propagates; the span just finishes without error.type='asyncio.exceptions.CancelledError'.
  • django: Fixes DD_DJANGO_DATABASE_SERVICE and DD_DJANGO_DATABASE_SERVICE_NAME, which were previously generated as DD_DJANGO-DATABASE_SERVICE and DD_DJANGO-DATABASE_SERVICE_NAME. The hyphenated names were invalid POSIX identifiers and unusable from most shells. Hyphens in integration names are now normalized to underscores when building env var names. The old hyphenated names are preserved as aliases for backward compatibility. #17952
  • telemetry: tolerate malformed installed distribution metadata so a single bad dist-info entry no longer floods stderr with repeated tracebacks.
  • profiling: Fixes an issue where the lock profiler silently stopped capturing lock events when running under ddtrace-run with gevent installed.
  • LLM Observability: The OpenAI integration now preserves assistant message content when tool_calls are present on the same message. #17760
  • openfeature: This fix resolves an issue where DataDogProvider.initialize() returned before configuration was received, causing the OpenFeature SDK to mark the provider as ready to serve evaluations too early and flag evaluations to silently return default values. The provider now waits for configuration before returning.
  • openfeature: Fixes targeting key handling in the OpenFeature provider. None targeting key is now correctly passed to the native evaluator instead of being coerced to empty string. Flags that don't require a targeting key (static, rule-based) now evaluate successfully without one, matching the Datadog provider spec. Additionally, the Rust binding now correctly maps TargetingKeyMissing errors from libdatadog instead of returning a generic error code.
  • tracing: Fixes an issue where the svc.auto process tag produced garbled values such as python_-m_unittest when a process was launched with the full command as a single sys.argv[0] string (e.g. from a Docker ENTRYPOINT, a process manager, or a subprocess call with an unsplit command). The correct module or script nam...
Read more