Skip to content

chore: always build native heap profiling - #19701

Draft
KowalskiThomas wants to merge 2 commits into
mainfrom
kowalski/native-heap-always-build
Draft

chore: always build native heap profiling#19701
KowalskiThomas wants to merge 2 commits into
mainfrom
kowalski/native-heap-always-build

Conversation

@KowalskiThomas

Copy link
Copy Markdown
Collaborator

No description provided.

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.gitlab/package.yml                                                     @DataDog/python-guild @DataDog/apm-core-python
.gitlab/scripts/build-wheel-helpers.sh                                  @DataDog/python-guild @DataDog/apm-core-python
ddtrace/internal/datadog/profiling/heap_gotter/__init__.py              @DataDog/profiling-python
ddtrace/profiling/profiler.py                                           @DataDog/profiling-python
releasenotes/notes/native-heap-profiling-live-heap-336bc489c93989da.yaml  @DataDog/apm-python
setup.py                                                                @DataDog/python-guild
src/native_heap_gotter/Cargo.toml                                       @DataDog/profiling-python
src/native_heap_gotter/lib.rs                                           @DataDog/profiling-python
tests/profiling/test_native_heap_gotter.py                              @DataDog/profiling-python

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 5 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.contrib.internal.pytorch._distributed -> ddtrace.contrib.internal.pytorch._rank_root -> ddtrace.contrib.internal.pytorch._distributed
ddtrace.contrib.internal.django.patch -> ddtrace.contrib.internal.django.response -> ddtrace.contrib.internal.django.patch
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 255 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 255 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=134)
ddtrace.debugging._debugger -×-> ddtrace.trace  (product:debugging -> product:tracing, score=132)
ddtrace.llmobs._telemetry -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.profiling.collector.stack -×-> ddtrace.trace  (product:profiling -> product:tracing, score=132)
ddtrace.llmobs._integrations.langchain -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@datadog-official

datadog-official Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

🚦 33 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp310-cp310, v126532274-233089d-musllinux_1_2_x86_64, 1]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp313-cp313, v126532274-233089d-musllinux_1_2_x86_64, 1]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v126532274-233089d-musllinux_1_2_x86_64, 1]   View in Datadog   GitLab

View all 33 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: abd963f | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Aug 14, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-08-14 13:01:04

Comparing candidate commit abd963f in PR branch kowalski/native-heap-always-build with baseline commit 37d2e79 in branch main.

Found 0 performance improvements and 7 performance regressions! Performance is the same for 613 metrics, 10 unstable metrics.

scenario:httppropagationextract-datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match

  • 🟥 execution_time [+4.881µs; +5.158µs] or [+7.205%; +7.613%]

scenario:httppropagationinject-ids_only

  • 🟥 execution_time [+2.635µs; +2.791µs] or [+12.401%; +13.133%]

scenario:iastaspects-lower_aspect

  • 🟥 execution_time [+29.637µs; +35.970µs] or [+12.680%; +15.390%]

scenario:iastaspects-upper_aspect

  • 🟥 execution_time [+30.011µs; +33.856µs] or [+12.936%; +14.593%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+79.200µs; +84.764µs] or [+19.287%; +20.642%]

scenario:telemetryaddmetric-1-count-metric-1-times

  • 🟥 execution_time [+394.895ns; +445.220ns] or [+15.378%; +17.338%]

scenario:tracer-small

  • 🟥 execution_time [+27.917µs; +30.461µs] or [+8.003%; +8.732%]

@KowalskiThomas

Copy link
Copy Markdown
Collaborator Author

This worked as expected:

dog@signal-bundler-main-canary-86c7dc54d4-cmx49:/usr/local/lib/python3.12/site-packages/ddtrace$ find . -name *.so
./profiling/_threading.cpython-312-aarch64-linux-gnu.so
./profiling/collector/_memalloc.cpython-312-aarch64-linux-gnu.so
./profiling/collector/_task.cpython-312-aarch64-linux-gnu.so
./profiling/collector/_exception.cpython-312-aarch64-linux-gnu.so
./profiling/collector/_fast_poisson.cpython-312-aarch64-linux-gnu.so
./profiling/collector/_lock.cpython-312-aarch64-linux-gnu.so
./profiling/collector/_sampler.cpython-312-aarch64-linux-gnu.so
./appsec/_ddwaf/libddwaf/aarch64/lib/libddwaf.so
./appsec/_iast/_taint_tracking/_native.cpython-312-aarch64-linux-gnu.so
./appsec/_iast/_ast/iastpatch.cpython-312-aarch64-linux-gnu.so
./appsec/_shared/_stacktrace.cpython-312-aarch64-linux-gnu.so
./internal/datadog/profiling/stack/_stack.cpython-312-aarch64-linux-gnu.so
./internal/datadog/profiling/ddup/_ddup.cpython-312-aarch64-linux-gnu.so
./internal/datadog/profiling/libdd_wrapper.cpython-312-aarch64-linux-gnu.so
./internal/datadog/profiling/libdd_heap_gotter.cpython-312-aarch64-linux-gnu.so
./internal/_tagset.cpython-312-aarch64-linux-gnu.so
./internal/_encoding.cpython-312-aarch64-linux-gnu.so
./internal/native/_native.cpython-312-aarch64-linux-gnu.so
./internal/_threads.cpython-312-aarch64-linux-gnu.so
./vendor/psutil/_psutil_linux.cpython-312-aarch64-linux-gnu.so
./vendor/psutil/_psutil_posix.cpython-312-aarch64-linux-gnu.so

@taegyunkim

Copy link
Copy Markdown
Contributor

@KowalskiThomas Is this to facilitate internal testing?
I remember that @vlad-scherbich's plan is to move to src/native once native heap profiling instead of having separate rust module.

@KowalskiThomas

Copy link
Copy Markdown
Collaborator Author

@taegyunkim Yes that PR isn't intended to be merged (or even reviewed to be honest), but I needed a PR for all the required CI actions to trigger. This was debugged over >100 messages in #profiling-full-host-heap-project this morning 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants