refactor(python): pair live-heap scenario to 3.14/3.15 migration gate

Replace the standalone python_live_heap_3.13 scenario with a paired
python_live_heap_3.14 (baseline) + python_live_heap_3.15 (candidate) so
live-heap actually tests the 3.14->3.15 migration delta like the other four
families. Both dirs are wheel-only (persistent live-heap is unreleased), so
they build against DDTRACE_INSTALL_URL and are excluded from main CI until the
feature ships.

Update the downstream gate to 10 scenarios: add the live-heap row + coverage
entry, extend the gate regexp to include live_heap, clarify wheel-install
rules, and note floor-version (oldest-supported) coverage as future work.
Drop the now-unused base_images/Dockerfile.python-3.13 and gitignore .vscode.

Author: vlad-scherbich

.gitignore

@@ -6,6 +6,7 @@ data/*
 __pycache__/
 
 .idea
+.vscode
 gems.locked
 
 .DS_Store

README.md

@@ -55,7 +55,7 @@ Both use [dd-octo-sts](https://github.com/DataDog/dd-octo-sts-action) (`dd-trace
 **Inputs:**
 
 - `dd_trace_py_commit_sha` — commit to test (required)
-- `test_scenarios` — regexp passed to `TEST_SCENARIOS` (dd-trace-py passes the [8-scenario 3.14/3.15 migration gate](scenarios/python_downstream_gate/README.md); the downstream workflow default alone is `python.*`)
+- `test_scenarios` — regexp passed to `TEST_SCENARIOS` (dd-trace-py passes the [10-scenario 3.14/3.15 migration gate](scenarios/python_downstream_gate/README.md); the downstream workflow default alone is `python.*`)
 
 ## Creating new tests 
 

scenarios/python_downstream_gate/README.md

@@ -1,6 +1,6 @@
 # Python downstream gate (dd-trace-py)
 
-Eight prof-correctness scenarios exercise the profiling stack on **3.14
+Ten prof-correctness scenarios exercise the profiling stack on **3.14
 (baseline)** and **3.15 (candidate)** for the same workloads. They are the
 default set when dd-trace-py triggers downstream CI on profiling changes.
 
@@ -12,11 +12,12 @@ default set when dd-trace-py triggers downstream CI on profiling changes.
 | exceptions | `python_exceptions_3.14` | `python_exceptions_3.15` |
 | async-gen | `python_async_gen_3.14` | `python_async_gen_3.15` |
 | lock | `python_lock_3.14` | `python_lock_3.15` |
+| live-heap | `python_live_heap_3.14` | `python_live_heap_3.15` |
 
 ## Profiler coverage
 
 Each family asserts one **profile type** end-to-end (collector + export + pprof
-shape). Together they exercise four collectors on the 3.14 → 3.15 path — **not**
+shape). Together they exercise five collectors on the 3.14 → 3.15 path — **not**
 the full Python profiling matrix.
 
 | Family | Profile type asserted | Collectors / setup | Not exercised in this scenario |
@@ -25,44 +26,58 @@ the full Python profiling matrix.
 | exceptions | `exception-samples` | **Exception** profiler (`DD_PROFILING_EXCEPTION_ENABLED`) | Other collectors at default/disabled |
 | async-gen | `wall-time` | Full profiler via `ddtrace-run` (`DD_PROFILING_ENABLED`); asyncio async-generator workload | Explicit CPU-only assertion; lock/heap/exception not targeted |
 | lock | `lock-acquire` | **Lock** profiler (`DD_PROFILING_LOCK_ENABLED`); threaded lock churn | Stack, heap, exceptions |
+| live-heap | `heap-space` + `heap-live-samples` | **Persistent live-heap** snapshot (`DD_PROFILING_HEAP_ENABLED`); OBJ-domain `bytes`; stack/lock off | CPU stack, wall-time, lock, exceptions, alloc-space |
 
 **Not covered by this gate** (other prof-correctness scenarios or dd-trace-py riot
-tests): CPU/stack as the primary signal, heap-live samples, PyTorch, gevent/uwsgi
-integration shapes, Flask/FastAPI HTTP workloads, native alloc, deep stack, GIL
-contention, etc. Expand the downstream regexp to `python.*` when ready for
-broader E2E coverage.
+tests): CPU/stack as the primary signal, PyTorch, gevent/uwsgi integration
+shapes, Flask/FastAPI HTTP workloads, native alloc, deep stack, GIL contention,
+etc. Expand the downstream regexp to `python.*` when ready for broader E2E
+coverage.
 
 ## Default downstream regexp
 
 GitLab and GitHub triggers in dd-trace-py both pass the same regexp (not the
 downstream workflow default of `python.*`):
 
 ```
-python_(mem_domain|exceptions|async_gen|lock)_3\.(14|15)
+python_(mem_domain|exceptions|async_gen|lock|live_heap)_3\.(14|15)
 ```
 
 Override via `workflow_dispatch` → `test_scenarios`, or when triggering
 `downstream-python.yml` manually.
 
-## Wheel install (3.15)
+## Wheel install
 
-3.15 scenarios require a **dd-trace-py wheel** at image build time:
+Every scenario builds against a **dd-trace-py wheel** via `DDTRACE_INSTALL_URL`
+(as `downstream-python.yml` does:
+`https://dd-trace-py-builds.s3.amazonaws.com/<sha>/install.sh`), pre-installed in
+the base image. Two cases need it unconditionally:
 
-- Set `DDTRACE_INSTALL_URL` when running tests (as `downstream-python.yml` does:
-  `https://dd-trace-py-builds.s3.amazonaws.com/<sha>/install.sh`).
-- Do **not** `pip install ddtrace` from PyPI in the scenario Dockerfile — 3.15
-  wheels may not be published yet.
+- **All 3.15 folders** — PyPI wheels may not be published for 3.15 yet.
+- **Both live-heap folders (3.14 and 3.15)** — the persistent live-heap profile
+  is recent and not yet in a published release.
 
-3.15 folders are excluded from prof-correctness `main` CI until 3.15 wheels are
-generally available; they run via the dd-trace-py downstream gate.
+These folders are excluded from prof-correctness `main` CI (which uses PyPI
+ddtrace) until the relevant wheels are generally available; they run via the
+dd-trace-py downstream gate. The other 3.14 folders also run on `main` CI.
 
 ## Local run
 
 ```sh
 export DDTRACE_INSTALL_URL="https://dd-trace-py-builds.s3.amazonaws.com/<commit-sha>/install.sh"
-TEST_SCENARIOS='python_(mem_domain|exceptions|async_gen|lock)_3\.(14|15)' go test -v -run TestScenarios
+TEST_SCENARIOS='python_(mem_domain|exceptions|async_gen|lock|live_heap)_3\.(14|15)' go test -v -run TestScenarios
 ```
 
+## Future work
+
+This gate tests the **migration delta** (3.14 → 3.15). A separate axis worth
+adding is **floor coverage**: run the same workloads on the oldest supported
+interpreter (currently 3.9; 3.10 once 3.15 drops 3.9) to catch regressions for
+users who jump directly from an old version to the newest — e.g. 3.9 → 3.15.
+Note prof-correctness only ships base images for 3.10–3.15 today, so a true 3.9
+floor scenario would need a new base image. This is intentionally out of scope
+for the 3.14/3.15 delta gate.
+
 ## Further reading
 
 - Full migration playbook (unit / E2E / staging A/B):

scenarios/python_live_heap_3.14/Dockerfile

@@ -0,0 +1,25 @@
+ARG BASE_IMAGE="prof-python-3.14"
+FROM $BASE_IMAGE
+
+# ddtrace is pre-installed in the base image when DDTRACE_INSTALL_URL is set
+# (dd-trace-py downstream CI). Do not pip-install from PyPI here: the persistent
+# live-heap profile is recent and may not be in a published release yet, so this
+# scenario runs against the just-built wheel until it ships.
+COPY ./scenarios/python_live_heap_3.14/main.py /app/main.py
+WORKDIR /app
+
+ENV EXECUTION_TIME_SEC="15"
+# Enable the live-heap profiler and make the snapshot the deliverable.
+ENV DD_PROFILING_MEMORY_ENABLED=true
+ENV DD_PROFILING_HEAP_ENABLED=true
+# Sample interval == object size -> ~one sample per object -> stable
+# per-stack proportions for both heap-space and heap-live-samples.
+ENV DD_PROFILING_HEAP_SAMPLE_SIZE=16384
+# Export several snapshots during the run so the persistent live set is
+# exercised across upload intervals (not just at shutdown).
+ENV DD_PROFILING_UPLOAD_INTERVAL=5
+# Isolate the heap profiler so other collectors don't add noise.
+ENV DD_PROFILING_STACK_ENABLED=false
+ENV DD_PROFILING_LOCK_ENABLED=false
+
+CMD python main.py

scenarios/python_live_heap_3.13/README.md scenarios/python_live_heap_3.14/README.mdscenarios/python_live_heap_3.13/README.md renamed to scenarios/python_live_heap_3.14/README.md

@@ -1,7 +1,9 @@
-## Live-Heap Profiling
+## Live-Heap Profiling (3.14 baseline)
 
 Validates that the Datadog Python profiler's **persistent live-heap profile**
-correctly reports the set of live (still-allocated) sampled objects.
+correctly reports the set of live (still-allocated) sampled objects. This is the
+**3.14 baseline** half of the `3.14 -> 3.15` migration pair (see
+`python_live_heap_3.15`).
 
 Unlike `alloc-space`/`alloc-samples` (which count every allocation over the
 interval), the live-heap profile is a running snapshot of what is *currently
@@ -21,11 +23,11 @@ differ only in count:
 Equal sizes mean the 80/20 split holds for both metrics the live-heap profile
 reports: `heap-space` (live bytes = count x size) and `heap-live-samples` (live
 object count). It uses `bytes` (`PyObject_Malloc`, the OBJ allocator domain) so
-the heap profiler tracks the objects identically on Python 3.12 and 3.13,
-independent of the `DD_PROFILING_MEMORY_MEM_DOMAIN_ENABLED` toggle (`bytearray`
-would be OBJ on 3.12 but MEM on 3.13). The objects are held alive while the
-process idles through several upload intervals (`DD_PROFILING_UPLOAD_INTERVAL=5`),
-so each exported heap snapshot contains the full live set.
+the heap profiler tracks the objects identically across versions, independent of
+the `DD_PROFILING_MEMORY_MEM_DOMAIN_ENABLED` toggle (`bytearray` moved OBJ -> MEM
+in 3.13). The objects are held alive while the process idles through several
+upload intervals (`DD_PROFILING_UPLOAD_INTERVAL=5`), so each exported heap
+snapshot contains the full live set.
 
 ## Expected Profile
 
@@ -43,6 +45,7 @@ before the live set is fully built.
 
 ## Notes
 
-The persistent live-heap profile is recent; run against a ddtrace build that
-includes it (e.g. via `DDTRACE_INSTALL_URL` pointing at a dd-trace-py S3 wheel)
-until it ships in a release.
+The persistent live-heap profile is recent, so this scenario runs against a
+ddtrace build that includes it (via `DDTRACE_INSTALL_URL` pointing at a
+dd-trace-py S3 wheel) rather than a PyPI release. It is excluded from
+prof-correctness `main` CI until the feature ships in a release.

…hon_live_heap_3.13/expected_profile.json …hon_live_heap_3.14/expected_profile.jsonscenarios/python_live_heap_3.13/expected_profile.json renamed to scenarios/python_live_heap_3.14/expected_profile.json scenarios/python_live_heap_3.13/expected_profile.json renamed to scenarios/python_live_heap_3.14/expected_profile.json

@@ -14,8 +14,9 @@
 #   - heap-space        (live bytes)  -> count * size  -> 80/20
 #   - heap-live-samples (live objects) -> count        -> 80/20
 # We use ``bytes`` (PyObject_Malloc / OBJ domain), which the heap profiler
-# tracks identically on 3.12 and 3.13, independent of the MEM-domain toggle
-# (``bytearray`` would be OBJ on 3.12 but MEM on 3.13).
+# tracks identically across Python versions and independent of the MEM-domain
+# toggle (``bytearray`` moved OBJ -> MEM in 3.13). This keeps the scenario a
+# clean live-heap check across the 3.14 -> 3.15 migration.
 OBJ_SIZE = 16384  # 16 KiB, well above the pymalloc small-object threshold
 N_MAJOR = 1600  # ~80% of the live set  (1600 * 16 KiB ~= 25 MiB)
 N_MINOR = 400  # ~20% of the live set  ( 400 * 16 KiB ~=  6 MiB)

scenarios/python_live_heap_3.13/main.py scenarios/python_live_heap_3.14/main.pyscenarios/python_live_heap_3.13/main.py renamed to scenarios/python_live_heap_3.14/main.py

@@ -1,16 +1,12 @@
-ARG BASE_IMAGE="prof-python-3.13"
+ARG BASE_IMAGE="prof-python-3.15"
 FROM $BASE_IMAGE
 
-# Copy the Python target into the container
-COPY ./scenarios/python_live_heap_3.13/main.py \
-    ./scenarios/python_live_heap_3.13/requirements.txt \
-    /app/
-RUN chmod 644 /app/*
-
+# ddtrace is pre-installed in the base image when DDTRACE_INSTALL_URL is set
+# (dd-trace-py downstream CI). Do not pip-install here — PyPI wheels may not
+# exist for 3.15 yet, and the persistent live-heap profile is recent.
+COPY ./scenarios/python_live_heap_3.15/main.py /app/main.py
 WORKDIR /app
 
-RUN pip install --no-cache-dir -r requirements.txt
-
 ENV EXECUTION_TIME_SEC="15"
 # Enable the live-heap profiler and make the snapshot the deliverable.
 ENV DD_PROFILING_MEMORY_ENABLED=true
@@ -25,5 +21,4 @@ ENV DD_PROFILING_UPLOAD_INTERVAL=5
 ENV DD_PROFILING_STACK_ENABLED=false
 ENV DD_PROFILING_LOCK_ENABLED=false
 
-# Run the program when the container starts
 CMD python main.py

…s/python_live_heap_3.13/requirements.txt …s/python_live_heap_3.14/requirements.txtscenarios/python_live_heap_3.13/requirements.txt renamed to scenarios/python_live_heap_3.14/requirements.txt scenarios/python_live_heap_3.13/requirements.txt renamed to scenarios/python_live_heap_3.14/requirements.txt

@@ -0,0 +1,52 @@
+## Live-Heap Profiling (3.15 candidate)
+
+Validates that the Datadog Python profiler's **persistent live-heap profile**
+correctly reports the set of live (still-allocated) sampled objects. This is the
+**3.15 candidate** half of the `3.14 -> 3.15` migration pair (see
+`python_live_heap_3.14`).
+
+Unlike `alloc-space`/`alloc-samples` (which count every allocation over the
+interval), the live-heap profile is a running snapshot of what is *currently
+live*: allocations are added when sampled and subtracted when freed, and the
+snapshot is exported non-destructively on every upload. It is attached as a
+second pprof (`<prefix>.<pid>.<seq>.heap.pprof`) alongside the primary profile.
+
+## Test Application
+
+`main.py` retains a known live set for the whole run, split into two
+distinctly-named call sites that allocate **equal-size** objects (16 KiB) and
+differ only in count:
+
+- `retain_major` - 1,600 objects (~25 MiB, ~80% of the live set)
+- `retain_minor` -   400 objects (~6 MiB, ~20% of the live set)
+
+Equal sizes mean the 80/20 split holds for both metrics the live-heap profile
+reports: `heap-space` (live bytes = count x size) and `heap-live-samples` (live
+object count). It uses `bytes` (`PyObject_Malloc`, the OBJ allocator domain) so
+the heap profiler tracks the objects identically across versions, independent of
+the `DD_PROFILING_MEMORY_MEM_DOMAIN_ENABLED` toggle (`bytearray` moved OBJ -> MEM
+in 3.13). The objects are held alive while the process idles through several
+upload intervals (`DD_PROFILING_UPLOAD_INTERVAL=5`), so each exported heap
+snapshot contains the full live set.
+
+## Expected Profile
+
+Assertions run against the heap snapshot pprof only (selected with
+`pprof-regex`). `DD_PROFILING_HEAP_SAMPLE_SIZE=16384` (== object size) yields
+roughly one sample per object, keeping the per-stack proportions stable.
+
+- `heap-space` (live bytes):
+  - `^<module>;Target.run;Target.retain_major$` ~= 80%
+  - `^<module>;Target.run;Target.retain_minor$` ~= 20%
+- `heap-live-samples` (live object count): same ~80/20 split.
+
+`allow_first_profile_failure` tolerates the first snapshot, which may be taken
+before the live set is fully built.
+
+## Notes
+
+Requires a **dd-trace-py wheel** at image build time (set `DDTRACE_INSTALL_URL`,
+as `downstream-python.yml` does). Do **not** `pip install ddtrace` from PyPI —
+3.15 wheels may not be published yet. Excluded from prof-correctness `main` CI
+until 3.15 wheels are generally available; runs via the dd-trace-py downstream
+gate.