feat(python): add cpu-time gate scenarios for 3.14/3.15 migration (PROF-15511)

README.md

@@ -77,7 +77,7 @@ Both use [dd-octo-sts](https://github.com/DataDog/dd-octo-sts-action) (`dd-trace
 **Inputs:**
 
 - `dd_trace_py_commit_sha` — commit to test (required)
-- `test_scenarios` — regexp passed to `TEST_SCENARIOS` (dd-trace-py passes the [6-scenario 3.14/3.15 migration gate](scenarios/python_downstream_gate/README.md); the downstream workflow default alone is `python.*`)
+- `test_scenarios` — regexp passed to `TEST_SCENARIOS` (dd-trace-py passes the [8-scenario 3.14/3.15 migration gate](scenarios/python_downstream_gate/README.md); the downstream workflow default alone is `python.*`)
 
 ## Creating new tests 
 

scenarios/python_cpu_3.14/Dockerfile

@@ -0,0 +1,15 @@
+ARG BASE_IMAGE="prof-python-3.14"
+FROM $BASE_IMAGE
+
+COPY ./scenarios/python_cpu_3.14/requirements.txt /app/requirements.txt
+RUN pip install -r /app/requirements.txt
+
+COPY ./scenarios/python_cpu_3.14/main.py /app/main.py
+WORKDIR /app
+
+ENV EXECUTION_TIME_SEC=10
+ENV _DD_PROFILING_STACK_ADAPTIVE_SAMPLING_ENABLED=0
+ENV DD_PROFILING_ENABLED=true
+ENV DD_PROFILING_MEMORY_ENABLED=false
+
+CMD ddtrace-run python main.py

scenarios/python_cpu_3.14/README.md

@@ -0,0 +1,24 @@
+## CPU stack profiling (3.14 baseline)
+
+Validates that the Datadog Python profiler's **stack collector** reports
+`cpu-time` samples with correct call stacks and `thread name` labels under a
+CPU-bound workload. This is the **3.14 baseline** half of the `3.14 -> 3.15`
+migration pair (see `python_cpu_3.15`).
+
+Reuses the workload from `scenarios/python_cpu`: two tight loops (`a` and `b`)
+with a 2:1 relative CPU share. Memory profiling is disabled so the assertion
+targets stack samples only.
+
+## Expected profile
+
+- `cpu-time`:
+  - `<module>;.*main;.*b` ~= 66% (`MainThread`)
+  - `<module>;.*main;.*a` ~= 33% (`MainThread`)
+
+`cpu-time` is best-effort per sample, so `error_margin` is generous.
+`scale_by_duration` normalizes across run lengths.
+
+## Notes
+
+The 3.14 scenario runs on prof-correctness `main` CI (PyPI ddtrace). The 3.15
+candidate runs only via the dd-trace-py downstream gate (`DDTRACE_INSTALL_URL`).

scenarios/python_cpu_3.14/expected_profile.json

@@ -0,0 +1,37 @@
+{
+  "test_name": "python_cpu_3.14",
+  "stacks": [
+    {
+      "profile-type": "cpu-time",
+      "stack-content": [
+        {
+          "regular_expression": "<module>;.*main;.*b",
+          "percent": 99,
+          "error_margin": 0,
+          "labels": [
+            {
+              "key": "thread name",
+              "values": [
+                "MainThread"
+              ]
+            }
+          ]
+        },
+        {
+          "regular_expression": "<module>;.*main;.*a",
+          "percent": 33,
+          "error_margin": 100,
+          "labels": [
+            {
+              "key": "thread name",
+              "values": [
+                "MainThread"
+              ]
+            }
+          ]
+        }
+      ]
+    }
+  ],
+  "scale_by_duration": true
+}

scenarios/python_cpu_3.14/main.py

@@ -0,0 +1,30 @@
+import os
+from time import time
+
+
+class CPUBurner:
+    def __init__(self) -> None:
+        self.x = 0
+        self.i = 0
+
+    def a(self) -> None:
+        self.i = 0
+        while self.i < 1000000:
+            self.x += self.i
+            self.i += 1
+
+    def b(self) -> None:
+        self.i = 0
+        while self.i < 2000000:
+            self.x += self.i
+            self.i += 1
+
+    def main(self) -> None:
+        execution_time_sec = float(os.getenv("EXECUTION_TIME_SEC", "10"))
+        end = time() + execution_time_sec
+        while time() < end:
+            self.a()
+            self.b()
+
+
+CPUBurner().main()

scenarios/python_cpu_3.14/requirements.txt

@@ -0,0 +1 @@
+ddtrace

scenarios/python_cpu_3.15/Dockerfile

@@ -0,0 +1,16 @@
+ARG BASE_IMAGE="prof-python-3.15"
+FROM $BASE_IMAGE
+
+# ddtrace is pre-installed in the base image when DDTRACE_INSTALL_URL is set
+# (dd-trace-py downstream CI). Do not pip-install here — PyPI wheels may not
+# exist for 3.15 yet.
+
+COPY ./scenarios/python_cpu_3.15/main.py /app/main.py
+WORKDIR /app
+
+ENV EXECUTION_TIME_SEC=10
+ENV _DD_PROFILING_STACK_ADAPTIVE_SAMPLING_ENABLED=0
+ENV DD_PROFILING_ENABLED=true
+ENV DD_PROFILING_MEMORY_ENABLED=false
+
+CMD ddtrace-run python main.py

scenarios/python_cpu_3.15/README.md

@@ -0,0 +1,25 @@
+## CPU stack profiling (3.15 candidate)
+
+Validates that the Datadog Python profiler's **stack collector** reports
+`cpu-time` samples with correct call stacks and `thread name` labels under a
+CPU-bound workload. This is the **3.15 candidate** half of the `3.14 -> 3.15`
+migration pair (see `python_cpu_3.14`).
+
+Reuses the workload from `scenarios/python_cpu`: two tight loops (`a` and `b`)
+with a 2:1 relative CPU share. Memory profiling is disabled so the assertion
+targets stack samples only.
+
+## Expected profile
+
+- `cpu-time`:
+  - `<module>;.*main;.*b` ~= 66% (`MainThread`)
+  - `<module>;.*main;.*a` ~= 33% (`MainThread`)
+
+`cpu-time` is best-effort per sample, so `error_margin` is generous.
+`scale_by_duration` normalizes across run lengths.
+
+## Notes
+
+This scenario is wheel-only: PyPI ddtrace wheels may not exist for 3.15 yet, so
+it is excluded from prof-correctness `main` CI and runs via the dd-trace-py
+downstream gate (`DDTRACE_INSTALL_URL`).

scenarios/python_cpu_3.15/expected_profile.json

@@ -0,0 +1,37 @@
+{
+  "test_name": "python_cpu_3.15",
+  "stacks": [
+    {
+      "profile-type": "cpu-time",
+      "stack-content": [
+        {
+          "regular_expression": "<module>;.*main;.*b",
+          "percent": 66,
+          "error_margin": 100,
+          "labels": [
+            {
+              "key": "thread name",
+              "values": [
+                "MainThread"
+              ]
+            }
+          ]
+        },
+        {
+          "regular_expression": "<module>;.*main;.*a",
+          "percent": 33,
+          "error_margin": 100,
+          "labels": [
+            {
+              "key": "thread name",
+              "values": [
+                "MainThread"
+              ]
+            }
+          ]
+        }
+      ]
+    }
+  ],
+  "scale_by_duration": true
+}

scenarios/python_cpu_3.15/main.py

@@ -0,0 +1,30 @@
+import os
+from time import time
+
+
+class CPUBurner:
+    def __init__(self) -> None:
+        self.x = 0
+        self.i = 0
+
+    def a(self) -> None:
+        self.i = 0
+        while self.i < 1000000:
+            self.x += self.i
+            self.i += 1
+
+    def b(self) -> None:
+        self.i = 0
+        while self.i < 2000000:
+            self.x += self.i
+            self.i += 1
+
+    def main(self) -> None:
+        execution_time_sec = float(os.getenv("EXECUTION_TIME_SEC", "10"))
+        end = time() + execution_time_sec
+        while time() < end:
+            self.a()
+            self.b()
+
+
+CPUBurner().main()

scenarios/python_downstream_gate/README.md

@@ -1,13 +1,14 @@
 # Python downstream gate (dd-trace-py)
 
-Six prof-correctness scenarios exercise the profiling stack on **3.14
+Eight prof-correctness scenarios exercise the profiling stack on **3.14
 (baseline)** and **3.15 (candidate)** for the same workloads. They are the
 default set when dd-trace-py triggers downstream CI on profiling changes.
 
 ## Scenarios
 
 | Family | 3.14 (baseline) | 3.15 (candidate) |
 |--------|-------------------|------------------|
+| cpu (stack) | `python_cpu_3.14` | `python_cpu_3.15` |
 | exceptions | `python_exceptions_3.14` | `python_exceptions_3.15` |
 | async-gen | `python_async_gen_3.14` | `python_async_gen_3.15` |
 | lock | `python_lock_3.14` | `python_lock_3.15` |
@@ -16,17 +17,18 @@ default set when dd-trace-py triggers downstream CI on profiling changes.
 
 | Family | Profile type asserted | Collectors / setup |
 |--------|----------------------|--------------------|
+| cpu (stack) | `cpu-time` + `thread name` | Stack via `ddtrace-run`; memory off; CPU-bound loops |
 | exceptions | `exception-samples` + `exception type` | Exception profiler |
 | async-gen | `wall-time` | Full profiler via `ddtrace-run`; asyncio async-generator workload |
 | lock | `lock-acquire` + `lock-release` + `lock name` | Lock profiler; threaded lock churn |
 
-Feature-specific pairs (mem_domain, live_heap) and extended coverage (cpu, alloc,
+Feature-specific pairs (mem_domain, live_heap) and extended coverage (alloc,
 asyncio, …) land in follow-up PRs.
 
 ## Default downstream regexp
 
 ```
-python_(exceptions|async_gen|lock)_3\.(14|15)
+python_(cpu|exceptions|async_gen|lock)_3\.(14|15)
 ```
 
 Override via `workflow_dispatch` → `test_scenarios`, or when triggering
@@ -41,7 +43,7 @@ Override via `workflow_dispatch` → `test_scenarios`, or when triggering
 
 ```sh
 export DDTRACE_INSTALL_URL="https://dd-trace-py-builds.s3.amazonaws.com/<commit-sha>/install.sh"
-TEST_SCENARIOS='python_(exceptions|async_gen|lock)_3\.(14|15)' go test -v -run TestScenarios
+TEST_SCENARIOS='python_(cpu|exceptions|async_gen|lock)_3\.(14|15)' go test -v -run TestScenarios
 ```
 
 ## Further reading