You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Replaces Riot and the temporary Hatch canary with direct uv test environments while preserving the useful build-once design.
Deletes riotfile.py, .riot, Riot/Hatch runtime dependencies, and the transition contract.
Preserves 1,936 ordered environment instances, 1,886 stable IDs, and all 193 named selectors used by local development and CI.
Stores shared dependencies and variables in tests/environments/core.json, reusable profiles in definitions/, and resolved instances in Windows-safe named-node shards.
Builds ddtrace once per Python version, installs each checked-in lock into a reusable uv dependency prefix, and launches every resolved command in a fresh subprocess.
Routes local tests, generated GitLab jobs, docs, lock maintenance, dependency checks, and integration-registry tooling through uv.
Removes the nested extension-cache venv and runs that tool as a uv script with declared build dependencies.
Keeps tracer and tracer-uwsgi selectors exclusive, removing five duplicate jobs (about 16.2 runner-minutes in the measured pipeline).
Riot reused editable base environments; it did not build or reuse dev wheels. The uv runner retains that reuse without a second environment manager.
Testing
Local validation on the final migration commit:
scripts/test_environments.py check: 1,936 instances, 193 named nodes, 1,886 unique environments.
Lock comparison: 1,885 moved lock bodies are unchanged. The integration-registry lock intentionally drops Riot and its transitive dependencies.
uv tooling and generator tests: 30 passed on Python 3.12.
Integration-registry tests: 4 passed on Python 3.13.
Exact CI extension-cache restore command: passed after adding the conditional patchelf build dependency.
Test-environment, suitespec, CI dependency, installability, and dependency-coverage checks: passed. Existing latest-range warnings remain.
Full scripts/lint checks: passed after the final Windows portability correction.
Performance evidence:
Experiment
Result
First Hatch canary, independent build
Failed after roughly 4, 20, and 14 minutes across attempts
Hatch reusing the Python 3.12 base
555 s
Equivalent Riot shard in the same pipeline
565 s
Local full config generation before in-process IDs
24.6 s
Local full config generation after
0.69 s
First local uv Python 3.12 editable build
46.65 s build + 0.16 s install
Warm base fingerprint check, including container startup
1.97 s
The 10-second Hatch difference is 1.8% and within single-run noise. Hatch added environment ownership and bootstrap failure modes without a measured runtime win, so direct uv is the simpler path.
Real CI checkpoint 0d13ea1a36 routed all generated tests through uv. Its configuration job passed; the monolithic JSON inventory hit the 100 KB file gate, and all six base jobs failed before consumers ran. This commit fixes the size gate by sharding around preserved names, resolves exact preinstalled interpreters, removes the nested extension-cache environment, and locally reproduces the replacement command. The next real run must validate the base-build correction and provide clean end-to-end timings.
Risks
This is a large mechanical configuration migration. Stable names, IDs, order, commands, dependency order, and variables are validated, but broad platform and service coverage still depends on the fresh CI run.
Existing pytest/xdist sessions can run multiple tests in one interpreter. This PR starts one process per environment command; it does not falsely claim one clean process per span-producing test.
Snapshot tests still require both a fresh Python process and isolated test-agent session. The enforcement migration is ranked as the top correctness item in the CI bottleneck document.
Additional Notes
docs/contributing-ci-performance.rst is the living, ranked register for isolation, duration-blind ordering, snapshot-agent cost, cache duplication, retries, subprocess paths, measurements, and next experiments.
This is internal test infrastructure with no customer-facing behavior change. The PR has the changelog/no-changelog label.
Found 0 performance improvements and 8 performance regressions! Performance is the same for 613 metrics, 10 unstable metrics.
Explanation
This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:
🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline
We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.
If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.
Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.
More details about the CI and significant changes
You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.
CIs of the difference of means are often centered around 0%, because often changes are not that big:
---------------------------------(------|---^--------)-------------------------------->
-0.6% 0% 0.3% +1.2%
| | |
lower bound of the CI --' | |
sample mean (center of the CI) -------------' |
upper bound of the CI ----------------------'
As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).
For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:
----------------------------------------|---------|---(---------^---------)---------->
0% 1% 1.3% 2.2% 3.1%
| | | |
significant impact threshold --------------' | | |
lower bound of CI --------------' | |
sample mean (center of the CI) --------------------------' |
upper bound of CI ----------------------------------'
scenario:httppropagationinject-ids_only
🟥 execution_time [+2.532µs; +2.699µs] or [+11.690%; +12.458%]
scenario:iastaspects-lstrip_aspect
🟥 execution_time [+70.676µs; +77.535µs] or [+24.634%; +27.024%]
scenario:iastaspects-translate_aspect
🟥 execution_time [+61.566µs; +68.173µs] or [+14.695%; +16.272%]
scenario:iastaspectsospath-ospathbasename_aspect
🟥 execution_time [+134.427µs; +141.640µs] or [+33.278%; +35.064%]
scenario:iastaspectssplit-rsplit_aspect
🟥 execution_time [+13.843µs; +18.774µs] or [+9.736%; +13.204%]
scenario:span-start
🟥 execution_time [+1.252ms; +1.409ms] or [+8.273%; +9.308%]
Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AI GeneratedLargely based on code generated by an AI or LLM. This label is the same across all dd-trace-* reposchangelog/no-changelogA changelog entry is not required for this PR.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mabdinur
added
changelog/no-changelog
Description
Replaces Riot and the temporary Hatch canary with direct uv test environments while preserving the useful build-once design.
riotfile.py,.riot, Riot/Hatch runtime dependencies, and the transition contract.tests/environments/core.json, reusable profiles indefinitions/, and resolved instances in Windows-safe named-node shards.tracerandtracer-uwsgiselectors exclusive, removing five duplicate jobs (about 16.2 runner-minutes in the measured pipeline).Riot reused editable base environments; it did not build or reuse dev wheels. The uv runner retains that reuse without a second environment manager.
Testing
Local validation on the final migration commit:
scripts/test_environments.py check: 1,936 instances, 193 named nodes, 1,886 unique environments.scripts/compile-test-environment-locks: validated 1,886 locks; compiled 0; removed 0.patchelfbuild dependency.scripts/lint checks: passed after the final Windows portability correction.Performance evidence:
The 10-second Hatch difference is 1.8% and within single-run noise. Hatch added environment ownership and bootstrap failure modes without a measured runtime win, so direct uv is the simpler path.
Real CI checkpoint
0d13ea1a36routed all generated tests through uv. Its configuration job passed; the monolithic JSON inventory hit the 100 KB file gate, and all six base jobs failed before consumers ran. This commit fixes the size gate by sharding around preserved names, resolves exact preinstalled interpreters, removes the nested extension-cache environment, and locally reproduces the replacement command. The next real run must validate the base-build correction and provide clean end-to-end timings.Risks
Additional Notes
docs/contributing-ci-performance.rstis the living, ranked register for isolation, duration-blind ordering, snapshot-agent cost, cache duplication, retries, subprocess paths, measurements, and next experiments.This is internal test infrastructure with no customer-facing behavior change. The PR has the
changelog/no-changeloglabel.