Skip to content

fix(python): update profile expectations - #182

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
taegyunkim/fix-python-profile-expectations
Aug 14, 2026
Merged

fix(python): update profile expectations#182
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
taegyunkim/fix-python-profile-expectations

Conversation

@taegyunkim

@taegyunkim taegyunkim commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • update Python stack expectations for module-qualified time.sleep and gevent frames
  • disable tracing-only background services in the Python profiler test base image

Fixes the failures from https://github.com/DataDog/prof-correctness/actions/runs/31654920913.

Why

These scenarios install unpinned dependencies. Run 31496036887 passed on the same prof-correctness commit shortly before dd-trace-py v4.13.0 was released, while the next scheduled runs pulled the new release and exposed two issues.

  • Native frame names: dd-trace-py #19069, shipped in v4.13.0, changed native-frame display names from the bare function name to <module>.<function>. The observed stacks therefore changed from sleep to time.sleep and from loop.run to gevent.libev.corecext.loop.run; the workload and expected durations did not change.
  • Unrelated background threads: wall-time percentages use every profiled thread as the denominator. Code Origin's SignalUploader and the remote-config poller can appear as long-lived Dummy-1 threads. Greenlet's two branches consequently varied from 29-33% when a background thread was present to 48-49% when it was not. There is no single correct percentage for that unstable denominator.

The Python base image already disables tracing because these scenarios test the profiler. This change also disables Code Origin and remote configuration, removing their unrelated background threads. The original duration-independent percentage targets and error margins remain unchanged.

With dd-trace-py v4.13.0 and both background services disabled locally, greenlet's branches measured 47% and 49% against the original 50% ± 15% expectations. uWSGI measured 47% for the requester and 45% for the worker against the original 50% ± 20% and 42% ± 10% expectations.

The other scenarios named in the Slack messages passed. They were only grouped into the same failed matrix shards.

Testing

  • ran the greenlet and uWSGI containers locally with dd-trace-py v4.13.0 and the updated base environment, then analyzed their generated profiles against the original relative expectations
  • replayed the stack-name expectations against artifacts from run 31654920913
  • go test ./analysis/... ./cmd/list-scenarios/... ./cmd/prof-analyze/...
  • go test -run '^TestSchemaValidation$' -count=1

Comment thread scenarios/python_greenlet_3.12/expected_profile.json Outdated
@taegyunkim
taegyunkim marked this pull request as ready for review August 13, 2026 14:30
@taegyunkim
taegyunkim requested a review from a team as a code owner August 13, 2026 14:30
@taegyunkim
taegyunkim force-pushed the taegyunkim/fix-python-profile-expectations branch 3 times, most recently from eb208da to 1f8f9e1 Compare August 13, 2026 15:01

@taegyunkim taegyunkim left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KowalskiThomas Just turned off code origins and remote config which simplified things a bit

@KowalskiThomas KowalskiThomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find with the other threads -- note sure why we had to disable CO for Spans for that to work (RC makes sense, CO for Spans less so), but in any case LGTM.

@taegyunkim

Copy link
Copy Markdown
Contributor Author

/code blockers

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Aug 14, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-08-14 14:13:52 UTC ℹ️ Start processing command /code blockers


2026-08-14 14:13:53 UTC ℹ️ Devflow:

Checking merge blockers for #182...


2026-08-14 14:13:59 UTC ℹ️ Devflow: /code blockers

Detected 1 merge blocker(s) to address:

🔴 Blocking

  • This pull request contains unsigned commits.
    Commit 1f8f9e1 is not signed.
    Hint: Sign your commits, then force-push. See our FAQ.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants