Skip to content

Use pystats for free threading performance statistics #131253

Open
@colesbury

Description

@colesbury

Feature or enhancement

It'd be helpful to have a counter for lock contention (the number of times a thread blocks on a lock acquisition), especially in the free threading builds. We already have infrastructure to record Python statistics in pystats.h. It'd be great to extend this.

I think there are a few steps needed to support this:

  1. pystats should be thread-safe in the free threading build and not inhibit multithreaded scaling. In practice, I think this means that stats should be recorded per-thread and aggregated when a thread exits (and possibly aggregated more often as well).

  2. Some stats are cheap to collect and widely useful, including the above mentioned lock contention and some GC stats. We should consider recording some stats even when Python isn't built with --enable-pystats so that they are available in release builds, while being careful not to regress performance.

  3. Consider providing a Python API for retrieving stats in addition to the existing behavior of dumping stat counters to a file. We don't have to pursue this immediately, but I think it'd be useful in the same ways that gc.get_stats() is useful.

Free threading stats:

  • Lock contention counter (number of times a thread blocks on a lock acquisition)
  • Stop the world counter (and possibly total duration)

cc @nascheme @zooba @mdboom

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions