Skip to content

Use runtime offsets for interpreter state access#861

Draft
jonashaag wants to merge 1 commit into
benfred:masterfrom
jonashaag:use-runtime-interpreter-offsets
Draft

Use runtime offsets for interpreter state access#861
jonashaag wants to merge 1 commit into
benfred:masterfrom
jonashaag:use-runtime-interpreter-offsets

Conversation

@jonashaag

Copy link
Copy Markdown

Summary

  • read Python 3.13+ interpreter layout metadata from _Py_DebugOffsets
  • retain and use the runtime threads_head, imports_modules, and ceval_gil offsets for interpreter validation, sampling, thread-name lookup, and GIL lookup
  • locate _PyRuntime through either its symbol or the binary's PyRuntime section, retaining the existing generated-binding fallback when runtime metadata is unavailable
  • preserve the existing public low-level Rust function signatures

Closes #852.

Motivation

The generated Python 3.14 bindings assume PyInterpreterState.imports.modules is at offset 7664. A Python 3.14.6 runtime reports offset 7688 through _Py_DebugOffsets. Reading the generated offset produces an invalid sys.modules pointer and EFAULT.

Besides losing thread names, this is expensive because the empty thread-name cache causes py-spy to retry walking remote sys.modules on every sample. In an observed 100 Hz workload, affected 100-sample windows spent 3.2–4.3 seconds in thread-name lookup while frame, code, filename, and line-table reads took about 90 ms. Using the runtime offset resolved the thread name once, produced about 1,493 subsequent cache hits, and removed sampling-lag warnings.

CPython places _Py_DebugOffsets first in _PyRuntime specifically so out-of-process debuggers can locate fields without relying on a build-specific struct layout. py-spy already used runtime_state.interpreters_head from this table. This change retains the interpreter-state offsets it also needs and uses them consistently.

This is narrower than the full offset-based stack/object reader proposed in #831: frame and object access remains on the existing trait-based path. Unlike #855, it does not add another generated binding for each affected Python patch/build layout.

Validation

  • pre-commit run --all-files
  • cargo test --release with Python 3.13
  • cargo test --lib
  • cargo check --all-targets
  • cargo check --lib --no-default-features

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failed to get thread names on python v3.14.5

1 participant