(EDIT: my finger slipped and submitted the issue before I'm finished typing. A moment please before I fix it up incrementally...) Done as of UTC 2025-04-14 09:16+00:00

Synopsis

We're vying over the control for sys.settrace() with coverage.py,¹ which results in coverage data ceasing being collected from each Python process once the first LineProfiler instance has been .enable()-ed. There are multiple strategies we can employ to mitigate this, each with their advantages and drawbacks.

The current state

One thing that has bugged devs for the repo for a long time is that coverage.py is behaving unexpectedly, under-reporting on test coverage and neglecting code paths that has clearly been executed. This has the side effect of cluttering codecov output and PR diffs, since they are polluted with false alarms – to the point that coverage reports may cause more confusion and annoyance² than they offer insights.

While that is unfortunately the current state of affairs, I still believe that coverage.py is invaluable as a QA tool and we can maybe try harder to fix it. So I took a deeper look and...

The reason

• Every time we call LineProfiler.enable(), we PyEval_SetTrace() with a C-level Py_tracefunc and the profiler object (_line_profiler.pyx (L338))
• Every time we call LineProfiler.disable(), we completely purge the tracing facilities by nulling the pointers to both the Py_tracefunc and the tracing object (unset_trace.c (L6)).

Hence whenever we run an in-process test which uses a LineProfiler, coverage.py only sees up to the point that the first LineProfiler is enabled, and the coverage-tracing function isn't restored even after the profiler has been disabled. The fundamental issue here is that tracing tools all have to use the same sys.settrace(), and Python doesn't natively provide for a way for tools to work cooperatively.

How to mitigate?

Naïve Python implementation

line_profiler/line_profiler.py::LineProfiler.enable() can sys.gettrace() to get the current tracer, and then .disable() can sys.settrace() and put it back.

PROS

It's simple and elegant. It also works on the Python level, so no need to go spelunking in _line_profiler.pyx.

CONS

• It simply DOESN'T WORK in practice, since it only "works" for tracers set in Python-space and not in C-space. Specifically:
  • If a tracer is set on the C-level (PyEval_SetTrace(Py_tracefunc func, PyObject *obj)), sys.gettrace() would only retrieve obj and silently drop all info related to func.
  • And then when one proceeds to sys.settrace() to "restore the previous tracer", sys.settrace() supplies a default Py_tracefunc (Python/sysmodule.c::trace_trampoline() (L1101)) which essentially just calls the obj.³
• Coverage-tracing is disabled as long as the profiler is active.

C implementation

line_profiler/_line_profiler.pyx::LineProfiler.enable() can retrieve references to both the Py_tracefunc and the tracer object, stash them somewhere, and restore them in .disable().

PROS

It's more robust, working for both pure-Python and C-level tracers.

CONS

• There is no public C API for retrieving the Py_tracefunc: we'll have to hack into the thread state with PyThreadState_Get() and get the non-public member ->c_tracefunc (see Python/legacy_tracing.c::setup_tracing() (L585); meanwhile, sys.gettrace() retrieves the ->c_traceobj).
• Coverage-tracing is still disabled as long as the profiler is active.

C-wrapper implementation

• line_profiler/_line_profiler.pyx::LineProfiler.enable() can retrieve references to both the Py_tracefunc callback and the tracer object, and stash them somewhere.
• line_profiler/_line_profiler.pyx::python_trace_callback() then retrieves said references from the LineProfiler object, and calls them on exit.
• The old tracer object and callback are to be restored in LineProfiler.disable().

PROS

It allows for interoperability between LineProfiler and other monitoring toolings.

CONS

• It's the most complex of the three solutions.
• Such wrapping of other tracers may not be idiomatic usage and may cause unforeseen issues.