Reference counting related regression in Python 3.14a7?

[wjakob]

Bug report

Bug description:

Dear Python team,

I am the lead of the nanobind project, which is a C++<->Python binding tool. I always try to follow recent Python versions to react to any API/ABI changes and catch issues before they end up in public releases. With the just-released Python 3.14a7, a whole bunch of testcases in the nanobind test suite start to fail (example: https://github.com/wjakob/nanobind/actions/runs/14372986244/job/40299387939). These are all tests that create and pass objects and expect reference counts to behave in a certain way. This works consistently for PyPy and all tested CPython version from 3.8 all the way until 3.14a6, and so it is therefore surprising to see such a change in behavior in an alpha version bump.

I'm wondering what could cause this? Are there known issues in 3.14a7? Did reference counting / garbage collection change in some way that could cause such behavior to arise?

Thank you,
Wenzel

CPython versions tested on:

3.14

Operating systems tested on:

macOS, Linux

[srinivasreddy]

Could you please post a simple reproducible test case here ?

[wjakob]

The issue occurs in the test suite nanobind, which is a C++ project. From my experience, CPython core developers don't accept that and want a pure C API reproducer. Converting this change into such a reproducer will be a significant undertaking. I opened this ticket to already give a heads-up that something in the behavior of Python 3.14a7 changed, and to ask if making such a repro would be a waste of time because the issue is perhaps already known. Meanwhile, I am also trying to bisect this to a specific commit..

[wjakob]

I bisected this change to commit 053c285 (which should be considered joint with cd69d55 that occurs two commits down in the tree -- the interpreter is not functional without that second change.)

(CC @mpage, @markshannon)

How to reproduce (this works with 3.14a6, breaks with 3.14a7)

$ git clone --recursive https://github.com/wjakob/nanobind
$ cd nanobind
$ cmake .
$ make
$ pytest

[ZeroIntensity]

Yeah, reference counts aren't considered stable, because we add all sorts of optimizations to skip refcounting. There's a discussion from a few months ago about relying on reference counts in downstream C API tests; basically, don't do it.

From the Py_REFCNT docs as well:

Note that the returned value may not actually reflect how many references to the object are actually held. For example, some objects are immortal and have a very high refcount that does not reflect the actual number of references. Consequently, do not rely on the returned value to be accurate, other than a value of 0 or 1.

We should probably note this in the glossary too. (Edit: #132352)

[mpage]

Hi Wenzel, 053c285 is intended to reduce the number of reference counting operations that are performed on objects that are pushed / popped from the interpreter's operand stack, so it could produce the change in behavior that you're seeing. For example, the following code will print 1 with the change and 2 without it:

import sys


def test():
    l = [1, 2, 3]
    # The frame holds a reference to l
    #
    # Prior to 053c285 the interpreter would increment the refcount
    # on l when it pushed l onto the stack. Consequently, the following
    # line prints `2`.
    #
    # After 053c285 the interpreter does not increment the refcount
    # on l when it pushes it onto the stack. Consequently, the following
    # line prints `1`.
    print(sys.getrefcount(l))


if __name__ == "__main__":
    test()

Looking at the output from the failed test run, it looks like many of the failing tests are expecting the incref that was previously performed by the interpreter when it pushed an argument onto the stack for the call to sys.getrefcount.

[wjakob]

Thank you for the clarifications @mpage and @ZeroIntensity. For C extension projects, it can be quite important to inspect reference counts in the test suite. Reference counting bugs can and have crept in in the past, with obviously catastrophic results when they occur in core parts of a binding framework. So I think the right answer is not "don't rely on Py_REFCNT" which seems too dogmatic, but rather to use it in a way that still allows us to catch regressions, e.g. by measuring the relative change of reference counts done by the specific project or (worst case) specializing the tests to Python minor versions.

It's exciting that removal of reference counting from the stack has such a good impact on perf @mpage -- nice work! I will close this issue since it it is not a bug, and I can adapt my tests with this information.

[ZeroIntensity]

measuring the relative change of reference counts done by the specific project or (worst case) specializing the tests to Python minor versions.

FWIW, I'd go with the latter. Sometimes you might not see any change, such as when the object is immortal. (Hint: you can identify those objects with PyUnstable_IsImmortal in 3.14.)