Set --confcutdir for python tests

[oliverdain]

Describe the bug
Per pytest-dev/pytest#11261, as of at least pytest 8.5 (though StackOverflow reports this started with 7.4 and I've seen it with 8.3) pytest no longer just looks up the directory tree looking for conftest.py files. Instead it either walks up the directory tree to the level of --confcutdir and no higher or, if that isn't set it walks up to the first "project marker file" (e.g. a pytest.ini, pyproject.toml, etc.) and stops there. If --confcutdir isn't set and no project marker file is found it doesn't walk up the directory tree at all (presumably for security as that might allow anybody to add code outside the project that would get executed).

Currently Pants does not set --confcutdir so if users manually upgrade pytest their conftest.py files no longer get run.

Pants version
I'm using 2.28.0 but, AFAIK, all version are affected.

OS
Encountered on Linux but, per above, I think all versions are affected.

Additional info
there's a long Slack thread about this here: https://pantsbuild.slack.com/archives/C046T6T9U/p1760658668479529

At the end of that thread, @benjyw suggested:

And if you feel like tackling it, it would be pretty straightforward, to update pytest_args in src/python/pants/backend/python/goals/pytest_runner.py
--confcutdir=./src should do it

[benjyw]

Hmm, so if you have a pytest.ini/pyproject.toml that should be included in the sandbox (as long as [pytest].config_discovery is set to True, which is the default). So I'm curious why this is not happening in your case?

[benjyw]

It feels like should be the solution, not an ad-hoc special casing of confcutdir but instead have the same behavior with or without pants?

[oliverdain]

Oh - I didn't have a pytest.ini or a pyproject.toml before - just a pants.toml. I added a pyproject.ini specifically to work around this problem and, since I didn't know about config_discovery I manually mapped it in:

files(
   name = 'pytest_marker',
   sources = ['pytest.ini']
)

python_test_utils(
   name='conftest',
   sources=['conftest.py'],
   dependencies=[':pytest_marker'],
)

which worked. But it sounds like I don't need the files and dependencies bit as that should just work out of the box.

[benjyw]

I guess the tricky part for users is that Pants upgraded its default pytest version, so they are getting new behavior unexpectedly (unless they read in the release notes that we upgraded the default, and understood the consequences). So I can see the argument for hacking around it. But I still think it's better not to if there is another solution.

[oliverdain]

is that Pants upgraded its default pytest version

FWIW, the version that Pants uses by default does not have this behavior. I encountered this when I manually upgraded pytest as the older versions spit out tons of extraneous messages with Python >= 3.12 (see pytest-dev/pytest#10894). But, since pytest has that issue you probably will have to upgrade the version that ships with Pants sooner or later.

unless they read in the release notes that we upgraded the default, and understood the consequences

Yeah, it's just unexpected behavior and not easy to debug - it took me a couple of hours of hunting to figure out what was going on. Perhaps you could detect if there's a conftest.py but no project marker file and log a warning for the user? If nothing else, a bit of documentation might be helpful.

[benjyw]

Ah OK, so that actually reaffirms my thinking: you would have encountered this issue whether you were using Pants or not (although in the latter case it's more likely you would have had a pyproject.toml I suppose).

I like the warning idea though!