support conftest/ folders

[ofer-pd]

What's the problem this feature will solve?

I'm tired of putting so much code in a single tests/conftest.py file.

Describe the solution you'd like

It'd be nice if I could expand conftest.py into a folder, like this:

tests/conftest/__init__.py    # loaded instead of conftest.py; imports from sibling modules
tests/conftest/fixtures.py
tests/conftest/setup.py

Alternative Solutions

I could create a different folder, like tests/lib/, add code there, and import that code from tests/conftest.py, but I'd prefer to keep the rest of the tests/ namespace free. For example, the way I organize my tests is identical to how I organize my code. If I have a source file lib/foo.py, I'll create tests for it in tests/lib/test__foo.py. If I used tests/lib/ to instead store setup code for the test suite, that would conflict with the above convention.

[RonnyPfannschmidt]

There should be better support for including modules in toplevel contests

They shouldn't be packages themselves to limit scope

[The-Compiler]

I like the idea, and I think it fits in very well with the model how pytest currently works (things from conftest.py being magically available and such).

We have the pytest_plugins setting which can be used to split off a conftest.py but it's relatively unknown and hidden. Or we have a from helpers import * in a conftest.py which has a couple caveats:

• It's an unused (star) import that still magically does something
• Despite maybe being named helpers.py, things get hairy if stuff from there is imported elsewhere too (and the same fixture is e.g. unintentionally defined twice)
• And indeed like OP says, it's hard to find a proper name to convey "this file/folder is actually helpers for conftest.py and not a test file"

Big +1 for this solution from my side. It expands on what is already common practice instead of introducing something entirely new. Thus, it seems very obvious if you know what a conftest.py is, and is both easy to use and to teach.

@RonnyPfannschmidt what do you mean with "They shouldn't be packages themselves to limit scope"?

[The-Compiler]

Oh, I missed the tests/conftest/__init__.py # loaded instead of conftest.py; imports from sibling modules part above.

My mental model was that pytest automatically loads everything in conftest/ as if it was a conftest.py, so there would be no __init__.py needed. Personally I'd prefer that, otherwise it seems like a footgun to be able to e.g. define a fixture in conftest/stubs.py but still need a from conftest.stubs import * or whatever. I think imports from conftest (or other test files) are always tricky, as they:

• Don't make it clear exactly where things are defined in the eyes of pytest
• Still are an unused star-import (see previous comment)
• Heavily depend on pytest's import mode and other details. If a tests/submod/conftest/__init__.py does from conftest.stubs import fake_thing, is that tests/conftest/stubs.py or is it tests/submod/conftest/stubs.py?

...which I now suppose is what @RonnyPfannschmidt meant with "they shouldn't be packages themselves" above too?

[RonnyPfannschmidt]

I think it's necessary to define behavior in more detail

Like is test collection allowed in there

How to handle fspath wrsppers for contest packages

There's a lot going on

Id much prefer a config entry that lists a glob of projects global plugin files tham expanding conftest to a package and expanding all its special handling correctly

[The-Compiler]

I think that'd deviate from pytest's general "convention over configuration" philosophy. I do recognize that this has bitten us in the past, but I think there's a lot of value in seeing a conftest.py and knowing that you'll find pytest stuff in there, rather than one project calling it helpers.py, the next calling it testsupport.py and a third one calling it fixtures.py and hooks.py.

The test collection question should be an easy answer, we already have the same situation with def test_*(): functions in a conftest.py now (they just get silently ignored at the moment), so I don't think that introduces anything new.

I'm sure there will be some internals to adjust with how central of a concept conftest.py has become (and there are already some subtleties with root/non-root conftest files and such). But from what I can gather (arguably from a rather quick look), it didn't seem to me like something that'd be a bad fit for how things are structured now (both internally and externally).

[RonnyPfannschmidt]

I believe we should limit it to toplevel conftests

[ofer-pd]

I'm fine with that. I currently only use a top-level confest anyway. (I haven't yet dabbled in folder-level conftests.)