feat: Introduce testing.constructors module and a pytest plugin

[FBruzzesi]

Description

TL;DR

• This PR extracts the backend constructor machinery that used to live in tests/conftest.py into a first class, documented module (narwhals.testing.constructors) and ships it as an installable pytest plugin (registered via the pytest11 entry point). Downstream libraries that build on Narwhals can get parametrised dataframe/lazyframe fixtures, without copy pasting our conftest.
• It also reworks how coverage is collected in CI (dropping pytest-cov in favour of a coverage run -> combine -> report flow), because registering an entry point plugin changes import timing in a way that breaks coverage measurement.

Motivation

• The constructor fixtures (constructor, constructor_eager, the --constructors flag) were Narwhals-internal, undocumented, and trapped inside tests/conftest.py. Downstream projects that wanted to test against the same backend matrix had to vendor our conftest.
• Related to [Enh]: Provide testing module to enable downstream libraries testing #739 * not marking it as closing, since the goal is to fully dogfood the plugin in our own suite first.
• The previous design used bare callables with __name__ sniffing ("pandas" in str(constructor)) for backend detection. Moving to a typed frame_constructor object with explicit metadata (implementation, requirements, eager/lazy, nullability, GPU) makes that introspection first class.

What changed

1. New module: narwhals/testing/constructors.py

The core abstraction is frame_constructor, a callable wrapper around a backend builder function.

• Each constructor wraps exactly one backend builder and turns a column oriented dict into a native frame.
• Metadata lives on the instance: implementation, requirements, is_eager, is_nullable, needs_gpu.
• Registration is explicit and performed via the @frame_constructor.register(...) decorator, which instantiates the wrapper and stores it in a shared class level _registry.
• is_* properties delegate to Implementation (is_pandas, is_polars, is_pandas_like, is_spark_like, is_duckdb, is_lazy, needs_pyarrow, is_available, etc.), replacing the old string sniffing.
• __call__(obj, namespace, **kwds) builds the native frame and then wraps it via namespace.from_native(...), so the fixture returns a Narwhals frame directly.

Module level helpers (all part of __all__):

• available_backends() / available_cpu_backends(): names whose libraries are importable.
• get_backend_constructor(name): typed lookup, raises ValueError with the valid set on a bad name.
• prepare_backends(include=..., exclude=...): filtered, sorted selection (exclude takes precedence).
• is_backend_available(*packages): thin importlib.util.find_spec check.
• pyspark_session() / sqlframe_session(): session factories (moved out of tests/utils.py).
• DEFAULT_BACKENDS: the historical default subset (pandas, pandas[pyarrow], polars[eager], pyarrow, duckdb, sqlframe, ibis).

2. New module: narwhals/testing/pytest_plugin.py

The installable pytest plugin, wired up in pyproject.toml:

[project.entry-points.pytest11]
narwhals_testing = "narwhals.testing.pytest_plugin"

exposes

Fixtures (auto-parametrised against the selected, installed backends):

Fixture	Parametrised over
nw_frame	every selected backend (eager + lazy)
nw_dataframe	eager backends only
nw_lazyframe	lazy backends only
nw_pandas_like_frame	pandas-like eager backends

CLI options (under a narwhals option group):

• --nw-backends=pandas,polars[lazy],duckdb: comma separated selection. Defaults to DEFAULT_BACKENDS intersected with what is installed.
• --all-nw-backends: every installed CPU backend (overrides --nw-backends; excludes modin and pyspark[connect]).
• --use-external-nw-backend: escape hatch so a downstream suite can register its own parametrisation; the plugin still adds the options but stops parametrising the fixtures.
• Environment overrides: NARWHALS_DEFAULT_BACKENDS overrides the default list (useful e.g. under cudf.pandas).

3. Supporting type aliases: narwhals/testing/typing.py

Public aliases for downstream type hints: Data (the column oriented input mapping), FrameConstructor, DataFrameConstructor, LazyFrameConstructor, and the NarwhalsNamespace protocol (anything exposing from_native).

4. Test suite migration

The test suite partially moved off the old pattern to keep the changes more contained.

• Compatibility shims: constructor, constructor_eager, and constructor_pandas_like are kept as aliases over the new nw_* fixtures, wrapped in _PatchedFrameConstructor / _PatchedDataFrameConstructor proxies that default namespace to narwhals. This lets the legacy call sites keep working during the transition. A TODO marks their eventual removal.
• Because fixtures now return a Narwhals frame directly (constructor(data) instead of nw.from_native(constructor(data))), call sites simplified. Example from join_test.py: the from_native_lazy(constructor(df)) helper collapsed to constructor(df).lazy().
• tests/utils.py: the Constructor / ConstructorEager / ConstructorPandasLike type aliases are now re-exported from the conftest proxies; the pyspark_session / sqlframe_session helpers moved into narwhals.testing.constructors. uses_pyarrow_backend now keys off str(constructor) instead of __name__.
• New dedicated tests: tests/testing/constructors_test.py (registry, properties, prepare_backends, equality/hash) and tests/testing/plugin_test.py (uses pytest's pytester to assert the plugin parametrises eager/lazy fixtures and that --use-external-nw-backend disables parametrisation).

5. Coverage and CI rework

Registering the entry point plugin forces narwhals/__init__.py to import early (before pytest-cov installs its tracer), which silently dropped coverage. Fix:

1. Drop the pytest-cov dependency.
2. Drive coverage manually: coverage run -m pytest ... && coverage combine && coverage report, so the tracer starts at interpreter startup before any import.
3. Configure subprocess/execv/fork patches in [tool.coverage.run]. execv and fork are unsupported on Windows (coverage raises), so Windows CI jobs collapse those to subprocess via the COVERAGE_PATCH_EXECV / COVERAGE_PATCH_FORK env vars; coverage dedupes the final list. COVERAGE_PROCESS_START=pyproject.toml is set across the workflows.

Tooling changes:

• New run-ci-coverage Make target (run -> combine -> report, with optional COV_SOURCE to scope coverage, e.g. src/narwhals/_spark_like). run-ci stays for non coverage runs (doctests, narrow-dep, tpch, ibis, modin).
• All workflows migrated --constructors to --nw-backends and --all-cpu-constructors to --all-nw-backends.
• utils/import_check.py: a testing allowlist permits the deliberate lazy imports of every backend inside the constructors.
• utils/sort_api_reference.py: testing added to the skip set (its API doc is hand ordered).
• Docs: new docs/api-reference/testing.md documenting fixtures, options, and a quick start.
• CONTRIBUTING.md updated for the renamed flags.

Public API surface

Newly importable and documented:

• narwhals.testing.frame_constructor (and the constructors module helpers above).
• narwhals.testing.typing aliases.
• The pytest fixtures nw_frame, nw_dataframe, nw_lazyframe, nw_pandas_like_frame and the --nw-* options.

Quick start (downstream usage)

import narwhals as nw
import narwhals.stable.v2 as nw_v2
from narwhals.testing.typing import Data, DataFrameConstructor, LazyFrameConstructor


def test_shape(nw_dataframe: DataFrameConstructor) -> None:
    data: Data = {"x": [1, 2, 3]}
    df = nw_dataframe(data, namespace=nw)
    assert df.shape == (3, 1)


def test_laziness(nw_lazyframe: LazyFrameConstructor) -> None:
    lf = nw_lazyframe({"x": [1, 2, 3]}, namespace=nw_v2)
    assert isinstance(lf, nw_v2.LazyFrame)

pytest --nw-backends="pandas,polars[lazy]"
pytest --all-nw-backends

Out of scope

• A nw_series_constructor fixture. Ergonomically nice, but a separate feature request.
• Dropping the constructor / constructor_eager / constructor_pandas_like compatibility shims. Tracked by a TODO; they stay until every test calls the nw_* fixtures directly.

Old description

This PR introduces two new features:

1. A rework of tests/conftest.py into narwhals/testing/constructors
   • TBD what should be publicly exposed in here
2. A pytest plugin that exposes the fixtures for dataframe/lazyframe constructors.
   • I wonder if we should already return them as narwhals DataFrame/LazyFrame. If we find a way to achieve the following points, then I personally think we should do so, as we almost always run nw.from_native right after.
     • Distinguish between native frame and nw.from_native kwargs -> For this it's enough to provide it as different proper arguments in __call__, e.g. nw_kwargs, backend_kwargs.
     • How do we know which version a user would use? Should we require to pass either the Version.X or the namespace {nw, nw_v1, nw_v2} ? If we do so, I would argue that we must enforce it and never assume for the user. In my opinion nw_frame_constructor(data, version=nw) would still look better than nw.from_native(nw_frame_constructor(data)).
       Update: feat: Let constructor fixtures return narwhals object directly #3569
   • In light of the CI failures for downstream projects, I also realized that having fixtures called constructor and constructor_eager is not really intuitive. I am happy to brainstorm better names for downstream users. And similarly the flags --all-cpu-constructors, --constructors. (*)
     Update: refactor: Rename exposed fixtures and pytest options #3556
   • We could provide a nw_lazy_constructor for "free"
   • I believe it might be useful to expose a series constructor (under the name nw_series_constructor). I know it's possible to make a dataframe and then get a column. But it would definitely improve users ergonomics. However, that's feature request/improvement and out of scope of this PR

(*) Renamed in #3556 and #3569

• fixtures: nw_frame, nw_dataframe, nw_lazyframe
• pytest options: --nw-all-backends (misleading that has no CPU in it?), --nw-backends=.... I think the term constructor might be be too vague, and there is no other case in the codebase. While we have a bunch of cases in which we allow to pass a backend

pyest-cov changes

The [project.entry-points.pytest11] registers the pytest plugin which we dogfood in the testsuite here. When pytest starts, it loads all entry-point plugins before pytest-cov installs its coverage tracer.

Importing narwhals.testing.pytest_plugin forces Python to load narwhals/__init__.py (to resolve the package), which eagerly imports the entire public API.

On main, no entry point existed, so no narwhals code loaded before coverage started.

The fix:

1. Remove pytest-cov dependency
2. Add coverage options in pyproject.toml to track subprocess, execv and fork (the latter two are not available on windows, hence multiple hacks are performed in GithubAction's to dynamically populate the fields in pyproject.toml)
3. Manually run coverage run -m pytest ..., coverage combine and coverage report, so that coverage run starts the tracer at Python startup before any imports

---

Thanks to @EdAbati and @dangotbanned - I mostly adjusted from your work, not much more than that!

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Related issue [Enh]: Provide testing module to enable downstream libraries testing #739 (not marking as closing, since we should fully dogfood in the test suite)

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

[dangotbanned]

Just a quick one from me RE:

Don't panic due to the number of file changes - I will explain

We have 257 file changes because all tests change the imports as:

- from tests.utils import Constructor, ConstructorEager 
+ from narwhals.testing.typing import Constructor, ConstructorEager

Is it possible to add something like this, and keep the imports in place (temporarily)?:

# tests.utils.py
from narwhals.testing.typing import Constructor as Constructor , ConstructorEager as ConstructorEager

I really REALLY want this PR, but the diff 😳 (yes yes, I know I'm guilty of this too 😉)

[FBruzzesi]

@dangotbanned reverted such commit 😉

---

Coverage is not picked up in CI though 🥹

[camriddell]

Overall, I'm much happier with this version as you eliminated the reliance of string checking for class attributes that may/may not exist and am implicit registration mechanism.

A few minor comments/food-for-thoughts

[camriddell]

Any preference to do something if the key is already set?

1. Raise an error
2. emit a warning
3. let the user replace whatever was there before (current)

I think the current is fine, just wanted to see if you had thoughts here.

[FBruzzesi]

I think emitting a warning would be ok

[camriddell]

Ah, the implementation I saw was different from what you had linked to! There are some differences, but to your point we're on the path to an improvement :)

Happy to have a chat about the design on another channel (Discord?) or perhaps the next iteration of the test suite! 😆

[camriddell]

If duplication is a concern, we can have a secondary mechanism register_with. I'm okay with this volume of duplication so will leave this decision to you :)

class frame_constructor:
    ...
    def register_with(self, name, ...): 
        def dec(func): 
            ... # register and instantiate 
            from copy import replace # I don't think this is supported by all of our python versions :(
            return replace(self, name=name, func=func, ...) 
       return dec

[camriddell]

@FBruzzesi I still need to look at the rest of the suite (so far only dug into constructors.py), I had finished my thoughts on that file so figured I would share them with you as it'll take me a bit more time to review the rest of the changes.

[FBruzzesi]

Hey everyone @dangotbanned @camriddell @MarcoGorelli @EdAbati 👋🏼
I know we have a lot of issues and PRs ongoing and open, yet this feature is something I am looking for quite some time, and especially useful in downstream project integration.

I know the diff is quite large - I hope it's manageable, but please let me know how I can make it easier to review (I am going to update the current description soon). Unlike other past cases, I don't see how this could be split down into multiple PRs.

---

For context, there are a few followups to fully dogfooding on this PR, e.g. feat/testing-constructors-xfail, yet I am not planning to work on it before this one is at least somehow validated

[EdAbati]

I'd be very happy to take a look and thank you again for working on this!

Today I'm busy but have some time tomorrow :)

[dangotbanned]

but please let me know how I can make it easier to review (I am going to update the current description soon).

One thing that could make it easier is if the description was interspersed with examples.

E.g.

• how we were doing things before?
• why was that bad?
• how do we improve?
• what are we doing instead?
• why is the new way is better? (and I'm sure it is btw 😘)