Decouple fixtures from POs

[kmike]

I think we have a few issue with the current test fixture approach:

1. You must create a PO before creating a test fixture. This is backwards.
2. It should be possible to test several POs against a single test fixture.
3. It should be easier to check PO against a fixture without involving pytest test discovery.
4. When a field raises an exception, other fields can't be tested #254 (a sepatate issue)

To address 1-3, I think we should un-tie fixtures from concrete page objects. You have a folder with data, it contains inputs and outputs. As soon as there are enough inputs for page object to produce the output, it should be possible to test any page object against this folder, without special requirements for the folder name. There could be a sepate layer for pytest discoverability.

[kmike]

I also wonder if serialization could be done in a different way: not implicit file names mapping, but a manifest file. Something like this:

  {
    "version": 2,
    "inputs": {
      "HttpResponse": {
        "body": "inputs/response.html",
        "url": "https://example.com/product/1",
        "status": 200,
        "headers": {"content-type": ["text/html"]}
      },
      "HttpClient": {
        "responses": ["inputs/api-product.json", "inputs/api-reviews.json"]
      }
    },
    "output": "output.json",
    "frozen_time": "2026-03-18T19:25:41+00:00"
  }

This could make it easier to inspect what's there (e.g. outside web-poet), and deserialization can be more flexible (it doesn't need to match serialization exactly - you can rename a file if you want).

[wRAR]

Analyzing the current web_poet.testing.fixture.Fixture API:

• type_name - currently returns the parent dir name ("pages.MyPage"). Needed for get_page() and short_name.
• short_name - currently returns type_name/test_name ("pages.MyPage/test-1"). Needed for some parts of pytest integration as it's an unique ID.
• get_page() - returns an instance of the PO created from the deps. Needed for get_output(). We can't use this if we don't know the PO type.

Then we have get_output() and assert methods that call it. The pytest integration calls all of those assert methods, it doesn't call get_output() directly but I guess other consumers could.

It looks like there are two possible new APIs:

1. An instance of Fixture is still tied to a specific PO, whose type name is passed to its constructor in addition to the fixture path. If it's not passed, the name of the parent dir is assumed to be it.
2. An instance of Fixture can be freestanding in which case you need to pass a PO type name to everything that needs it, which is mostly get_output() and assert methods, and by extension the pytest integration. A pytest test item should be tied to a specific PO in any case so it will be able to pass its type name to all those methods.

Both have their benefits and the choice seems to depend on the use cases we want to support. It may also be possible to have both, but does it make sense?

[wRAR]

I also wonder if serialization could be done in a different way: not implicit file names mapping, but a manifest file.

For this, it may be possible (and would be nice I guess?) to replaceenhance only web_poet.serialization.api.SerializedDataFileStorage keeping the dict[str, SerializedLeafData] protocol that it, serialize() and deserialize() use.

How Fixture uses this:

• Fixture.save() serializes its content with serialize(), creates an instance of SerializedDataFileStorage and asks it to write the serialized data. We may want to take the output format version in Fixture.save() or just always write the newest one.
• Fixture.get_page() creates an instance of SerializedDataFileStorage, asks it to read and return the serialized data and calls deserialize() to get the PO instance. Here we will not need to change anything, hopefully.

[kmike]

Both have their benefits and the choice seems to depend on the use cases we want to support. It may also be possible to have both, but does it make sense?

For me it looks cleaner for Fixture not to be aware of page object which uses it. You may have methods which take a page object (or just move this all to a separate function / class), but it doesn't seem right for Fixture to receive page object in __init__.

[wRAR]

I've pushed #258 with straightforward changes for this. Now the old use case still works and also you can instantiate a Fixture with a path to a directory that doesn't have a PO name as its parent, and call assert methods passing a PO name to them.

No idea what to do with the type_name attribute, as the current implementation is so simple that you don't even know in any of the Fixture code whether the parent directory is a PO name or not, should we pass a bool flag to the Fixture constructor saying "don't try to use the parent directory name"? :) In that case we could also raise an exception in all code that didn't receive a PO name explicitly.

For short_name, should we deprecate it if its only use is to have something unique for pytest? The pytest plugin can calculate something internally, based on some different logic.

Edit: I've added initial support for this to the pytest plugin, the discovery code (which is not modified yet) can now pass a PO name to TestCase that it creates, so that it's used instead of the dir name. Fixture.short_name is no longer called by pytest code and could be deprecated if we wanted.

[wRAR]

For the pytest discovery to work we need to be able to get the PO name when we find a fixture folder in web_poet.testing.pytest.pytest_collect_file(). If we cannot rely on the folder path we seem to need some registry that can be accessed when only knowing that path. In any case, what's the planned file layout that is different feom the current one that we want to support here?

[wRAR]

enhance only web_poet.serialization.api.SerializedDataFileStorage

This depends on the scope for the manifest file. The serialization API is only responsible for the deps, and that's currently self-contained, so if we want to have e.g. output and frozen_time fields in the manifest, the read part of the API can ignore that while the write part will need to return a part of the manifest that describes the deps so that the caller could put it into the manifest (which is possible but ugly especially if we want a single write() function that supports multiple format versions).

SerializedDataFileStorage() also takes the path to the inputs dir now, so if we want to keep the API it makes sense that the manifest would need to be also stored there and not next to it.

[wRAR]

On the second read I noticed that the new format wants to put small non-bytes data righ tin the manifest, this is very nice but breaks all of the serialization API, as SerializedLeafData is a mapping from file names to file contents. So the cleanest (or the only possible?) approach is to make a completely new serialization API with new functions. Not sure if we will be able to make it in a way that allows future extensions with a version bump though.

[wRAR]

One thing that could be made more extensible is that currently Fixture.get_page() separately gets the serialized data from the disk (via SerializedDataFileStorage()) and deserializes it (via deserialize()), and Fixture.save(), correspondingly, separately serializes the data and writes it. If these calls should be specific to the format 1, we should add an abstraction layer that supports all formats instead of adding that support to Fixture directly.

[wRAR]

Regarding the pytest part.

As I see it, correct me if I'm wrong, we want this to be supported:

1. There is a bunch of fixtures in random paths, and a bunch of POs, and we put (PO, fixture path) pairs into some JSON registry, with several POs being able to use the same fixture. This registry is needed for two things: learning where are the fixtures (as they could be in random paths) and learning what to run when the user wants to run all tests or all tests for a given PO (no registry is needed when a user provides the PO and the fixture path explicitly).
2. The user can run all tests, all tests for a specific PO, a specific fixture for a specific PO and a specific field (etc.) test of a specific fixture for a specific PO.
3. If there is no registry, the fixture parent name will still be considered its linked PO, and a single (PO, fixture path) pair will be collected.

We need to design the full test node ID first, and such a node ID needs to start with a file path if we want the expected test integration. I only see two possibilities for this file path: the registry or something in the fixture dir (e.g. output.json as we have now). The registry being optional, the second option looks better. Then it seems like the ID needs to be in a form of path/to/test-1/output.json::my.pages.ProductPage::price, so that pytest immediately knows which fixture and which PO to use. This seems to solve almost everything we want, and the registry will not even be needed to use these per-PO IDs, only when you need to run everything. The only awkward thing is "run all tests for a specific PO", we'll need to use -k my.pages.ProductPage or add a web-poet specific pytest option for this.

In this case the node IDs collected by older web-poet even for the same directory structure will no longer work (maybe we can have two collection code paths though?).

The mechanism for finding the registry in this case can be anything we want I think, as that's just a bunch of Python code not tied to the pytest APIs, we could just look for a specific file starting from the current dir.

Thoughts?

[wRAR]

We need to design the full test node ID first, and such a node ID needs to start with a file path if we want the expected test integration.

(about this, it seems to be possible to do a completely custom test collection without pytest_collect_file() but I won't be surprised if all stuff like IDE etc. integration may break)