extensions: extension directories get no automatic Connect content cleanup

[ian-flores]

Summary

VIP's own Connect tests get automatic content cleanup: any content they create is tracked and removed at the end of the test and again at the end of the run. Extension directories loaded via --extensions do not. An extension that publishes content to Connect leaves it behind unless its author writes their own cleanup.

This is a known, deliberate gap rather than an oversight — it is documented in the docstring of src/vip_tests/conftest.py and in the AGENTS.md that vip scaffold writes into every generated extension — but it is worth tracking as its own issue because the workaround is manual and easy to forget.

Background

Three fixtures implement the cleanup, all in src/vip_tests/conftest.py:

• _connect_created_guids (session-scoped) — the shared list of created content GUIDs
• _connect_content_cleanup (autouse=True) — per-test cleanup
• _connect_end_of_run_sweep (session-scoped, autouse=True) — end-of-run sweep

When #609 moved VIP's core fixtures into src/vip/fixtures.py and registered them as a pytest plugin, these three deliberately stayed behind in the conftest, directory-scoped to src/vip_tests.

Why they were not moved

Two of the three are autouse=True. A plugin registered through the pytest11 entry point is active in any pytest session where vip is installed, not only when VIP's own tests run. Making autouse fixtures global therefore has effects far outside VIP:

1. Every test in every unrelated project that happens to have vip installed would pay for Connect-content bookkeeping it never asked for.
2. Worse, _connect_content_cleanup requests connect_client, which calls require_connect_api_key. A project with its own unrelated vip.toml that configures [connect] without an API key would have every one of its tests fail during setup.

That second case is not hypothetical: moving these fixtures globally broke an existing selftest that configures [connect] with no API key specifically to exercise deselection behaviour.

So the trade was deliberate — a missing convenience for extension authors, rather than a class of spurious setup failures for everyone else.

Impact

An extension that creates Connect content and does not clean up after itself accumulates content on the deployment being validated. That runs against VIP's own guarantee that its tests are non-destructive and leave nothing behind, and it is exactly the sort of thing a validation run against a customer deployment should not do.

The current workaround, documented in the generated AGENTS.md, is for the extension author to clean up explicitly — either with their own fixture or by calling connect_client.cleanup_content(guids) directly.

Possible directions

Whatever the fix, "register the existing fixtures globally" is not it, for the reasons above. Some options worth weighing:

• An opt-in fixture that extension authors request by name, so nothing is autouse and nothing fires in unrelated projects. Costs a line in the extension, but it is explicit and safe.
• Scope the autouse behaviour to collected items that carry a VIP product marker, so it activates for extension tests that opt into @pytest.mark.connect and stays dormant everywhere else.
• Have vip verify register the cleanup fixtures for the duration of its own run only, rather than for every pytest process that imports the plugin.

The first is the smallest and most predictable. The second is the most convenient for extension authors but needs care: it must not resurrect the keyless-[connect] failure, since requesting connect_client at all is what triggers it.