Allow multiple cases per fixture file#5149
Conversation
|
I like this idea. One other big benefit it has over my original idea is that it keeps the blame history semi-legible - while the input part would get clobbered on any full test file re-writes, the output part stays unchanged, leaving part of it in tact, which is nice. |
Co-authored-by: GiGaGon <107241144+MeGaGiGaGon@users.noreply.github.com>
|
|
||
| [case keep_short_strings] | ||
| x = "abc" | ||
| # output |
There was a problem hiding this comment.
Should we change this to [output]? The magical comment is a bit confusing.
There was a problem hiding this comment.
I think no, with the idea being that any of these smaller case cells has the same format as a pre-change test file, so they can share the same parsing/mental machinery
There was a problem hiding this comment.
mypy's format (which this borrows from) uses a bracket marker for this ([out]). Should I move the multi-cell format the same way ([case ] + [output]) and leave single-case files on # output?
Fixes #4506.
Description
In #4506, MeGaGiGaGon flagged that the longest fixture files in
tests/data/cases/are painful to work with.preview_long_strings.pyandpreview_long_strings__regression.pyare both 1000+ lines: input and output for any given case sit hundreds of lines apart, and when a test fails the assertion points at the end-of-file mismatch with nothing tying it back to the original case.MeGaGiGaGon's first proposal was to split big files into folders of numbered files. JelleZijlstra pushed back on numbering ("renumbering files would be needlessly annoying") and suggested an alternative shape: sections within larger files, like mypy's test data format.
This PR takes the mypy-style path. Fixture files can group multiple cases under
[case <name>]headers, and the two longest files are migrated as the first consumers.The format is opt-in. Files without a
[case ]header at the top run through the legacy single-case loader unchanged. Cell names appear in pytest IDs (test_simple_format[preview_long_strings::assign_x_1]), and on failure the assertion message attaches the file path, the cell header line, and the# outputmarker line, so the traceback points at the case under test rather than the file boundary.A
# flags:line above the first[case ]applies to every cell, and cell flags take precedence on overlap. That makes preview-to-stable migrations more granular: cells can graduate one at a time, while neighbors in the same file keep their preview flags.Documentation in
docs/contributing/the_basics.mdcovers the format and the merge semantics.I picked
preview_long_strings*for the first migrations because they're the worst offenders on file length.Checklist - did you ...
--previewstyle, following the stability policy?CHANGES.mdif necessary?