Support failure persistence via seed replay

[moodmosaic]

Right now, failures are lost unless manually tracked. Rendezvous should persist failing test cases across runs. It should save the seed of a failing case, and replay it before generating new cases.

Expected behavior:

• Failing cases are saved under rendezvous-regressions/.
• Replayed first on next run.
• (Optional?) Configurable path and grouping (e.g., by module or test).

[BowTiedRadone]

Great idea! The only thing I don't fully agree with is running failing cases first by default. I would instead add a flag to check historic failures or historic failures + novel cases. The reasoning behind this is that we saw in real life that some users won't want to stick to the default behavior of replaying failures first.

[BowTiedRadone]

The flag could be --ci and the options could be check, explore, and both. Default: explore (novel cases).

[moodmosaic]

The flag may be replay instead of ci, (if we decide to add another flag).

[moodmosaic]

If we have a regression file, it’s best to take it into account; if the file is there, it should be used. Console logging is important so that the users aren’t confused (as happened in our case with proptest).