Skip to content

Commit a096eaf

Browse files
authored
docs: enhance documentation on random seed impact in PICT (#151)
1 parent 2f8330d commit a096eaf

2 files changed

Lines changed: 6 additions & 2 deletions

File tree

api/pictapi.h

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -294,7 +294,9 @@ PictDeleteTask
294294
// Allocates a new model. Models need to be deleted with DeleteModel
295295
//
296296
// Parameters:
297-
// randomSeed A seed used to randomize the engine
297+
// randomSeed A seed used to randomize the engine heuristic. Different seeds
298+
// can legitimately produce different row counts while still
299+
// satisfying the requested coverage.
298300
//
299301
// Returns:
300302
// Non-nullptr Allocation succeeded (a handle is returned)

doc/pict.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,9 @@ The test cases are printed to the standard output stream. The first line of the
7272

7373
If a model and options given to the tool do not change, every run will result in the same output. However, the output can be randomized if ```/r``` option is used. A randomized generation prints out the seed used for that particular execution to the error output stream. Consequently, that seed can be fed into the tool with ```/r:seed``` option to replay a particular generation.
7474

75-
Different random seed values will often produce a different number of total test cases. This is because packing n-way combinations is a “hard problem” for which PICT and other tools use heuristics. These heuristics are deterministic, but they are dependent on initial conditions. Sometimes the algorithm is lucky and packs all of your desired combinations into fewer test cases. Variations of 5% - 10% are common.
75+
Different random seed values will often produce a different number of total test cases. This is because packing n-way combinations is a “hard problem” for which PICT and other tools use heuristics. These heuristics are deterministic, but they are dependent on initial conditions. Sometimes the algorithm is lucky and packs all of your desired combinations into fewer test cases. Variations of 5% - 10% are common.
76+
77+
The seed only changes how efficiently combinations are packed into rows; it does not relax the requested coverage target. Two valid seeded runs may therefore have different row counts while still covering the same required combinations.
7678

7779
All errors, warning messages, and other auxiliary information is printed to the error stream.
7880

0 commit comments

Comments
 (0)