accretive-audit.md

Accretiveness audit — Factor concept exercises

What "Accretive" means

A concept exercise ships a stub in which every word is "unimplemented" throw ;. The exercise is Accretive if a student who correctly implements the first K tasks (obeying every instruction for those tasks) and leaves the remaining tasks as stubs sees all tests for tasks 1..K pass.

We want every concept exercise to be Accretive.

How an exercise fails to be Accretive

The whole <slug>-tests.factor file is compiled as one unit before any test runs ([ require ] [ test ] bi in lib/exercism-tools/exercism-tools.factor). That gives two distinct failure modes.

Mode A — compile-time. A tuple class, tuple slot accessor (foo>>/>>foo), constant, symbol, or word referenced anywhere in the tests is missing from the stub and is introduced by a task later than task 1. Until that task is done the entire test file fails to compile, so no task passes. Ordinary words are safe because the stub pre-declares them all (they only throw at runtime); the trap is a definition a later task adds. When the missing definition is created in task 1 (e.g. lasagna's constant, role-playing-game's tuple), the exercise is still Accretive — doing task 1 makes the file compile.

Mode B — runtime. A test for task K calls a word the instructions assign to a later task M > K. The file compiles, but that word is still a stub and throws, so task K's test fails even though the student did tasks 1..K correctly. This typically happens for opaque/stateful structures (a queue, set, disjoint-set, mutable resource, global registry) whose task-K behaviour can only be observed through a later task's word.

Methodology

Verified empirically with the factor runtime, via bin/check-accretive:

• Mode A: compile the test file against the shipped stub. If it does not compile, the stub is missing a definition (Mode A — unless that definition is created in task 1).
• Mode B: for each prefix K, rebuild the exemplar with every word the instructions assign to a task > K re-stubbed (all tuples/constants/symbols kept, so compilation always succeeds and runtime ordering is isolated), strip STOP-HERE, run, and check that no test in tasks 1..K fails.

Run it with FACTOR=/path/to/factor bin/check-accretive [slug...].

Results

The initial audit found 11 of 47 not Accretive. All 11 have since been fixed — every concept exercise is now accretive (47/47). The table records what the audit flagged and how each was resolved.

Original verdict	Count	Exercises
Mode A (fixed)	5	bering-bearings, boatswains-bilge, dragons-descendants, factory-failsafe, pirates-path
Mode B (fixed)	5	garden-gathering, lighthouse-logbook, poetry-club, quayside-crew, tellers-triage
Structural (fixed)	1	telegraphers-tape

Mode A — a definition was introduced after task 1 (fixed)

• factory-failsafe — ERROR: machine-error was created in task 3, but the tests reference machine-error / machine-error? at compile time, so tasks 1–2 could not compile. Fixed by reordering so defining the error class is task 1; the check/monitor tasks follow.
• pirates-path — gold-count (a MEMO: word, task 4) was a comment in the stub, so the tests referenced an undefined gold-count. Fixed by shipping it as a : stub; task 4 now asks the student to change : to MEMO:.
• dragons-descendants — the subtuples fire-dragon / ice-dragon / volcano-dragon and their constructors were defined across tasks 2–4 (the stub was comment-only). Fixed by bundling the whole tuple hierarchy into task 1 (the student still defines it); the descendant constructors are pre-stubbed so the file compiles.
• bering-bearings — the polar/relative tuples, direction symbols, and the >cartesian/flip generics were scattered across tasks. Fixed by bundling all the data and generic declarations into task 1; the per-class methods stay in their own tasks.
• boatswains-bilge — task 5's valve tuple, <valve> constructor, and is-open slot were not declared; the tests reference <valve>/is-open>>, so tasks 1–4 could not compile. Fixed by pre-shipping the valve skeleton (TUPLE: valve < disposable is-open ;, a stubbed <valve>, and M: valve dispose*); task 5 fills in the bodies.

factory-failsafe, dragons-descendants, and bering-bearings now rely on a task-1 definition to compile (like lasagna and role-playing-game): the bare stub does not compile on its own, which bin/check-accretive recognises via its ACCRETIVE_VIA_TASK1 list.

Mode B — an early task's test called a later task's word (fixed)

Each was a test-design issue: the test for an early task observed its result through a word the student had not been asked to write yet. All five were fixed with test-only changes — moving the forward-referencing assertion into the task that owns the word, or (for opaque structures) observing through a library word.

• tellers-triage — task 1 [ new-queue serve-all ] needed serve-all (task 4) and task 2 also used serve-all/next-name. Tasks 1–2 now observe the min-heap through library heap-empty? / heap-size / heap-peek.
• lighthouse-logbook — a task-1 freshness test [ empty-log dup "x" sight ] needed sight (task 2); moved to task 2.
• garden-gathering — a task-2 test used release (task 3) to check that ids keep increasing; moved to task 3.
• poetry-club — task 1 used circle-of (task 3) and task 2 used same-circle? (task 4); a disjoint-set is only observable through those, so tasks 1–2 now observe circles through library equiv?.
• quayside-crew — a task-3 test [ <crane> 5 over hoist-crate tonnage>> ... ] used hoist-crate (task 4) to show cranes are independent; moved to task 4.

Structural — tests not grouped by task (fixed)

• telegraphers-tape — the tests used descriptive print labels, not TASK: markers, and each was end-to-end (construct + read + dispose together), so no prefix of tasks passed on its own. Fixed by regrouping the tests under per-task TASK: markers (task order unchanged) and adding checkpoints for the early tasks: task 1 reads through wire>> to prove the constructor stored the wire; task 2 checks input-stream?. Each group now calls only words from its own task and earlier. (Like the Mode-A fixes, the task-1 test touches the tuple, so the exercise is accretive-via-task1.)

Suggested track-wide rules (to keep new exercises Accretive)

1. The stub must make the test file compile on its own — pre-declare every tuple (with all slots), constant, symbol, error class, and word the tests reference (words as "unimplemented" throw ;). This makes Mode A impossible and is checkable with no per-exercise metadata.
2. Each task's tests must exercise only words from that task and earlier tasks (no Mode B).
3. Every exemplar should import kernel (so a re-stubbed body's throw resolves; also assumed by bin/check-accretive).

bin/check-accretive enforces (1) and (2) and can run in CI.