Skip to content

Commit 44c9c30

Browse files
committed
fix: eliminate double-correct distractors and hint bugs in sets J/L/N
Audit of the modal/deontic/belief translation generators against the decoded 2008 originals. All correct answers (option a) were already faithful; the defects were in distractors and hint text. Set L (deontic) — 4 double-correct distractors marked a right answer wrong under shuffled options (each was logically equivalent to the key, three with hints that admitted it): - template1 negate: ∼(Eu · F{u}) → (Eu · ∼F{u}) [A⊃∼B ≡ ∼(A·B)] - template3 either-not: ∼(E{u} · F{u}) → (∼Eu ∨ ∼F{u}) [De Morgan] - template6 both-and: ((x)E{x} · (x)F{x}) → (∃x)(E{x} · F{x}) [∀ distributes] - template13 required: ∼R∼V{u} → R∼V{u} [∼R∼A ≡ OA] Replacements are genuinely inequivalent, authentic-style errors matching the original's underline/connective/O-R-swap distractors. Set J (modal) — template25 offered the bare "Everyone is necessarily A" phrasing, which Gensler §11.2 treats as ambiguous (de re vs de dicto), while keying de re and marking de dicto wrong. Dropped the bare phrasing (the 2008 engine only ships the disambiguated "has the property of being necessarily" form). Corrected stale "26 of 31" coverage comment. Set N (belief) — cosmetic: HINT_FORGOT_UNDERLINE_U.replace('u', …) hit the "u" in "You", garbling three hints; retargeted to 'the u'. And willingDontCombine showed the belief-flavored don't-combine hint on a willing template; added HINT_DONT_COMBINE_WILL. Added regression guards (content/generators.test.ts): no option hint may admit logical equivalence, and no hint may be corrupted by a bad "You" replacement.
1 parent 3fe0235 commit 44c9c30

7 files changed

Lines changed: 136 additions & 84 deletions

File tree

content/generators.test.ts

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,4 +32,56 @@ describe('generators dispatcher', () => {
3232
expect(out.subSets.length).toBe(2);
3333
expect(out.subSets[0]!.questions.length).toBe(3);
3434
});
35+
36+
// Regression guard for the double-correct distractor bug found in the
37+
// 2026-07 audit: several Set L distractors were logically equivalent to
38+
// the correct answer and shipped hints that openly admitted it (e.g.
39+
// "By De Morgan, this is logically equivalent", "a more verbose form").
40+
// A distractor that concedes equivalence is a second correct answer.
41+
// Sweep every generated option hint for that tell-tale wording.
42+
it('no distractor hint admits logical equivalence to the answer', () => {
43+
const FORBIDDEN =
44+
/logically equivalent|by de morgan|unnecessarily verbose|more verbose form|verbose but/i;
45+
for (const key of registeredSetKeys()) {
46+
const gen = getGenerator(key)!;
47+
for (const seed of [1, 7, 42, 99, 12345, 2718]) {
48+
const set = gen(seed, 20);
49+
for (const subset of set.subSets) {
50+
for (const q of subset.questions) {
51+
for (const opt of q.options) {
52+
expect(
53+
opt.hint == null || !FORBIDDEN.test(opt.hint),
54+
`${key} seed ${seed}: option "${opt.label}" hint admits equivalence: ${opt.hint}`
55+
).toBe(true);
56+
}
57+
}
58+
}
59+
}
60+
}
61+
});
62+
63+
// Regression guard for the setN `.replace('u', …)` bug: the underline
64+
// hint began with "You forgot…", so replacing the first "u" hit the one
65+
// in "You", rendering garbage like "Yo x forgot…". No hint should be
66+
// corrupted this way.
67+
it('no option hint is corrupted by a bad "You" replacement', () => {
68+
for (const key of registeredSetKeys()) {
69+
const gen = getGenerator(key)!;
70+
for (const seed of [1, 7, 42, 99, 12345]) {
71+
const set = gen(seed, 20);
72+
for (const subset of set.subSets) {
73+
for (const q of subset.questions) {
74+
for (const opt of q.options) {
75+
if (opt.hint) {
76+
expect(
77+
/\bYo\b|Yox|Yo /.test(opt.hint),
78+
`${key} seed ${seed}: corrupted hint: ${opt.hint}`
79+
).toBe(false);
80+
}
81+
}
82+
}
83+
}
84+
}
85+
}
86+
});
3587
});

content/sets/__snapshots__/setJ.generator.test.ts.snap

Lines changed: 40 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -556,124 +556,124 @@ Wrong quantifier — should be universal.",
556556
"options": [
557557
{
558558
"id": 0,
559-
"label": "$ (Dh \\cdot \\lozenge \\sim Dh) $",
559+
"label": "$ (Fg \\cdot \\lozenge \\sim Fg) $",
560560
},
561561
{
562562
"hint": "‘F is a contingent property of x’ means ‘x is F but could have lacked F’ — \`(Fx · ◇∼Fx)\`.
563-
A contingent property of Harry means Harry actually has itfirst conjunct should beDh’, not ‘◇Dh’.",
563+
A contingent property of George means George actually has itfirst conjunct should beFg’, not ‘◇Fg’.",
564564
"id": 1,
565-
"label": "$ (\\lozenge Dh \\cdot \\lozenge \\sim Dh) $",
565+
"label": "$ (\\lozenge Fg \\cdot \\lozenge \\sim Fg) $",
566566
},
567567
{
568568
"hint": "‘F is a contingent property of x’ means ‘x is F but could have lacked F’ — \`(Fx · ◇∼Fx)\`.
569569
Thats the necessary-property form. Contingent means it actually has it but COULD have lacked it.",
570570
"id": 2,
571-
"label": "$ \\square Dh $",
571+
"label": "$ \\square Fg $",
572572
},
573573
{
574574
"hint": "‘F is a contingent property of x’ means ‘x is F but could have lacked F’ — \`(Fx · ◇∼Fx)\`.
575-
Self-contradiction. The second conjunct should saycould have lacked it’ — \`◇∼Dh\`.",
575+
Self-contradiction. The second conjunct should saycould have lacked it’ — \`◇∼Fg\`.",
576576
"id": 3,
577-
"label": "$ (Dh \\cdot \\sim Dh) $",
577+
"label": "$ (Fg \\cdot \\sim Fg) $",
578578
},
579579
],
580-
"prompt": "Being dishonest is a accidental property of Harry.",
580+
"prompt": "Being faithful is a contingent property of George.",
581581
},
582582
{
583583
"answer": "",
584584
"correctId": [
585585
0,
586586
],
587-
"id": "gen.J.22.7",
587+
"id": "gen.J.30.7",
588588
"options": [
589589
{
590590
"id": 0,
591-
"label": "$ \\lozenge (\\exists x)Cx $",
591+
"label": "$ (x)(Cx \\supset \\square Cx) $",
592592
},
593593
{
594-
"hint": "Translate ‘possible some’ as ‘◇(∃x)’.
595-
Wrong scopethat meanstheres some x such that x could be charming’ (anyone). Different claim.",
594+
"hint": "‘All A’s have the property of being necessarily B’ uses the simple (box-inside) form — \`(x)(Ax ⊃ ☐Bx)\`. Not ambiguous: ‘property of being necessarily …’ disambiguates.
595+
Thats the de dicto (box-outside) reading. ‘Property of being necessarily …’ disambiguates to box-inside.",
596596
"id": 1,
597-
"label": "$ (\\exists x)\\lozenge Cx $",
597+
"label": "$ \\square (x)(Cx \\supset Cx) $",
598598
},
599599
{
600-
"hint": "Translate ‘possible some’ as ‘◇(∃x)’.
601-
That meanspossibly everyone’ — wrong quantifier.",
600+
"hint": "‘All A’s have the property of being necessarily B’ uses the simple (box-inside) form — \`(x)(Ax ⊃ ☐Bx)\`. Not ambiguous: ‘property of being necessarily …’ disambiguates.
601+
The sentence isnt ambiguous. ‘Property of being necessarily …’ disambiguates.",
602602
"id": 2,
603-
"label": "$ \\lozenge (x)Cx $",
603+
"label": "Ambiguous between (x)(Cx ⊃ ☐Cx) and ☐(x)(Cx ⊃ Cx)",
604604
},
605605
{
606-
"hint": "Translate ‘possible some’ as ‘◇(∃x)’.
607-
You forgot the ‘∼’. (You added anot’.)",
606+
"hint": "‘All A’s have the property of being necessarily B’ uses the simple (box-inside) form — \`(x)(Ax ⊃ ☐Bx)\`. Not ambiguous: ‘property of being necessarily …’ disambiguates.
607+
Your answer has too many boxes.",
608608
"id": 3,
609-
"label": "$ \\sim \\lozenge (\\exists x)Cx $",
609+
"label": "$ \\square (x)(Cx \\supset \\square Cx) $",
610610
},
611611
],
612-
"prompt": "It’s possible that someone is charming.",
612+
"prompt": "Every carpenter has the property of being necessarily charming.",
613613
},
614614
{
615615
"answer": "",
616616
"correctId": [
617617
0,
618618
],
619-
"id": "gen.J.30.8",
619+
"id": "gen.J.25.8",
620620
"options": [
621621
{
622622
"id": 0,
623-
"label": "$ (x)(Px \\supset \\square Vx) $",
623+
"label": "$ (x)\\square Vx $",
624624
},
625625
{
626-
"hint": "All A’s have the property of being necessarily B’ uses the simple (box-inside) form \`(x)(Ax ⊃ ☐Bx)\`. Not ambiguous: ‘property of being necessarily …’ disambiguates.
627-
Thats the de dicto (box-outside) reading. ‘Property of being necessarily disambiguates to box-inside.",
626+
"hint": "F is a necessary property of x’ means ‘in all possible worlds, x would be F’ \`☐Fx\`.
627+
Thats the de dicto reading; ‘has the property of being necessarily virtuousdisambiguates to de re.",
628628
"id": 1,
629-
"label": "$ \\square (x)(Px \\supset Vx) $",
629+
"label": "$ \\square (x)Vx $",
630630
},
631631
{
632-
"hint": "All A’s have the property of being necessarily B’ uses the simple (box-inside) form \`(x)(Ax ⊃ ☐Bx)\`. Not ambiguous: ‘property of being necessarily …’ disambiguates.
633-
The sentence isnt ambiguous. ‘Property of being necessarily …’ disambiguates.",
632+
"hint": "F is a necessary property of x’ means ‘in all possible worlds, x would be F’ \`☐Fx\`.
633+
You need another modal operator.",
634634
"id": 2,
635-
"label": "Ambiguous between (x)(Px ⊃ ☐Vx) and ☐(x)(Px ⊃ Vx)",
635+
"label": "$ (x)Vx $",
636636
},
637637
{
638-
"hint": "All A’s have the property of being necessarily B’ uses the simple (box-inside) form \`(x)(Ax ⊃ ☐Bx)\`. Not ambiguous: ‘property of being necessarily …’ disambiguates.
639-
Your answer has too many boxes.",
638+
"hint": "F is a necessary property of x’ means ‘in all possible worlds, x would be F’ \`☐Fx\`.
639+
Wrong quantifiershould be universal.",
640640
"id": 3,
641-
"label": "$ \\square (x)(Px \\supset \\square Vx) $",
641+
"label": "$ (\\exists x)\\square Vx $",
642642
},
643643
],
644-
"prompt": "All plumbers have the property of being necessarily virtuous.",
644+
"prompt": "Everyone has the property of being necessarily virtuous.",
645645
},
646646
{
647647
"answer": "",
648648
"correctId": [
649649
0,
650650
],
651-
"id": "gen.J.27.9",
651+
"id": "gen.J.25.9",
652652
"options": [
653653
{
654654
"id": 0,
655-
"label": "$ \\square Sm $",
655+
"label": "$ (x)\\square Cx $",
656656
},
657657
{
658658
"hint": "‘F is a necessary property of x’ means ‘in all possible worlds, x would be F’ — \`☐Fx\`.
659-
You need another modal operator.",
659+
Thats the de dicto reading; ‘has the property of being necessarily confuseddisambiguates to de re.",
660660
"id": 1,
661-
"label": "$ Sm $",
661+
"label": "$ \\square (x)Cx $",
662662
},
663663
{
664664
"hint": "‘F is a necessary property of x’ means ‘in all possible worlds, x would be F’ — \`☐Fx\`.
665-
Your answer isnt a wffmodal operators dont take parentheses around their argument: write ‘☐A’, not ‘☐(A)’ or ‘(☐A)’.",
665+
You need another modal operator.",
666666
"id": 2,
667-
"label": "$ \\square (Sm) $",
667+
"label": "$ (x)Cx $",
668668
},
669669
{
670670
"hint": "‘F is a necessary property of x’ means ‘in all possible worlds, x would be F’ — \`☐Fx\`.
671-
Thats the contingent-property form, not necessary.",
671+
Wrong quantifiershould be universal.",
672672
"id": 3,
673-
"label": "$ (Sm \\cdot \\lozenge \\sim Sm) $",
673+
"label": "$ (\\exists x)\\square Cx $",
674674
},
675675
],
676-
"prompt": "Being scholarly is a necessary property of Madonna.",
676+
"prompt": "Everyone has the property of being necessarily confused.",
677677
},
678678
],
679679
"shuffleOptions": true,

content/sets/__snapshots__/setL.generator.test.ts.snap

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -319,9 +319,9 @@ There’s no ‘ought’ in the prompt — drop the \`O\`.",
319319
},
320320
{
321321
"hint": "‘Don’t combine doing A with doing B’ → \`∼(A{u} · B{u})\` — both action letters are imperative (underlined).
322-
By De Morgan, this is logically equivalent — but the canonical form for ‘either don’t A or don’t B’ keeps the ‘∨’ explicit.",
322+
You forgot to underline the imperative agent letterdescriptive \`Au\` and imperative \`A{u}\` are different wffs. Both parts are imperatives here — underline both u’s.",
323323
"id": 1,
324-
"label": "$ \\sim (S\\underline{u} \\cdot F\\underline{u}) $",
324+
"label": "$ (\\sim Su \\vee \\sim F\\underline{u}) $",
325325
},
326326
{
327327
"hint": "‘Don’t combine doing A with doing B’ → \`∼(A{u} · B{u})\` — both action letters are imperative (underlined).

0 commit comments

Comments
 (0)