Measure the writers this category harms, and let the boundary move - #76
Conversation
The calibration corpus gains 206 classroom essays by adult learners of English (PELIC, University of Pittsburgh, 2006-2012): first submitted version, writing classes, 662+ words, one text per student, selected by a fixed rule so `fetch --source pelic` yields the same ids and hashes on any machine. The affiliation proxy stands down; the population is here. At the boundary the corpus supported before they joined, 25/100, the tool flagged 9 of their 206 essays and none of the 90 published texts. The boundary is now 30/100 (2 of 296, interval 0.2-2.4%), English has its own threshold for the first time, and the report paragraph that said no group sits above the rest now computes who is flagged at the recommended boundary and says so. The learners also caught a defect: chat.eager-opener, admitted on zero hits in published prose, fired in eleven essays because its pattern took "Of course," with a comma. It now requires the exclamation mark, in both packs, with tests that the old pattern fails. Not done, deliberately: refitting the human-rate gates on the pooled corpus. With learners as the majority, twelve rules lose their gate and the connector gates inflate up to seven-fold. That is issue #75. Version 0.6.0: the first change since 0.5.0 that touches Core. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015PEbbiYSNPw7jE3LrPNhyF
… the step below shown Fable: the report quoted a language's best bound beside its recommended threshold. The two coincided while nothing was flagged anywhere, so the defect was invisible until English earned its own threshold with two texts flagged at it: "at 30/100, under 1.4%" when the bound at 30 is 2.7%. PublishedLanguage now carries the bound at the threshold and the report quotes that; a test asserts it and fails against the old code. PELIC's creators were named nowhere: the generated page now carries a Sources table with the citation, the manifest carries the DOI, and the teaching documents that still described a corpus of ninety are current. The page also shows what happened one step below the boundary, because that step is why the boundary moved. Copilot: `run` no longer succeeds with manifest texts missing; a download is validated before it earns its name and a cached non-CSV is discarded; a group with no row at the boundary is no longer reported as flagged nothing; duplicate answer ids are refused. The CLI's inability to write a Spanish report is #77. The learners' 4.4% at 25 and the pooled 30 are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015PEbbiYSNPw7jE3LrPNhyF
Committee round 1 (Fable: method · Copilot: code · DeepSeek: prose, pending)Both delivered request changes; every claim was verified before acting. Fixed in
Not changed, with reasons:
|
Committee round 2 (DeepSeek: prose, honesty, Spanish)Verdict request changes; applied in Applied: every surface in this PR now says "before 2022" rather than "before generative models existed" — GPT-3 is from 2020, so the old phrase overclaimed on the most-read pages; the rest of the repository is #78. The Spanish student sheet no longer presents a bound with no supported threshold in the same shape as a rate at one ("medida sobre solo veinticinco textos y todavía sin un umbral respaldado"). "Tightens as it grows" promised a direction; README and the generated page now say the numbers follow the data either way. The step-below sentence is computed: it says "the only flags anywhere in the corpus at that boundary" only when that is true, and "why the boundary does not sit at 25" rather than "why it sits where it does". The report caveats name the encyclopedia revisions they had omitted, and the Spanish ones say "de esta versión", "a un umbral", "acotar con precisión", with agreement fixed. Plus "almost none", "studies report", the learner interval in SKILL item 3, and the two Spanish idioms. Not applied as proposed: dropping the 5% from SKILL item 3. The 5% is not an unsupported number; it is the target the boundary is derived from (the lowest score whose interval stays under it), and the calibration page leads with the same words. What was wrong is that the skill quoted it with no provenance, so an assistant would repeat it bare. It now reads: "The boundary is the lowest score whose 95% interval stays under the project's 5% target … at 30/100 the published build flagged 2 of 296 … interval reaching 2.4%. Quote the interval; never the point estimate or the target alone." Three reviewers, three lenses, every claim verified. Ready to merge once CI is green; release order in the PR description. |
What changes
fetch --source pelicreproduces the exact selection: first version, writing classes, 662+ words, one text per student by lowest id. Verified: the tool and an independent Python selection produce the same 206 ids and identical SHA-256s.sourceHash.chat.eager-openeracceptedOf course,with a comma (an ordinary concession) alongsideCertainly!. Eleven learner essays fired it; zero published ones had, by register rather than merit. Both packs now require!; four tests added and shown to fail against the old pattern.VerdictBands,PublishedCalibration,EvidenceReport,RulePack, fixtures and tests. Blog articles andPARAPHRASE.mdkeep their figures: dated studies.v0.6.0and thendesktop-v0.6.0, since the download page already points at it.What deliberately does not change
The human-rate gates (
humanRatePer1000) stay fitted on the 90 published texts. Refitting on the pooled corpus was tried: with learners as the majority, twelve English rules lose their gate entirely and the connector gates inflate two- to seven-fold - learner rates spent on everyone, the population trap #59 documents. A gate per group is issue #75, with the numbers.Verification
runregeneratesDocs/CALIBRATION.mdand the embedded calibration at engine 0.6.0;SkillDocumentTestspins SKILL.md to it.Closes nothing; opens #75.
🤖 Generated with Claude Code
https://claude.ai/code/session_015PEbbiYSNPw7jE3LrPNhyF