Rules taught in academic English fire two to three times as often on second-language learners

[peopleworks]

The calibration corpus now contains 206 essays by adult learners of English (PELIC, University of Pittsburgh, 2006–2012, one text per student, 662+ words — see Docs/Calibration/README.md). Measuring the rule set against them, next to the 65 published English articles and encyclopedia revisions already there, shows a handful of rules firing two to three times as often on learner prose:

Rule	Learner essays it fires in	Published English texts it fires in	Ratio
rhet.in-conclusion	40%	14%	×2.9
lex.crucial	21%	12%	×1.7
lex.just	47%	29%	×1.6
rhet.not-only-but	27%	17%	×1.6
lex.actually	21%	14%	×1.5

None of these rules is wrong about machines. They are the connectors and intensifiers that English-for-academic-purposes courses teach, so a writer who learned English in a classroom uses them more than a writer who did not, and the tool reads the lesson as a tell.

The effect on the published number is visible and is now on the page rather than averaged away: the learner group sits highest in every column of Docs/CALIBRATION.md, and at the boundary the corpus supported before they joined (25/100) it flagged 9 of 206 — 4.4%, interval 2.3–8.1% — against 0 of 90 for the published texts. The boundary moved to 30/100 as a result.

What this issue is not. It is not a request to drop rhet.in-conclusion because learners use it. Reweighting a rule to fit one population is the trap #59 documents, and a rule that measures the genre rather than the machine is still measuring something the calibration can price. The human-rate gate (humanRatePer1000) already exists for exactly this and is fitted on the whole corpus.

What it is. Three questions the new group makes answerable:

1. Should the human-rate gate be fitted per group as well as per language, so a rate learned on published articles is not spent on a classroom essay? The mechanism exists; the question is whether the corpus is large enough per group to support it (the gate needs eight texts per rule).
2. Are there rules whose share on learners is high enough that they should be marked as genre (GenreGate) rather than scored, the way the rule-of-three already is? rhet.in-conclusion at 40% is the candidate.
3. What is the corresponding picture for Spanish? Five Spanish rules fire on ordinary formal Spanish #61 is this issue in the other language, and the Spanish corpus is still 25 encyclopedia texts.

Any change here goes through the usual door: measure against the corpus first, then write the rule, and the published rate moves in whichever direction it moves.

[peopleworks]

Tried the obvious answer to question 1 — refitting the human-rate gates (thresholds) on the pooled 296-text corpus — before opening the pull request that adds the learner group, and it is not shippable as-is:

• Twelve English rules lose their gate entirely. The rate is a percentile of per-text usage, and once 206 learner essays are the majority, a rule learners rarely use (lex.utilize, lex.facilitate, lex.simply, rhet.in-terms-of…) has a percentile of zero. It stops being gated and goes back to firing on a single occurrence in the published articles where it was measured in the first place — 17% of them for lex.utilize.
• The connector gates inflate two- to seven-fold: lex.just 0.38 → 2.61 per 1,000 words, lex.crucial 0.16 → 1.16, rhet.in-conclusion 0.22 → 1.42, rhet.not-only-but 0.16 → 1.35. Those are learner rates being spent on everyone.
• Leave-one-out, each text would keep 4.0 findings instead of 6.5 and 39 of 296 would come back clean instead of 7 — which reads as an improvement and is mostly the gates describing the majority population.

So the gates stay fitted on the 90 published texts, the calibration page measures the engine as shipped, and the pooled refit is what this issue must not do. A gate per group is the design to evaluate; the corpus can support it for the connector rules (they fire in well over eight texts of each group) and not yet for the rarer ones.

[peopleworks]

Two more constraints on the design, from the method review of #76:

• The gap between groups is inflated by an unknown amount. The gates are fitted in-sample on the same 90 published texts that report 0 of 90, while the 9 of 206 for learners is out-of-sample. Any per-group comparison this issue produces has to be leave-one-out within each group, or the published texts get a head start nobody measured.
• The learner group's own threshold sits at the edge. 2 of 206 at 30 supports the 5% promise, but a resample gives 5 of 206 (5.6%) in the worst 2.5% of draws. A per-group gate that moves two essays across the line moves the group's threshold. Report the interval of the threshold, not just the threshold.