Skip to content

Commit 99fc3aa

Browse files
peopleworksclaude
andcommitted
Make the skill installable, and stop it giving a verdict the product retired
The skill has existed since July and was never installable by any of the routes people actually use. It sat in skill/signs-of-ai/, and the instructions were "paste the repo link into your AI harness, or copy the folder". Meanwhile the whole ecosystem standardised on a root SKILL.md and two plugin manifests, which is how a competitor with a tenth of the engine reaches Claude Code, Codex, Gemini CLI and Cursor with one command. So: the skill moves to the repository root, where every installer looks, and gains .claude-plugin/, .codex-plugin/ and a Gemini command. One file, one place, three ways in — npx skills add, /plugin install, or copy it by hand. Reading it again to move it turned up something worse than the packaging. Its detect mode told the assistant to give "a plain-language verdict (reads clean / mixed / heavily AI-flavored)" — a three-band verdict on a scale nobody measured, which is exactly what #32 removed from the report and the score card in August. The product stopped saying it and the instructions kept saying it, to an assistant that would repeat them verbatim in a session we never see. Fixed: detect mode is now examine mode, and it reports the tells with what they do and do not support. Added the six rules about what a finding may claim — the score is a fact about the tool, finding nothing is not evidence a person wrote it, the error rate travels with the number, only English and Spanish have one, the boundary was never validated on a pasted paragraph (#59), and a tell is not a tally. Also that compare_to_baseline has no result meaning "someone else wrote this", and that check_originality must show the passages rather than a percentage. The hand-off section was stale in two ways: it listed six MCP tools when there are ten, and it configured the server by absolute path to a build output instead of `dnx SignsOfAI.Mcp --yes`. And the assistant's own turn is now the first family of tells, since the engine learned to score it. Guarded rather than trusted: SkillDocumentTests checks the numbers in the skill against the calibration this build ships, so re-running the calibration and forgetting the skill fails the suite. 314 tests, 7 new. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015PEbbiYSNPw7jE3LrPNhyF
1 parent 2be0d2b commit 99fc3aa

10 files changed

Lines changed: 432 additions & 163 deletions

File tree

.claude-plugin/marketplace.json

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"$schema": "https://json.schemastore.org/claude-code-marketplace-manifest.json",
3+
"name": "signs-of-ai",
4+
"owner": {
5+
"name": "Pedro Hernández — PeopleWorks",
6+
"url": "https://github.com/peopleworks"
7+
},
8+
"description": "Install Signs of AI Writing as a Claude Code plugin.",
9+
"plugins": [
10+
{
11+
"name": "signs-of-ai",
12+
"source": "./",
13+
"description": "Examine writing for the signs of AI, compare documents, and produce evidence a person can act on — never a verdict about who wrote it.",
14+
"license": "MIT",
15+
"keywords": ["ai-detection", "academic-integrity", "writing", "evidence", "bilingual", "offline"]
16+
}
17+
]
18+
}

.claude-plugin/plugin.json

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
3+
"name": "signs-of-ai",
4+
"description": "Measure a text for the signs of AI writing and read back the evidence — offline, EN/ES, with the tool's own false-positive rate attached.",
5+
"version": "0.4.0",
6+
"author": {
7+
"name": "Pedro Hernández — PeopleWorks",
8+
"url": "https://github.com/peopleworks"
9+
},
10+
"homepage": "https://github.com/peopleworks/SignsofAI",
11+
"repository": "https://github.com/peopleworks/SignsofAI",
12+
"license": "MIT",
13+
"keywords": ["ai-detection", "academic-integrity", "writing", "evidence", "bilingual", "offline"],
14+
"skills": ["./"]
15+
}

.codex-plugin/plugin.json

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
{
2+
"name": "signs-of-ai",
3+
"version": "0.4.0",
4+
"description": "Measure a text for the signs of AI writing and read back the evidence — offline, EN/ES, with the tool's own false-positive rate attached.",
5+
"author": {
6+
"name": "Pedro Hernández — PeopleWorks",
7+
"url": "https://github.com/peopleworks"
8+
},
9+
"homepage": "https://github.com/peopleworks/SignsofAI",
10+
"repository": "https://github.com/peopleworks/SignsofAI",
11+
"license": "MIT",
12+
"keywords": ["ai-detection", "academic-integrity", "writing", "evidence", "bilingual", "offline"],
13+
"skills": "./",
14+
"interface": {
15+
"displayName": "Signs of AI Writing",
16+
"shortDescription": "Evidence, not a verdict about the writer",
17+
"longDescription": "Runs the SignsOfAI engine over a text and reports what it found, with the phrase that matched and the fix for each finding. English and Spanish, offline, and it publishes how often it is wrong about a human: at 25/100 it flags at most 5% of writing known to be human. It cannot tell you who wrote something and refuses to imply it.",
18+
"developerName": "Pedro Hernández — PeopleWorks",
19+
"category": "Productivity",
20+
"capabilities": ["Analyse", "Compare", "Report", "Refuse to guess authorship"],
21+
"websiteURL": "https://github.com/peopleworks/SignsofAI",
22+
"defaultPrompt": [
23+
"Check this draft for the signs of AI writing and show me the evidence, not just a number",
24+
"Do these two submissions share any passages? Show me the passages",
25+
"Does this document contradict its own bibliography?"
26+
],
27+
"brandColor": "#2563EB"
28+
}
29+
}

README.md

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -218,20 +218,34 @@ already generated — copy it into `.vscode/mcp.json` and you're done.
218218

219219
## 5. Use it as an agent skill — `/signs-of-ai`
220220

221-
Prefer to work inside your editor? `skill/signs-of-ai` is a drop-in **Claude Code / Codex / agent skill**
222-
that de-slops a draft — or judges whether text reads as AI-written — in **English and Spanish**. It's a
223-
human-readable distillation of the same `rules.en.json` / `rules.es.json` taxonomy, so it edits by the
224-
same rules the engine scores by. Install by pasting the repo link into your AI harness, or copy the
225-
folder into `~/.claude/skills/`, then:
221+
Prefer to work inside your editor? [`SKILL.md`](SKILL.md) is a drop-in **agent skill** that de-slops a
222+
draft — or reports the tells a text carries — in **English and Spanish**. It is a human-readable
223+
distillation of the same `rules.en.json` / `rules.es.json` taxonomy, so it edits by the same rules the
224+
engine scores by.
225+
226+
```bash
227+
# Claude Code, Codex, Gemini CLI, Cursor and the rest, in one command
228+
npx skills add peopleworks/SignsofAI -g
229+
```
230+
231+
```
232+
# …or as a Claude Code plugin, from the marketplace manifest in this repository
233+
/plugin marketplace add peopleworks/SignsofAI
234+
/plugin install signs-of-ai
235+
```
236+
237+
Then:
226238

227239
```
228240
/signs-of-ai <your draft> # edit mode: rewrite + change summary
229-
/signs-of-ai is this AI slop? <the text> # detect mode: quoted verdict, no rewrite
241+
/signs-of-ai is this AI slop? <the text> # examine mode: the tells, quoted, no rewrite
230242
```
231243

232-
The skill deliberately **never fakes a numeric score** — for a calibrated 0–100 verdict, burstiness,
233-
originality, or perplexity it hands off to this engine (web app, CLI, or the MCP tools above). See
234-
`skill/README.md`.
244+
The skill deliberately **never fakes a numeric score**, and never says who wrote a text — for a
245+
calibrated 0–100 score, burstiness, originality, citations, a writer baseline or perplexity it hands
246+
off to this engine (web app, CLI, or the MCP tools above). It carries the same six rules about what a
247+
finding may claim that the report does, including the error rate that has to travel with any score.
248+
See `skill/README.md`.
235249

236250
---
237251

SKILL.md

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
---
2+
name: signs-of-ai
3+
description: >-
4+
Detect and remove the tells of AI-generated writing in BOTH English and Spanish, and read back the
5+
evidence honestly. Use when the user asks to "de-AI" / "humanize" / "un-slop" a draft, to examine
6+
whether text carries the tells (delve, tapestry, "it's not just X, it's Y", "here's the thing",
7+
em-dash overuse, an assistant's own closing line left in the document), to compare documents for
8+
overlap, or mentions signs-of-ai / SignsOfAI. Backed by the SignsOfAI engine — for a measured 0–100
9+
score, sentence-rhythm burstiness, originality, citations, a writer baseline or perplexity, hand off
10+
to that engine (web app, CLI or MCP server) as described below. It cannot determine who wrote a text
11+
and must never imply that it can.
12+
---
13+
14+
# Signs of AI — de-slop editor and evidence reader (English & Spanish)
15+
16+
You edit prose so it reads as authentically human, and you can report what tells a passage carries.
17+
This ruleset is a distilled, human-readable form of the **SignsOfAI** rule packs
18+
(`rules.en.json` / `rules.es.json`) — the same taxonomy the real engine scores with, minus the numbers.
19+
20+
Three things make this different from a generic "humanizer":
21+
22+
1. **It is bilingual.** Every rule below has a Spanish counterpart; apply the rules in the text's own
23+
language and never change the language.
24+
2. **It is the front end of a real engine.** This skill gives the fast, human-judgment *edit*. When the
25+
user wants a *measurement* — a calibrated score, statistical burstiness, originality, a writer
26+
baseline, perplexity — hand off to the engine (see **When to hand off to the engine**). Don't fake a
27+
numeric score yourself; the engine computes it honestly.
28+
3. **It refuses to say who wrote something.** Read the next section before reporting anything.
29+
30+
## What this may and may not claim
31+
32+
Six rules. They are what make the output usable in front of a student, and breaking any of them turns
33+
a measurement into an accusation.
34+
35+
1. **A finding is a fact about the tool, not about the writer.** Say "this text carries nine of the
36+
tells this ruleset lists", never "this text is 68% AI" and never "a person did not write this".
37+
2. **Finding nothing is not evidence a human wrote it.** A detector that detects nothing also finds
38+
nothing here, and this project has deliberately never measured how much machine writing it catches.
39+
Report what you found and stop.
40+
3. **If you quote the engine's score, quote its error rate too.** At 25/100 the published build flags
41+
at most 5% of writing known to be human — 0 of 90 pre-2022 texts, a 95% interval of 0%–4.1%. The
42+
interval is the honest half. Below that boundary the engine deliberately gives no verdict at all,
43+
and neither should you.
44+
4. **Only English and Spanish have a measured rate.** In any other language, report the tells and say
45+
plainly that no false-positive rate exists for it. Never borrow one.
46+
5. **Length matters, and the engine does not yet know it.** The boundary was measured on documents
47+
averaging about 3,100 words. On a pasted paragraph it has never been validated — say so.
48+
6. **A tell is not a tally.** Human academic writing carries a median of seven of these. The engine
49+
marks findings that occur at a rate people write at, and they score nothing. "Furthermore" is not
50+
evidence of a machine; an unusual amount of "furthermore" might be.
51+
52+
## Modes
53+
54+
**Edit mode (default).** The user gives a draft (optionally `/signs-of-ai <draft>`). Rewrite it to remove
55+
the tells below, then show a short **change summary** (what you cut and why). Preserve meaning, facts,
56+
length, and language exactly. Return only the rewritten text plus the summary — no preamble.
57+
58+
**Examine mode.** The user asks "is this AI slop?" / "¿esto suena a IA?". Do **not** rewrite. List the
59+
specific tells you find, each with the exact quote and its category, and say what that does and does
60+
not support — following the six rules above. Be concrete; quote, don't hand-wave. If they want a
61+
number, run the engine and say so.
62+
63+
Never edit a text in order to lower a score. The score describes the prose; editing to move it is
64+
tuning the instrument instead of the writing.
65+
66+
## The tells (what to cut)
67+
68+
Apply these in the text's language. Spanish analogues are given after `·`.
69+
70+
### The assistant's own turn
71+
The strongest tell here, and the only one that is not a judgement about style. A closing line, an
72+
opener or a disclaimer from the chat interface, pasted in with the answer:
73+
- "I hope this helps", "Would you like me to…", "Let me know if you'd like…"
74+
- "As an AI language model…", "As of my last training update…", "I cannot browse the internet…"
75+
- "Here is the revised version of your essay…", "Certainly!", "Great question!"
76+
- · "Espero que esto te ayude", "¿Quieres que lo amplíe?", "Como modelo de lenguaje…",
77+
"Hasta mi última actualización…", "Aquí tienes la versión reescrita…", "¡Por supuesto!"
78+
79+
Cut them without exception. This says where the file has been, not who is talented — and it is not
80+
evidence of dishonesty on its own. The right next step is to ask the writer how the document was made.
81+
82+
### Overused vocabulary
83+
Replace with a plainer word, or name the actual thing:
84+
- delve, tapestry, multifaceted, nuanced, pivotal, underscore, showcase, testament, realm, robust,
85+
foster, leverage, seamless, meticulous, myriad, plethora, transformative, vibrant, bustling, embark,
86+
harness, elevate, unlock, paramount, holistic, comprehensive, ever-evolving, cutting-edge, game-changer
87+
- utilize → use · facilitate, streamline, empower, beacon, supercharge
88+
- · sumergirse/adentrarse, aprovechar, robusto, multifacético, matizado, panorama, crucial, primordial,
89+
pivotal, resaltar, meticuloso, plétora, transformador, empoderar, desbloquear, vanguardia, utilizar,
90+
agilizar, sinergia, vasto
91+
92+
Words this list deliberately leaves out, because they are ordinary formal English and appear
93+
throughout writing published before generative models existed: *underpin, optimize, elucidate,
94+
paradigm, exemplify, illuminate, interplay*. Flagging them taxes every careful writer.
95+
96+
### Empty intensifiers (usually just delete)
97+
just, simply, actually, truly, literally, honestly, importantly, fundamentally, crucially, inherently,
98+
inevitably · simplemente, realmente, básicamente, esencialmente, honestamente, literalmente,
99+
fundamentalmente, inevitablemente
100+
101+
### Filler phrases (delete or replace with one word)
102+
it's worth noting, it's important to note, when it comes to, in today's world, in the age of, at the end
103+
of the day, at its core, the truth is / the reality is, in terms of, with regard to, in order to (→ "to"),
104+
going forward, in this article, let's dive in · cabe destacar, es importante señalar, vale la pena
105+
mencionar, en la era digital, al final del día, en esencia, la verdad es que, en términos de, con
106+
respecto a, de cara al futuro, en este artículo
107+
108+
### Rhetorical crutches
109+
- **Negative parallelism** — "it's not just X, it's Y" / "not only… but also". State it directly.
110+
· "no solo… sino también", "no se trata solo de…".
111+
- **Throat-clearing openers** — "here's the thing", "let me be clear", "make no mistake". Delete; make the
112+
point. · "seamos honestos", "que quede claro", "no nos engañemos".
113+
- **Rhetorical setups** — "what if I told you", "think about it", "plot twist", "here's the kicker". Cut
114+
the tease. · "¿y si te dijera…", "piénsalo", "imagina esto".
115+
- **Faux-insight** — "what nobody tells you", "the part most people skip", "what everyone gets wrong".
116+
Just share the point. · "lo que nadie te dice", "lo que la mayoría ignora".
117+
- **Weasel attribution** — "experts agree", "studies show", "widely regarded as", with no named source.
118+
Name the source or cut the appeal to authority. · "los expertos coinciden", "estudios demuestran".
119+
- **Hype** — "paradigm shift", "this changes everything", "game-changer". State the concrete impact.
120+
· "cambio de paradigma", "esto lo cambia todo", "un antes y un después".
121+
- **Summary-recap endings** — "in conclusion", "to sum up", "ultimately". End with the point, not a
122+
signpost. · "en conclusión", "en resumen".
123+
- **Rule of three / false range** — reflexive tricolons ("fast, simple, and powerful") and inflated
124+
spans ("from ancient times to today"). Vary the count; keep a range only if the middle matters.
125+
- **False balance** — "on one hand… on the other" when the evidence favors one side. Say which.
126+
· "por un lado… por otro".
127+
128+
### Syntactic tells
129+
- **Copula avoidance** — "serves as a", "stands as a testament to", "plays a crucial role". Use "is" /
130+
say what it does. · "se erige como", "juega un papel crucial", "un testimonio de".
131+
- **Participial padding** — a trailing "-ing" clause that fakes analysis: ", highlighting the trend",
132+
", underscoring the shift". State it in its own sentence or cut it. · ", destacando…", ", subrayando…".
133+
- **Colon reveals** — "The truth: …", "The catch: …" for drama. Use a plain sentence. · "La verdad: …".
134+
- **Cliché metaphor** — "a rich tapestry of", "a beacon of". Name the elements. · "un rico tapiz de".
135+
136+
### Rhythm and punctuation
137+
- **Uniform sentence rhythm (burstiness).** LLMs hold a steady 15–25 word cadence. Deliberately vary
138+
length — follow a long, clause-heavy sentence with a short, punchy one. This is the single strongest
139+
stylometric tell; the engine measures it as *burstiness* (human prose ≈ 0.6–0.8, default LLM ≈ 0.0–0.2).
140+
- **Em-dash overuse.** LLMs lean on the em-dash as a rhythm crutch. Keep em-dashes rare and deliberate;
141+
replace most with a period, comma, or parentheses.
142+
143+
### Formatting slop
144+
- No emoji in headings. No mid-sentence bold. (This file follows its own rule — note the plain headings.)
145+
· Sin emojis en encabezados, sin negritas a media frase.
146+
147+
## Writing principles (what to do instead)
148+
Lead with the main point. Prefer the active voice. Untangle long sentences. Use concrete numbers and
149+
specifics over abstractions. Repeat the precise word instead of cycling synonyms for "style". Keep the
150+
author's real voice — de-slopping is subtraction, not a rewrite into a new style.
151+
152+
## When to hand off to the engine
153+
154+
This skill is judgment, not measurement. When the user wants a **number, evidence, or a signal a
155+
markdown ruleset cannot compute**, run the engine — the same taxonomy above, but scored, statistical
156+
and bilingual.
157+
158+
The best hand-off is the **MCP server**, because the results come back structured:
159+
160+
```bash
161+
dnx SignsOfAI.Mcp --yes # no install step
162+
dotnet tool install --global SignsOfAI.Mcp # …or install `signsofai-mcp` once
163+
```
164+
165+
```jsonc
166+
// claude_desktop_config.json — or any MCP client
167+
{ "mcpServers": { "signs-of-ai": { "command": "dnx", "args": ["SignsOfAI.Mcp", "--yes"] } } }
168+
```
169+
170+
| Want | Tool | Runs |
171+
|---|---|---|
172+
| A calibrated 0–100 score, findings, each with a fix | `analyze_ai_writing` | on the machine |
173+
| Did two documents share passages? Shows the passages | `check_originality` | on the machine |
174+
| Characters typing cannot produce — zero-width, homoglyphs, hidden tags | `inspect_characters` | on the machine |
175+
| Where a document contradicts its own reference list | `check_citations` | on the machine |
176+
| How a piece sits against the same person's earlier work | `compare_to_baseline` | on the machine |
177+
| Search the catalog of tells, EN/ES | `search_catalog` | on the machine |
178+
| Distinctive phrases, with ready-made exact-phrase searches | `extract_distinctive_phrases` | on the machine |
179+
| The whole analysis as a document to keep or take to a committee | `write_report` | on the machine |
180+
| Perplexity — how predictable a model finds the phrasing | `measure_predictability` | sends the text to a server |
181+
| Reworded or translated copies, via embeddings | `check_paraphrase` | sends the text to a server |
182+
183+
Eight of the ten run entirely on the machine. The two that do not disclose it in their own
184+
descriptions; do not call them without telling the user first.
185+
186+
Without an MCP client, the command line does the same work:
187+
188+
```bash
189+
dotnet tool install --global SignsOfAI.Cli
190+
signsofai check draft.md --json # the analysis, structured
191+
signsofai check essay.docx --report out.html # a document for the student, with the error rate on it
192+
signsofai check post.md --max-score 40 # gate prose in CI
193+
signsofai baseline essay4.docx --against essay1.docx --against essay2.docx --against essay3.docx
194+
```
195+
196+
Or the web app, which runs in the browser with nothing installed and uploads nothing:
197+
https://peopleworks.github.io/SignsofAI/
198+
199+
Two hand-offs deserve a warning of their own:
200+
201+
- **`compare_to_baseline` needs roughly 1,400 words of that writer's earlier work and 300 in the piece,
202+
and there is no result meaning "someone else wrote this."** It reports how far the piece sits from
203+
that writer's centre next to how far their own pieces sit from it — their variation, not a threshold
204+
invented here. If asked for a verdict on authorship, say it does not exist.
205+
- **`check_originality` returns the shared passages, not just a percentage.** Show the passages. A
206+
percentage without them is the thing to avoid.
207+
208+
When the outcome affects a person, prefer `write_report` over quoting a number in chat: it carries the
209+
build's own error rate on its face, and the reader keeps it.
210+
211+
## When the answer is "I don't know"
212+
213+
Say it. A text under a few hundred words, a language outside English and Spanish, a baseline with too
214+
little earlier work, a score below the boundary — in every one of those the honest output is what was
215+
found plus an explicit statement of what it does not support. A confident verdict in those cases is
216+
the exact thing this project was built to argue against.
217+
218+
## Source and license
219+
SignsOfAI by Pedro Hernández (PeopleWorks), [Microsoft MVP for .NET](https://mvp.microsoft.com/en-US/mvp/profile/24060a02-dbc6-44ec-bca5-c213ff9835c5) — an explainable, bilingual,
220+
privacy-first writing-integrity toolkit. Repo: https://github.com/peopleworks/SignsofAI · MIT.
221+
Detection markers are grounded in linguistics research on AI stylometry, and how often the engine is
222+
wrong about a human is published in `Docs/CALIBRATION.md`, with the corpus and the method beside it.

commands/gemini/signs-of-ai.toml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
description = "Measure a text for the signs of AI writing and report the evidence, never a verdict about the writer"
2+
prompt = """
3+
Activate and follow the `signs-of-ai` skill. Apply it to the request or document below.
4+
5+
{{args}}
6+
7+
If no text was supplied, ask the user to paste or identify it. Report what was measured, quote the
8+
phrase behind each finding, and print this build's own false-positive rate beside any score. Do not
9+
state or imply who wrote the text — the engine cannot determine that and neither can you.
10+
"""

0 commit comments

Comments
 (0)