Anyone tried it on Pangram?

[tkeg-expat]

I used pangram to check the rewrite and apparently it still says 100% AI

[logz05]

I got the same result - 100% AI.

[bigdog898555-cmyk]

is pangram good? ive been aminly using https://aidetector.com/app/my-scans/c9ec9df2-51b0-45b1-a2ab-c5b82098e72d quilbot ai and zerogpt

[guillaumemeyer]

Pangram detects whether content is AI-generated, not whether it is watermarked.
I am thinking of including an optional humanizer in this project with @vnsavitri.
Would that be valuable?

[gokrisflyer]

you are a hero! :)

[Yurakonoplya]

Worth keeping the two questions apart, because they answer different things and rarely agree.

Pangram, ZeroGPT and the rest answer "does this read like a machine wrote it". A watermark check answers "is the mark still in there". You can strip a watermark down to nothing and still score 100% AI, because the prose is the same prose it always was. It happens the other way round too: people paste writing they did themselves and get flagged.

So a 100% AI reading tells you nothing about whether this repo did its job. What does tell you is the output itself. Diff it against your input. Identical character for character apart from invisible codepoints means the scrub ran and nothing else did. Wording moved means the rewrite ran. For a statistical mark neither of those is the answer, and the only honest check is a sample whose key you hold, because without the key nobody scores it from outside, detector sites included. That last part is what I keep an MIT scorer for: https://github.com/Yurakonoplya/unmark-checker. Mark your own text, put it through, watch what the number does.

@guillaumemeyer on the humanizer, the part I would find valuable is it reporting what the pass cost, meaning and facts kept, rather than only whether some detector flipped. A rewrite that clears a detector by dropping half the content clears it for the wrong reason.