feat(rewrite): encode humanize benchmark finding with warning and regression guards

[poorvith-mp]

Summary

Encodes the empirical benchmark finding regarding prompt-based humanization into rewrite_text.py and test_clean_strategy.py. Emits a runtime warning when humanize is used in a strategy, guards the default configuration against regressing to humanize, and documents the detector mechanics.

Context

On the 8-doc watermarked benchmark corpus (02–04 Sep 2026), paraphrase@0.4 alone scored a Pangram human_like of 0.44. Appending humanize@0.2 collapsed it to ~0.02.

Prompting an LLM to "write like a human" forces it into formulaic transitions, uniform pacing, and clichéd conversational registers that modern AI detectors (e.g. Pangram) easily identify. Meanwhile, masked-LM infill (mlm@0.2 via roberta-large) perturbs token distributions bidirectionally without adopting an LLM conversational cadence, holding human_like at ~0.42 while improving the removal margin.

While config/clean_strategy.json was updated to paraphrase@0.8,mlm@0.2, this critical institutional memory was not encoded in code or test assertions.

Changes

1. Runtime Warning: rewrite_text.py:parse_strategy() emits a UserWarning when the humanize tactic is present in a parsed strategy, warning about the 0.44 → 0.02 collapse and suggesting paraphrase + mlm.
2. CLI Help & Docstring: Documented the benchmark finding under --tactic humanize and the module docstring.
3. Regression Guards:
   • test_parse_strategy_humanize_warning: asserts that parsing humanize@0.2 warns with the empirical score citation.
   • test_default_strategy_has_no_humanize: asserts that config/clean_strategy.json never includes a humanize step.
4. Docs: Updated skills/remove-ai-marks/references/removal-matrix.md with the empirical caution on humanize and documented mlm.

Verification

• python -m pytest tests/test_clean_strategy.py -v (23 passed in 0.39s)
• python -m ruff check service/scripts/rewrite_text.py tests/test_clean_strategy.py (All checks passed)
• python -m ruff format --check service/scripts/rewrite_text.py tests/test_clean_strategy.py (2 files already formatted)

Summary by CodeRabbit

• New Features

  • Added mlm as a Layer B rewording option.

• Documentation

  • Updated rewrite guidance with benchmark results for humanize and mlm.
  • Recommended paraphrase + mlm as an alternative to humanize.

• Warnings

  • Using humanize now displays a runtime warning with performance guidance.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: guillaumemeyer/watermarks-remover/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: eefddb68-d381-4369-9f1f-4610d065f620

📥 Commits

Reviewing files that changed from the base of the PR and between 5390d2b and 5ab71a7.

📒 Files selected for processing (1)

• service/scripts/rewrite_text.py

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds warnings and documentation for the humanize tactic, documents mlm as an alternative, and adds tests for warning behavior and the default strategy.

Changes

Humanize tactic guidance

Layer / File(s)	Summary
Strategy warnings and tactic documentation service/scripts/rewrite_text.py, skills/remove-ai-marks/references/removal-matrix.md, tests/test_clean_strategy.py	parse_strategy emits a UserWarning for humanize. Module documentation, CLI help, and the removal matrix describe benchmark results and recommend paraphrase plus mlm. Tests cover the warning and confirm that the default strategy excludes humanize.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to 5ab71

The recommended strategy is executable, and no actionable regression from this change is established.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: it adds a warning and regression tests for the reported humanize benchmark finding.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}