Layer B: intensity clause appended after {TEXT} causes models to echo/paraphrase it into the rewrite output #337

Summary

When a Layer B /clean strategy step combines a named tactic with an intensity (e.g. paraphrase@1.0, as used by the default paraphrase@0.8,mlm@0.2 strategy), build_prompt() in service/scripts/rewrite_text.py appends _intensity_clause(level) after the tactic prompt -- and the tactic prompt already has the real {TEXT} content embedded via its own trailing `

{TEXT}` block. So the final prompt sent to the LLM looks roughly like:

<tactic instructions>... Output only the rewritten text.

---
<the actual text to rewrite>

Modulate this rewrite so roughly a fraction 1.00 of tokens change: 0 would keep the wording unchanged, 1 rewrites everything. At low intensity keep the sentence structure... [rest of _intensity_clause]

Because the intensity instruction comes after the content block instead of before it, models frequently treat it as more content to respond to rather than a meta-instruction to silently apply -- and echo/paraphrase it back into the rewritten output instead of just rewriting the original sentence.

Reproduction

1. Configure a local Ollama backend (WATERMARKS_REWRITE_BACKEND=ollama; reproduced with both llama3.2:3b and qwen2.5:7b-instruct).
2. POST /clean on a short text file with the default strategy (paraphrase@0.8,mlm@0.2), or an explicit options.strategy: "paraphrase@1.0" override.
3. Inspect the cleaned output.

Observed (qwen2.5:7b-instruct, before fix)

Input: "This is a test sentence with an invisible mark and a soft hyphen inside it, meant to check that the cleaning pipeline actually rewrites the wording as part of Layer B."

Output:

Here is the rewritten text, with a fraction of 0.25 of tokens changed: [rewrite]
(Note: I changed the wording of some function words...)
If you would like a higher intensity rewrite, here is one with approximately 0.50 of tokens changed: [second rewrite]
(Note: I changed the wording of more tokens...)

Not usable as a drop-in replacement: it contains meta-commentary, offers multiple candidate rewrites, and restates prompt-internal parameters (the intensity fraction) as if they were source content.

Root cause

build_prompt():

else:
    base = _tactic_prompt(tactic, text, lang, original_lang)  # already embeds "...

---
{TEXT}"
    if rewrite_level is not None and tactic != "code":
        base = base + "

" + _intensity_clause(rewrite_level)  # appended AFTER the text block

Suggested fix

Build the tactic instruction header with an empty placeholder so the real text can be appended exactly once, at the very end, after every instruction clause (intensity, style):

header = _tactic_prompt(tactic, "", lang, original_lang)
suffix = "

---
"
if header.endswith(suffix):
    header = header[: -len(suffix)]
base = header
if rewrite_level is not None and tactic != "code":
    base = base + "

" + _intensity_clause(rewrite_level)
# ... style clause unchanged ...
if tactic is not None:
    base = base + "

---
" + text

This keeps every prompt template wording unchanged -- it only reorders where the content block sits relative to the intensity/style clauses.

Verified

With this reordering, the same qwen2.5:7b-instruct setup returns a clean single-sentence paraphrase with no leaked commentary. llama3.2:3b still leaks even with the fix applied (a separate, general instruction-following weakness of that smaller model) -- so this is specifically a prompt-structure defect, reproducible independent of which backend model is used.

Environment

• watermarks-remover v0.7.0 (Claude Code plugin install)
• Windows 11, Python 3.14
• Backend: Ollama (local), models llama3.2:3b and qwen2.5:7b-instruct