Core AI: Excerpt generation produces meta/explanatory content instead of clean excerpt after multiple attempts

[PriyankaGusani]

Description

While testing the Core AI excerpt generation in WordPress 7.0 RC, I observed inconsistent and incorrect output after multiple generations.

Expected behavior:

Each generation should return a clean, ready-to-use excerpt
Output should be concise and directly usable
No explanations, formatting notes, or metadata should be included

Actual behavior:

After multiple attempts, the AI generates meta-style content instead of a proper excerpt

The output I got:

Here's an excerpt that captures the key points concisely:

"Managing multiple WordPress sites manually is time-consuming and risky..."

This excerpt:

• Highlights the core problem
• Explains the automation solution concept
• Mentions features
  The response includes:
  Explanation of the excerpt
  Bullet points describing it
  Quotes around the content

Why this is an issue:

Not production-ready output
Breaks UX expectations (users expect direct content, not explanation)
SEO risk if saved without editing
Indicates possible prompt leakage or improper output sanitization

Step-by-step reproduction instructions

1. Opened a post in the editor
2. Clicked on “Generate excerpt” multiple times (3–4 times)
3. Observed the generated output

Screenshots, screen recording, code snippet

Environment info

WordPress Version: 7.0 RC (Beta)
Feature: Core AI (Excerpt Generation)
Browser: Chrome (latest)
Setup: Local/Development environment

Please confirm that you have searched existing issues in the repo.

• Yes

Please confirm that you have tested with all plugins deactivated except the AI plugin.

• Yes

Please confirm which theme type you used for testing.

• Block
• Classic
• Hybrid (e.g. classic with theme.json)
• Not sure

[jamesgiroux]

Root Cause Analysis

The system prompt for excerpt generation (includes/Abilities/Excerpt_Generation/system-instruction.php) doesn't explicitly instruct the LLM to return only the excerpt text. Without that constraint, models (especially via the HuggingFace fallback router) will sometimes include meta-commentary like "Here's an excerpt that captures the key points:" followed by the actual excerpt, bullet-point explanations, and quoted text.

The output sanitization at Excerpt_Generation.php:127 only trims surrounding whitespace and quote characters — it doesn't strip preamble or trailing commentary.

Current system prompt (trimmed):

You are an editorial assistant that generates excerpts...

Goal: ...generate a concise, engaging, and accurate excerpt...

The excerpt suggestion should follow these requirements:
- Be approximately 55 words...
- Should not contain any markdown, bullets, numbering, or formatting - plain text only
- ...

Nothing tells the model to respond with only the excerpt and nothing else.

Proposed fix

Add an explicit output-format constraint to the system instruction, before the requirements list:

Return ONLY the excerpt text itself. Do not include any preamble, introduction,
explanation, or meta-commentary such as "Here's an excerpt..." or "This excerpt
captures..." - respond with nothing but the excerpt.

And add one more requirement:

- Do not wrap the excerpt in quotation marks

This is a common pattern with LLMs — they default to being "helpful" by explaining their output unless explicitly told not to. The title generation prompt (Title_Generation/system-instruction.php) has the same gap but gets away with it more often because titles are inherently short and less prone to commentary.

Happy to open a PR with this fix if helpful.

[dkotter]

This is a fairly common thing to see when using smaller models, like those you may run locally. Definitely open to reviewing a PR that updates our system instructions to try and prevent this