Context
Kosh v2's reachability gate (PR #16) does a great job catching the three WordPress.com-native gates:
- Coming Soon splash (
bodyClass ~= wpcom-coming-soon-body)
- Password prompt (
bodyClass ~= login-password-protected or URL password-protected=login)
- Private site (
bodyClass ~= private-login or title === "Private Site")
The gap
Partners who use custom themes for pre-launch messaging bypass all three body-class checks. Example: https://neighborhoodnip.wpcomstaging.com returns HTTP 200 with:
- Real page titles set correctly per URL
- WooCommerce plugin active with real product structure
- Per-page canonical tags working
- Inner pages (
/about, /shop, /contact, /donate) all navigate cleanly
…but every visible page has the same "PARDON OUR DUST!" placeholder heading and no other content. Structurally it's a working WordPress site with awful content, so kosh grinds through the full 49-signal AEO audit and reports "40 of 49 fail because there's no content." The output is technically accurate but not useful — the user was trying to audit a real site.
I hit this yesterday running /kosh:aeo from Smithers. The gated-site heuristic didn't fire; kosh produced an 11-minute AEO run with a report full of "site has no content" findings.
Proposed heuristic
Add a Phase 0 / Phase 2 check for placeholder-only content patterns:
- After the first two Phase 2 page visits, compare their visible body text (or normalized H1s).
- If both pages return the same H1 (or the same visible-text hash, or 95%+ Jaccard similarity on body text), treat as a placeholder-only site.
- Emit a
[SMITHERS_GATE:placeholder]-style marker (or whatever unattended-mode signal you settle on) and stop with "site content not yet populated — re-run once the site has real content."
Alternative (simpler): if 3+ Phase 2 pages all have the same H1 tag content, flag as placeholder.
Why bother
Two real-world cases where this matters:
- Partners in the "site skeleton set up, waiting on copy" phase — TAMs running kosh to check structural readiness get a wall of "no content" findings that mask any actual technical issues worth flagging.
- Automated audits (e.g. Smithers's queue-all flow) burn tokens + time on sites that literally can't be scored yet.
Happy to help scope the heuristic if useful — the JSON from my Neighborhood Nip run is a good ground-truth example if you want it as a test case.
/cc @katodea @lmischner
Context
Kosh v2's reachability gate (PR #16) does a great job catching the three WordPress.com-native gates:
bodyClass ~= wpcom-coming-soon-body)bodyClass ~= login-password-protectedor URLpassword-protected=login)bodyClass ~= private-loginortitle === "Private Site")The gap
Partners who use custom themes for pre-launch messaging bypass all three body-class checks. Example:
https://neighborhoodnip.wpcomstaging.comreturns HTTP 200 with:/about,/shop,/contact,/donate) all navigate cleanly…but every visible page has the same "PARDON OUR DUST!" placeholder heading and no other content. Structurally it's a working WordPress site with awful content, so kosh grinds through the full 49-signal AEO audit and reports "40 of 49 fail because there's no content." The output is technically accurate but not useful — the user was trying to audit a real site.
I hit this yesterday running
/kosh:aeofrom Smithers. The gated-site heuristic didn't fire; kosh produced an 11-minute AEO run with a report full of "site has no content" findings.Proposed heuristic
Add a Phase 0 / Phase 2 check for placeholder-only content patterns:
[SMITHERS_GATE:placeholder]-style marker (or whatever unattended-mode signal you settle on) and stop with "site content not yet populated — re-run once the site has real content."Alternative (simpler): if 3+ Phase 2 pages all have the same H1 tag content, flag as placeholder.
Why bother
Two real-world cases where this matters:
Happy to help scope the heuristic if useful — the JSON from my Neighborhood Nip run is a good ground-truth example if you want it as a test case.
/cc @katodea @lmischner