Skip to content

Commit 7da3c18

Browse files
committed
fix(safety): a fake verification gate is the payload, not a wall
Seven of eighteen medium-confidence scam verdicts were hand-drawn "Security Check / I'm not a robot" cards on days-old numeric or .cfd/.shop throwaway domains with no MX and nothing at the root. The model described them accurately, named ClickFix in its own reason, and said medium, so under the confident policy nothing happened. It was following the prompt. The only confidence rule was "high means you would stake the block on it", and every high example showed the model seeing the payload directly. Nothing said the gate on a young spoofing domain IS the payload. One rule in the cloaking guidance and a worked example built from 5347567.shop, the one such host whose gate is still up, close that gap. A real provider challenge on an established domain stays missing evidence. Side-by-side on the same 17 hosts with real tools and the real model, enforcement off: scam-class high went 5 to 7, the live gate went medium to high, and nothing benign or gray was condemned. Six of the seven gate hosts had already died, so the rule could only be exercised on one; the model also shows run-to-run variance regardless of prompt. A test pins the rule and example so a prompt edit cannot drop them quietly.
1 parent ac449b2 commit 7da3c18

2 files changed

Lines changed: 49 additions & 0 deletions

File tree

services/safety/prompts/investigate_v1.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ Judge by these pairs — this is where the errors happen:
6565
- A redirect proves NOTHING on its own — legitimate services redirect constantly.
6666
- A form posting a password cross-origin, from a domain days younger than the brand it imitates, is strong evidence of phishing.
6767
- A render served from a datacenter/scanner egress that looks clean while the report text is specific and damning is a CLOAKING hypothesis, not an acquittal — good kits serve scanners a clean page. Weigh the report against the clean render; do not treat "looked fine" as proof.
68+
- A fake verification gate is the payload, not a wall in front of it. This needs TWO things together. First, you RENDERED the gate: a "Security Check", "I'm not a robot", "verify you are human" or "checking your browser" card that is NOT a real challenge provider (no reCAPTCHA, hCaptcha or Turnstile script host, no `cdn-cgi/challenge-platform`). A page that failed to render, or a dead 404, is not a gate; it is missing evidence and stays `uncertain`. Second, at least two of: domain registered or first certificate within the last ~30 days; a numeric or random-string hostname; a brand name imitated in the host or at the root; no MX; no legitimate content at the root or the reported path. With both, the gate itself is the social-engineering step (ClickFix). You have seen the harm. That is `scam_host`, `high`, and the screenshot is your evidence; do not downgrade to `medium` because you "could not see behind" it. Name the signals you counted in `evidence`. Without the second part, a fake gate on an established domain with a real root is `compromised_legit` or `uncertain`, never `scam_host` `high`. A REAL provider challenge on an established domain (a Cloudflare "Just a moment..." with the challenge-platform script, on a domain with history) is still missing evidence, as before.
6869
- A hard hit from `feed_lookup` (a threat feed or Web Risk) on the terminal host is corroborating external evidence.
6970
- Absence of evidence is not benignity. "The render failed" or "no feed hit" does not make a reported host safe.
7071

@@ -118,6 +119,10 @@ Verdict: `spam_gray`, `medium`. Reason: "Aggressive affiliate content with ad wa
118119
Bundle: reported for phishing, but `fetch_page` from the datacenter egress returns a plain "under maintenance" page; `resolve_chain` shows no redirect; `domain_intel` shows a 2-year-old domain with normal infrastructure; `feed_lookup` is clean. The report reason is specific ("stole my bank login") but nothing on the page corroborates it.
119120
Verdict: `uncertain`, `low`. Reason: "Specific phishing report but the render is a maintenance page from a scanner IP — possible cloaking, cannot confirm." Evidence: ["specific bank-phishing report", "render shows only a maintenance page", "clean render came from a datacenter egress"]. Scope: `host`. — A clean render from a scanner IP against a specific report is a cloaking hypothesis, not an acquittal; when you cannot separate the readings, say uncertain and let a human look.
120121

122+
**Example 10 — the fake gate IS the scam (ClickFix).**
123+
Bundle: sweep-found link to `https://5347567.shop/197721937`. `fetch_page` renders a centred white card: a shield icon, "Security Check", "Please confirm you are not a robot to continue", one large button reading "I'm not a robot ✓", footer "Protected by SSL · 3-D Secure". No form, no external script host except analytics, no reCAPTCHA or hCaptcha or Turnstile. The screenshot shows a hand-drawn card, not a provider widget. Root `https://5347567.shop/` returns "Cannot GET /". `domain_intel`: certificate issued 2 days ago, no MX, RDAP unavailable. `host_usage`: two links, one anonymous creator.
124+
Verdict: `scam_host`, `high`. Reason: "Rendered a hand-drawn 'I'm not a robot' card with no challenge provider, on a numeric domain whose first certificate is 2 days old, with no MX and nothing at the root: the gate is the ClickFix lure." Evidence: ["rendered fake verification card, no captcha provider script", "'3-D Secure' misused on a robot check", "numeric-string hostname", "first TLS certificate 2 days old (RDAP unavailable)", "no MX", "root returns 'Cannot GET /'"]. Scope: `host`. — Gate rendered, plus four of the second-part signals (numeric host, 2-day cert, no MX, empty root). Contrast with Example 9: there the gate was a real maintenance page on a 2-year-old domain and the harm was only alleged. Here the domain has no history, no product, and the card is the harm. `medium` would be the wrong answer; you are not waiting to see a payload, you are looking at it.
125+
121126
# Output
122127

123128
Return the structured verdict:

tests/unit/services/safety/test_investigation.py

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -706,3 +706,47 @@ def test_a_proposal_for_an_unknown_list_fails_parse(self):
706706

707707
with pytest.raises(pydantic.ValidationError):
708708
ListProposal(list="manual", domain="x.example", why="t")
709+
710+
711+
class TestPromptCarriesTheClickFixRule:
712+
"""Seven of eighteen medium-confidence scam calls were fake verification
713+
gates on young domains. The prompt had no rule letting the model treat
714+
the gate as the payload, so it obeyed the only high examples it had and
715+
hedged. This pins the rule so a prompt edit cannot quietly drop it."""
716+
717+
def test_rule_and_worked_example_are_present(self):
718+
from services.safety.investigation import _DEFAULT_PROMPT as p
719+
720+
assert "A fake verification gate is the payload" in p
721+
assert "ClickFix" in p
722+
assert "Example 10" in p
723+
assert "5347567.shop" in p
724+
# The counter-case stays: a real provider challenge is still missing evidence.
725+
assert "cdn-cgi/challenge-platform" in p
726+
727+
def test_rule_requires_a_rendered_non_provider_gate_plus_two_signals(self):
728+
"""One weak property must never be enough. The first draft used 'or'
729+
and a dead page went high on hostname shape alone."""
730+
from services.safety.investigation import _DEFAULT_PROMPT as p
731+
732+
rule = p[p.index("A fake verification gate is the payload") :]
733+
rule = rule[: rule.index("\n")]
734+
assert "needs TWO things together" in rule
735+
assert "you RENDERED the gate" in rule
736+
assert "NOT a real challenge provider" in rule
737+
assert "failed to render, or a dead 404, is not a gate" in rule
738+
assert "at least two of" in rule
739+
assert "never `scam_host` `high`" in rule
740+
assert "Name the signals you counted" in rule
741+
742+
def test_example_10_claims_only_what_its_bundle_observed(self):
743+
from services.safety.investigation import _DEFAULT_PROMPT as p
744+
745+
ex = p[p.index("**Example 10") :]
746+
ex = ex[: ex.index("\n\n") if "\n\n" in ex else None]
747+
assert "Verdict: `scam_host`, `high`" in ex
748+
assert "first TLS certificate 2 days old (RDAP unavailable)" in ex
749+
assert "root returns 'Cannot GET /'" in ex
750+
assert "plus four of the second-part signals" in ex
751+
# It must not overclaim coverage it did not check.
752+
assert "on any path" not in ex

0 commit comments

Comments
 (0)