Skip to content

Commit 92ebc87

Browse files
humanagentclaude
andcommitted
fix(pii): don't redact bare URLs in template redaction
buildRedactionPrompt's catch-all "similar identifiers" phrasing let the model sweep up plain links (booking pages, docs, social) as PII, even though a URL isn't personal data on its own. Adds an explicit carve-out mirroring the public-figure fix (#351), plus stress-harness cases for a bare-URL template and a URL with a genuinely embedded identifier (mailto:) that should still be caught. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 45ef6f7 commit 92ebc87

2 files changed

Lines changed: 20 additions & 0 deletions

File tree

src/api/v2/agent-templates/services/moderation.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -425,6 +425,8 @@ Rules:
425425
- Do NOT flag the assistant's OWN persona name: the invented first name or handle the template gives the agent itself (e.g. "You are Emma, a friendly yoga coach"). That is the product's identity, not a private third party.
426426
- Do NOT flag the names of well-known PUBLIC figures — business leaders, founders, CEOs, investors, politicians, athletes, entertainers, authors, or historical figures (e.g. "Patrick Collison", "Warren Buffett", "Taylor Swift"). Their names are already public, and a template may legitimately be built around one (recommending books like Patrick Collison, writing in the style of a famous author, etc.).
427427
- DO flag a personal name when it belongs to a PRIVATE individual — a non-famous person whose name is not otherwise public (a customer, patient, client, employee, colleague, family member, or friend). If a name happens to match a public figure's but the surrounding context clearly refers to a private individual (e.g. "email our customer Taylor Swift", "our new hire Warren Buffett"), treat it as private and flag it.
428+
- Do NOT flag bare URLs or web links (e.g. "https://calendly.com/jane-doe", a company site, a booking or social link) — a link is not personal data on its own, even if a slug, subdomain, or username in the path looks like a name. Templates legitimately reference websites, docs, and booking pages.
429+
- DO flag a URL if it embeds a structured identifier that would independently qualify — an email address, phone number, or physical address appearing literally within the URL (e.g. a mailto: link, or a query parameter carrying an email/phone). Flag only that embedded span, not the whole link, when the rest of the URL is separable.
428430
- Structured identifiers (email, phone, street/physical address, card/SSN/IBAN/account numbers) are ALWAYS PII and must always be flagged, regardless of whose they are.
429431
- If there is no PII, return an empty list.
430432

tests/pii-stress.manual.test.ts

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,24 @@ const CASES: { name: string; fields: RedactableFields }[] = [
124124
"Greet visitors and book slots. For VIP handling, flag our regular customer Taylor Swift, and route billing questions to our new hire Warren Buffett in accounting.",
125125
},
126126
},
127+
{
128+
name: "bare URLs/links — should NOT be redacted",
129+
fields: {
130+
agentName: "Scheduling Buddy",
131+
description: "Helps users book time and find resources online",
132+
prompt:
133+
"If someone wants to book time, send them to https://calendly.com/jane-doe. Our docs live at https://docs.example.com/getting-started, and our team's Twitter is https://twitter.com/janedoe123.",
134+
},
135+
},
136+
{
137+
name: "URL with embedded email — SHOULD redact only the embedded identifier",
138+
fields: {
139+
agentName: "Support Bot",
140+
description: "Handles support requests",
141+
prompt:
142+
"For escalations, use this mailto link: mailto:support-lead@example.com or the contact form at https://example.com/contact?ref=agent.",
143+
},
144+
},
127145
];
128146

129147
describe.runIf(KEY)("redactTemplatePii — live stress", () => {

0 commit comments

Comments
 (0)