Skip to content

Support extracting extras from profiles and addresses - #2907

Draft
noisysocks wants to merge 3 commits into
mainfrom
randerson/pir-extras-open-field
Draft

Support extracting extras from profiles and addresses#2907
noisysocks wants to merge 3 commits into
mainfrom
randerson/pir-extras-open-field

Conversation

@noisysocks

@noisysocks noisysocks commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Asana Task/Github Issue: https://app.asana.com/1/137249556945/task/1216773212741748

Tech Design: https://app.asana.com/1/137249556945/project/1209121419454298/task/1216773212741757

Description

Adds support for extracting extras from profiles and addresses.

This solves the immediate need to extract .street and .zip to fix PeopleFinders. In the medium term it gives us more flexibility when fixing rules that require filling a form with extra info from an extracted profile — capturing a new field becomes a config change rather than a code change.

Extras are omitted when empty, and their absence is tolerated everywhere. Profile hashing strips extras and collapses addresses back to one per city/state, so capturing them never churns an existing broker identifier.

Also adds support for filling out a fake DOB using $generated_dob$.

Testing Steps

  • Integration tests (npm run test-int) and unit tests (npm run test-unit) cover extraction and form filling for both profile- and address-level extras
  • Manual testing steps TBD

Checklist

Please tick all that apply:

  • I have tested this change locally
  • I have tested this change locally in all supported browsers
  • This change will be visible to users
  • I have added automated tests that cover this change
  • I have ensured the change is gated by config
  • This change was covered by a ship review
  • This change was covered by a tech design
  • Any dependent config has been merged

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Build Branch

Branch pr-releases/randerson/pir-extras-open-field
Commit 386662f3b0
Updated July 28, 2026 at 4:57:31 AM UTC

Static preview entry points

QR codes (mobile preview)
Entry point QR code
Docs QR for docs preview
Static pages QR for static pages preview
Integration pages QR for integration pages preview

Integration commands

npm (Android / Extension):

npm i github:duckduckgo/content-scope-scripts#pr-releases/randerson/pir-extras-open-field

Swift Package Manager (Apple):

.package(url: "https://github.com/duckduckgo/content-scope-scripts.git", branch: "pr-releases/randerson/pir-extras-open-field")

git submodule (Windows):

git -C submodules/content-scope-scripts fetch origin pr-releases/randerson/pir-extras-open-field
git -C submodules/content-scope-scripts checkout origin/pr-releases/randerson/pir-extras-open-field
Pin to exact commit

npm (Android / Extension):

npm i github:duckduckgo/content-scope-scripts#386662f3b0546aa97812f880b8e21aed52356fa6

Swift Package Manager (Apple):

.package(url: "https://github.com/duckduckgo/content-scope-scripts.git", revision: "386662f3b0546aa97812f880b8e21aed52356fa6")

git submodule (Windows):

git -C submodules/content-scope-scripts fetch origin pr-releases/randerson/pir-extras-open-field
git -C submodules/content-scope-scripts checkout 386662f3b0546aa97812f880b8e21aed52356fa6

@github-actions github-actions Bot added the semver-minor New feature — triggers minor version bump label Jul 27, 2026
Adds an open `extras: Record<string, string>` to extracted profiles and to
each of their addresses, so capturing a new field is a config change rather
than a code change. `fillForm` can then read those values back out.

The immediate need is `street` and `zip` to fix PeopleFinders.

Extras are omitted when empty, and absent extras are tolerated everywhere.
Profile hashing strips extras and collapses addresses back to one per
city/state, so capturing them never churns an existing broker identifier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noisysocks
noisysocks force-pushed the randerson/pir-extras-open-field branch from 06bee5b to c61f13f Compare July 27, 2026 01:57
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

[Beta] Generated file diff

Time updated: Tue, 28 Jul 2026 04:59:18 GMT

Android
    - android/autofillImport.js
  • android/brokerProtection.js

File has changed

Apple
    - apple/contentScopeIsolated.js

File has changed

Integration
    - integration/contentScope.js

File has changed

Windows
    - windows/contentScope.js

File has changed

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

File Lines Severity Finding
injected/src/features/broker-protection/** info No browser API overrides, prototype patches, wrapper-utils/DDGProxy changes, or global modifications. All changes are confined to broker-protection extraction, aggregation, and form-fill logic — not general page injection paths.
injected/src/features/broker-protection/extractors/address.js 107 info extractAddressFull now normalizes city via titleCase and state via normalizeState. Output shape changes vs raw DOM text, but comparisons use matchingPair / sameCityState (case-insensitive), so address matching behavior should remain stable.
injected/src/features/broker-protection/actions/extract.js 433–484 info dedupeAddresses replaces a simple city/state Map dedup with logic that prefers richer addresses (with extras) over bare city/state duplicates. Behavioral change is scoped to broker profile aggregation.
injected/src/features/broker-protection/actions/fill-form.js 50–51, 88–98, 154–158 info Form fill now resolves city/state and custom extras from the selected address when flat profile keys are absent. Extends existing fillForm DOM interaction patterns; no new unbounded observers, timers, or viewport/meta manipulation.
injected/integration-test/broker-protection-tests/broker-protection.spec.js + unit tests info Substantial integration and unit test coverage for extras extraction (text/title/href), address extras, fill-form extras lookup, and omitExtrasForHash identifier stability.

No web-compatibility warnings or errors identified.

Security Assessment

File Lines Severity Finding
injected/src/features/broker-protection/actions/extract.js 257 info Uses Object.prototype.hasOwnProperty.call(extractors, field) for extractor lookup — avoids polluted hasOwnProperty on the prototype chain. Matches existing broker-protection hygiene (isNestedCityStateSpec, normalizeState).
injected/src/features/broker-protection/execute.js 22–25 info fillForm reads userProfile directly from inputData (not data()), avoiding action.dataSource accidentally routing extracted profile data into the user-profile slot. Correct trust-boundary separation for native-provided vs extracted data.
injected/src/features/broker-protection/extractors/profile-url.js 137–151 info omitExtrasForHash strips extras at profile and address levels and collapses addresses to one per city/state before hashing — prevents identifier churn when extras are added. Positive for profile-matching stability.
injected/src/features/broker-protection/actions/fill-form.js 51, 154–158 info Page-derived extras flow into form fields via lookupValue. This follows the existing broker-protection trust model (extract broker DOM → fill opt-out forms on the same broker site). No new messaging, postMessage, or cross-origin data paths.
injected/src/features/broker-protection/extractors/extra.js (new) info New source file under injected/src/ is not yet listed in CORE_FILES (scripts/check-strict-core.js). Process/coding-standard gap, not a runtime security issue.

No security warnings, errors, or critical findings.

Risk Level

Low Risk — Broker-protection-only changes with no API shimming, messaging transport changes, captured-globals modifications, or message-bridge boundary changes; extensive test coverage added.

Recommendations

  1. Process (low): Add injected/src/features/broker-protection/extractors/extra.js to CORE_FILES in scripts/check-strict-core.js per strict-TypeScript policy for new injected/src/ files.
  2. Defense-in-depth (low): Consider asserting extras values are strings in createProfile / lookupValue before filling forms, since ProfileExtras documents string-only values.
  3. No action required on API fidelity, messaging security, or config gating — changes appropriately scoped to PIR automation.
Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Cursor review was not successful.

This PR requires a manual review and approval from a member of one of the following teams:

  • @duckduckgo/content-scope-scripts-owners
  • @duckduckgo/apple-devs
  • @duckduckgo/android-devs
  • @duckduckgo/team-windows-development
  • @duckduckgo/extension-owners
  • @duckduckgo/config-aor
  • @duckduckgo/breakage-aor
  • @duckduckgo/breakage

Adds a `$generated_dob$` fillForm element type that fills an
`<input type="date">` with a random date of birth consistent with the age on
the extracted profile, resolved through the same lookup chain as every other
field.

Also replaces the real-looking street addresses in the broker-protection test
fixtures with obviously synthetic ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Injected PR Evaluation: Web Compatibility & Security

Re-assessed on synchronize (641f8462f). Scope remains broker-protection only — no browser API overrides, messaging transports, captured-globals, or message-bridge changes.

New since last review (c61f13f3): $generated_dob$ fill-form element type and generateDateOfBirth() (commit 641f8462f), with unit + integration coverage including leap-day handling.


Web Compatibility Assessment

File Lines Severity Finding
fill-form.js 131–134 info if (!value) treats empty string as absent, so present-but-empty optional fields are skipped rather than erroring. Intentional (see fill-form-optional-empty test); behavior change vs prior strict empty-string rejection. No general-site impact — broker opt-out forms only.
fill-form.js 131 info Falsy check also treats 0 as missing. Unlikely for PII field types; worth noting if numeric extras are ever added.
extract.js 446–461 info dedupeAddresses retains multiple same-city/state rows when extras differ (e.g. distinct streets). Downstream matching (scrapedDataMatchesUserData) correctly ignores extras per unit tests.
address.js titleCase info extractAddressFull now title-cases city names from parse-address. sameCityState / matchingPair comparisons remain case-insensitive, so no matching regression expected.
generators.js 29–37 info generateDateOfBirth uses uncaptured Date (same pattern as existing generators). No browser API shimming; only invoked for native-configured $generated_dob$ elements on data-broker opt-out pages.

No findings at warning or error severity for general web compatibility.


Security Assessment

File Lines Severity Finding
extra.js (new file) info Not added to CORE_FILES in scripts/check-strict-core.js. Process gap per injected coding standards; no runtime security impact.
fill-form.js 939–940, 164–168 info Form values resolved from page-scraped extractedProfile + extras. Expected trust model for broker automation; native config gates which sites/actions run. No new exfiltration or messaging paths.
execute.js fillForm case info userProfile read from inputData directly (not data()), preserving correct data-source boundary. Good hygiene.
profile-url.js 137–150 info omitExtrasForHash strips extras and collapses addresses for identifier stability. Hash dedup key is case-sensitive (${city},${state}) while runtime dedup is case-insensitive — minor inconsistency, not exploitable.
extract.js 801 info Object.prototype.hasOwnProperty.call(extractors, field) avoids prototype-pollution false positives on the extractor map. Good defensive pattern.

No findings at warning, error, or critical severity.


Risk Level

Low Risk — Scoped to broker-protection extract/fill-form logic and tests; no changes to injected API shims, messaging security boundaries, or page-wide prototype/DOM behavior.


Recommendations

  1. (info) Add injected/src/features/broker-protection/extractors/extra.js to CORE_FILES to satisfy strict-TypeScript policy for new source files.
  2. (info) Consider aligning omitExtrasForHash address dedup with cityStateKey() lowercasing for consistency (optional; current tests pass).
  3. (info) No additional web-compat or security blockers identified for merge from an injected-runtime perspective.
Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-minor New feature — triggers minor version bump

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant