-
Notifications
You must be signed in to change notification settings - Fork 406
Anonymization app #2368
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Anonymization app #2368
Changes from 11 commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
381f99e
new app
lkostrowski 4d8a876
wip
lkostrowski 7dd2617
wip permissions
lkostrowski 517290a
scrable address
lkostrowski 6d5dcd9
no-op apl
lkostrowski 7a091fd
bulk operations support
lkostrowski 2a66365
Potential fix for pull request finding
lkostrowski 1e1e30c
cr fixes
lkostrowski eecd780
cr fixes
lkostrowski e705695
fix linking types node
lkostrowski c84b55b
cr fixes
lkostrowski a3a71b6
symlink for schema
lkostrowski d59d3f0
fix flaky test
lkostrowski cb08651
Merge branch 'main' into lkostrowski/clone-anonymization-app
lkostrowski 2bc706f
Merge branch 'main' into lkostrowski/clone-anonymization-app
lkostrowski 590805c
Merge branch 'main' into lkostrowski/clone-anonymization-app
lkostrowski aab36d1
cr fixes
lkostrowski 166e41b
Merge remote-tracking branch 'origin/main' into lkostrowski/clone-ano…
lkostrowski 2652da8
fix conflict leftover
lkostrowski cd71630
disable re scan cache
lkostrowski File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "saleor-app-anonymizer": minor | ||
| --- | ||
|
|
||
| Added the Anonymizer app to the monorepo. It lets you anonymize a customer's personal data from the Saleor Dashboard: it scrambles the billing and shipping details of all the customer's orders (names and phone cleared, street replaced with a placeholder, email replaced with a random address under a configurable domain, while city/postal code/country are kept) and then deletes the customer. The app now uses the shared monorepo tooling (ESLint, TypeScript and dependency catalog) and observability stack (Sentry, OpenTelemetry, structured logging, env validation). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| --- | ||
| "saleor-app-anonymizer": patch | ||
| --- | ||
|
|
||
| Fixed single-customer anonymization. Two issues are addressed: | ||
|
|
||
| - Only the first 100 of a customer's orders were fetched and anonymized, so a customer with more than 100 orders kept personal data on the remaining ones. The app now paginates through every page of the customer's orders before anonymizing them. | ||
| - A customer who had never placed an order could not be erased at all. The account is now deletable even when it has no orders, so erasure requests for those customers can be fulfilled. |
|
NyanKiyoshi marked this conversation as resolved.
Outdated
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "saleor-app-anonymizer": minor | ||
| --- | ||
|
|
||
| Added a bulk anonymization section. Before, the app could only anonymize a single customer looked up by email. Now the app can also scan the whole store and show how many non-anonymized orders and how many customers (excluding staff accounts) it found, then anonymize all those orders or delete all those customers in one click, with a progress bar. Each anonymized order is marked with the `saleor-anonymized: true` metadata, so already-processed orders are skipped on subsequent scans and failed ones are retried. Records that failed to process are listed as links that open in a new Dashboard tab. The app page was also reorganized into sections with explanatory text, matching the layout of other Saleor apps. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| APP_LOG_LEVEL=info | ||
|
|
||
| # Variable controlling Saleor domains that app can be installed on. | ||
| # It's a regex pattern tested against the Saleor API URL. Use `.*` to allow all. | ||
| ALLOWED_DOMAIN_PATTERN=.* | ||
|
|
||
|
lkostrowski marked this conversation as resolved.
|
||
| # Domain used to build anonymized customer emails (e.g. <uuid>@example.com). | ||
| # Exposed to the browser, so it must be prefixed with NEXT_PUBLIC_. | ||
| NEXT_PUBLIC_CUSTOMER_SCRAMBLE_DOMAIN=example.com | ||
|
|
||
| # How many orders/customers the bulk anonymization processes concurrently. | ||
| # Exposed to the browser, so it must be prefixed with NEXT_PUBLIC_. | ||
| # NEXT_PUBLIC_BULK_CONCURRENCY=5 | ||
|
|
||
| # Local development variables. When developed locally with Saleor inside docker, these can be set to: | ||
| # APP_IFRAME_BASE_URL = http://localhost:3000, so Dashboard on host can access iframe | ||
| # APP_API_BASE_URL=http://host.docker.internal:3000 - so Saleor can reach App running on host, from the container. | ||
| # If developed with tunnels, set this empty, it will fallback to default Next's localhost:3000 | ||
| # https://docs.saleor.io/developer/extending/apps/local-app-development | ||
| # APP_IFRAME_BASE_URL= | ||
| # APP_API_BASE_URL= | ||
|
|
||
| # OTEL related variables | ||
| # OTEL_ENABLED=true | ||
| # OTEL_SERVICE_NAME=saleor-app-anonymizer | ||
| # OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 # OTEL collector endpoint (http protocol) | ||
| # OTEL_ACCESS_TOKEN=token # token used to authenticate with OTEL collector | ||
| # NEXT_RUNTIME=nodejs | ||
|
|
||
| # Sentry error tracking | ||
| # NEXT_PUBLIC_SENTRY_DSN= | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| generated | ||
| graphql/schema.graphql |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| <div align="center"> | ||
| <img width="150" alt="saleor-app-anonymizer" src="https://user-images.githubusercontent.com/4006792/215185065-4ef2eda4-ca71-48cc-b14b-c776e0b491b6.png"> | ||
| </div> | ||
|
|
||
| <div align="center"> | ||
| <h1>Saleor Anonymizer App</h1> | ||
| </div> | ||
|
|
||
| <div align="center"> | ||
| <p>Anonymize customer data directly from the Saleor Dashboard.</p> | ||
| </div> | ||
|
|
||
| --- | ||
|
|
||
| ## Overview | ||
|
|
||
| The Anonymizer App helps anonymize a customer's personal data: | ||
|
|
||
| - Looks up a user and all of their orders by email. | ||
| - Scrambles the order billing and shipping details: | ||
| - First name / last name → cleared (set to an empty value). | ||
| - Phone → cleared (set to an empty value). Saleor accepts an empty phone and skips | ||
| validation, whereas a fake number would fail validation and would not be anonymous. | ||
| - Street address → replaced with a constant placeholder (`Anonymized`). | ||
| - City, postal code and country → kept intact, so the address stays valid. | ||
| - Email → replaced with a random `UUID`-based address under a configurable domain. | ||
|
lkostrowski marked this conversation as resolved.
lkostrowski marked this conversation as resolved.
|
||
| - Deletes the customer profile once all of their orders are anonymized. | ||
|
|
||
| The configurable scramble domain is controlled by the | ||
| `NEXT_PUBLIC_CUSTOMER_SCRAMBLE_DOMAIN` environment variable. | ||
|
|
||
| ### Bulk anonymization | ||
|
|
||
| The app can also process the whole store at once: | ||
|
|
||
| - A **Scan** walks all orders and customers and shows how many will be processed. | ||
| The results are kept in memory, so the actions below run without re-fetching. | ||
| - **Anonymize orders** scrambles every order that does not yet carry the | ||
| `saleor-anonymized: true` public metadata flag, then writes the flag. Orders that | ||
| already carry it are skipped, so re-runs are idempotent and failed orders are | ||
| retried on the next run (the flag is written only after a successful scramble). | ||
| - **Delete customers** deletes every non-staff customer account. Staff accounts are | ||
| never counted or deleted. | ||
| - Everything runs in the browser with a progress bar; records that failed to process | ||
| are listed as links opening in a new Dashboard tab. | ||
|
|
||
| The number of records processed concurrently is controlled by the | ||
| `NEXT_PUBLIC_BULK_CONCURRENCY` environment variable (default: 5). | ||
|
|
||
| ## Required permissions | ||
|
|
||
| - `MANAGE_ORDERS` | ||
| - `MANAGE_USERS` | ||
|
|
||
| ## Development | ||
|
|
||
| This app lives in the [saleor/apps](https://github.com/saleor/apps) monorepo and follows the | ||
| shared tooling (ESLint, TypeScript and dependency catalog) used across all apps. | ||
|
|
||
| ```bash | ||
| # from the repository root | ||
| pnpm install | ||
|
|
||
| # run only this app | ||
| pnpm --filter saleor-app-anonymizer dev | ||
|
|
||
| # generate GraphQL types | ||
| pnpm --filter saleor-app-anonymizer generate | ||
|
|
||
| # type-check / lint / test | ||
| pnpm --filter saleor-app-anonymizer check-types | ||
| pnpm --filter saleor-app-anonymizer lint | ||
| pnpm --filter saleor-app-anonymizer test | ||
| ``` | ||
|
|
||
| Copy `.env.example` to `.env` and fill in the required values before starting the app. | ||
|
|
||
| ## Authentication | ||
|
|
||
| This app runs entirely in the Dashboard iframe and uses the token it receives from the App Bridge | ||
| for every request to Saleor. It has no backend that needs to call Saleor on its own, so it does not | ||
| persist any auth data and needs no related configuration. To satisfy the App SDK, it plugs in a | ||
| `NoopAPL` (`src/lib/noop-apl.ts`) that stores nothing. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import { config } from "@saleor/eslint-config-apps/index.js"; | ||
| import nodePlugin from "eslint-plugin-n"; | ||
|
|
||
| /** @type {import("eslint").Linter.Config} */ | ||
| export default [ | ||
| ...config, | ||
| { | ||
| name: "saleor-app-anonymizer/custom-config", | ||
| files: ["**/*.ts"], | ||
| plugins: { | ||
| n: nodePlugin, | ||
| }, | ||
| rules: { | ||
| "n/no-process-env": "error", | ||
| }, | ||
| }, | ||
| { | ||
| name: "saleor-app-anonymizer/override-no-process-env", | ||
| files: ["next.config.ts", "src/env.ts", "src/instrumentation.ts", "src/instrumentations/*.ts"], | ||
| rules: { | ||
| "n/no-process-env": "off", | ||
| }, | ||
| }, | ||
| { | ||
| // TODO: remove this override once the recommended rules are fixed | ||
| name: "saleor-app-anonymizer/override-recommended", | ||
| files: ["**/*.{ts,tsx}"], | ||
| rules: { | ||
| "no-fallthrough": "warn", | ||
| "@typescript-eslint/no-unused-vars": "warn", | ||
| "@typescript-eslint/no-namespace": "warn", | ||
| "@typescript-eslint/no-empty-object-type": "warn", | ||
| }, | ||
| }, | ||
| ]; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.