fix: clean up ClickHouse analytics rows on user/team/website/link/pixel deletion

[anvme]

Third in the deletion-cleanup series after #4243 (link/pixel/board orphans on user/team delete) and #4245 (cascade websites + soft-delete leaks). Stacks on top of #4245 — until those merge, the GitHub diff includes their commits too. For incremental review now, focus on commit 871aa3b.

Problem

No delete path in the codebase touched ClickHouse before this. With CLICKHOUSE_URL set (the typical high-volume setup), every delete left analytics events orphaned forever:

• Deleting a website left its events in website_event, event_data, session_data, session_replay, website_revenue, website_event_stats_hourly, event_data_pivot.
• Deleting a link/pixel was even subtler: /api/send writes link.id and pixel.id into the website_id column for /q/<slug> and /p/<slug> events (src/app/api/send/route.ts:98), so link/pixel events orphan under their entity id, not a website id.

Solution

File	Change
src/lib/clickhouse.ts	New deleteByWebsiteIds(ids) helper. Issues ALTER TABLE ... DELETE WHERE website_id IN (...) against all 7 CH tables in parallel via client.command(). ClickHouse Materialized Views are insert-time triggers and do not cascade mutations, so the 3 MV target tables (website_revenue, website_event_stats_hourly, event_data_pivot) need explicit DELETEs.
deleteWebsite / deleteUser / deleteTeam / deleteLink / deletePixel	Call the helper after the Prisma transaction commits, non-cloud only. Cloud mode preserves CH analytics for billing/audit, matching the existing soft-delete retention philosophy.

For deleteUser / deleteTeam, the cleanup ID set is websites + links + pixels (boards generate no analytics events).

Why ALTER TABLE DELETE instead of lightweight DELETE FROM?

Lightweight DELETE FROM is generally recommended by CH docs, BUT it throws by default on tables with projections (website_event has 2 projections, session_data has 1). ALTER TABLE DELETE works uniformly across all 7 tables without per-query setting tweaks. Trade-off: heavier mutation, async by default — acceptable for the rare delete-on-erasure use case.

Failure handling

await and propagate (matches the existing Redis invalidation pattern). If CH is briefly unreachable, the function rejects after PG has already committed — same dual-write race as Redis. Outbox/retry pattern is flagged as a separate PR for closing that race across both Redis and CH consistently.

Test plan

• pnpm build-app clean
• E2E Test 1 (deleteWebsite): created website, fired 3 events via /api/send, deleted website. CH state: website_event 3 -> 0, website_event_stats_hourly 3 -> 0, all 7 mutations is_done=1 in system.mutations. Both projection tables (website_event, session_data) processed cleanly.
• E2E Test 2 (deleteLink with link.id used as website_id): created link, hit /q/<slug> to fire events, deleted link. website_event for link.id: 2 -> 0.
• E2E Test 3 (deleteUser non-cloud): created user owning a website + a link, fired events for both, deleted user. Both website-events and link-events: cleared.

Out of scope (separate PRs / discussion)

• Outbox/retry: durable cleanup record so partial failures (PG committed, CH unreachable) eventually retry and converge. Would also benefit Redis invalidation.
• Backfill: one-shot script to clean EXISTING orphan CH rows from past deletes (pre-this-PR).
• Cloud-mode CH erasure: env-flag option for tenants who require strict GDPR erasure even in cloud.
• resetWebsite: also wipes PG analytics without touching CH today; same fix pattern would apply.

[vercel]

@anvme is attempting to deploy a commit to the Umami Software Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Summary

This PR adds ClickHouse analytics cleanup to every deletion path (website, link, pixel, user, team) and closes a long-standing gap where hard-deleting any entity left orphan rows in all 7 CH tables forever. It also fixes soft-delete filters on list and collect routes and wires Redis slug-cache invalidation for link/pixel updates and deletes.

• src/lib/clickhouse.ts: New deleteByWebsiteIds(ids) helper issues ALTER TABLE … DELETE WHERE website_id IN {websiteIds:Array(UUID)} against all 7 CH tables in parallel; parameterised with UUID typing, guarded by the enabled flag and empty-array check.
• user.ts / team.ts: Pre-transaction entity collection replaced by a parallel findMany batch; non-cloud paths now call invalidateRedis() then deleteByWebsiteIds(clickhouseIds) after commit; cloud paths gain soft-delete coverage for links/pixels/boards.
• link.ts / pixel.ts: Redis slug-cache invalidation added for updates (old + new slug) and deletes; CH cleanup added for non-cloud deletes; deletedAt: null filter added to list and collect routes.

Confidence Score: 4/5

Safe to merge with one targeted fix in deleteWebsite.

The deleteWebsite non-cloud else branch calls CH cleanup but omits redis.client.del for the website key. Because load.ts warms that Redis key with a 24h TTL on every analytics load regardless of cloud mode, a non-cloud + Redis deployment will serve the deleted website from cache for up to a day after deletion. The analogous handlers in user.ts and team.ts correctly call invalidateRedis() before the CH cleanup — this is an isolated gap in website.ts only.

src/queries/prisma/website.ts — the non-cloud .then() branch needs redis.client.del alongside the ClickHouse call.

Important Files Changed

Filename	Overview
src/lib/clickhouse.ts	Adds deleteByWebsiteIds helper issuing ALTER TABLE DELETE against all 7 CH tables in parallel; parameterised query is safe, table list is hardcoded, enabled/empty-array guard is correct.
src/queries/prisma/website.ts	Adds CH cleanup to the non-cloud deleteWebsite path, but the new else branch omits redis.client.del for the website key, leaving a stale Redis entry for up to 24h unlike the parallel user.ts/team.ts handlers.
src/queries/prisma/user.ts	Rewrites deleteUser to collect all owned entities before the transaction and correctly calls both invalidateRedis() and deleteByWebsiteIds() after commit in non-cloud mode.
src/queries/prisma/team.ts	Rewrites deleteTeam with the same entity-collection + invalidateRedis + CH-cleanup pattern; both cloud and non-cloud paths are correct.
src/queries/prisma/link.ts	Adds Redis slug-cache invalidation on update, CH cleanup on delete (non-cloud), and deletedAt: null filter to list queries.
src/queries/prisma/pixel.ts	Mirrors link.ts changes for pixel: Redis slug invalidation on update, CH cleanup on delete, deletedAt: null filter to list queries.
src/app/(collect)/p/[slug]/route.ts	Adds deletedAt: null guard to pixel-lookup paths so soft-deleted pixels no longer respond to tracking requests.
src/app/(collect)/q/[slug]/route.ts	Adds deletedAt: null guard to link-lookup paths so soft-deleted links no longer respond to tracking requests.

Sequence Diagram

sequenceDiagram
    participant API
    participant PG as Postgres (Prisma tx)
    participant Redis
    participant CH as ClickHouse

    API->>PG: collect entity IDs (links, pixels, websites)
    API->>PG: transaction — hard-delete / soft-delete rows
    PG-->>API: commit
    API->>Redis: del link/pixel/website cache keys
    alt cloudMode
        note over Redis: website key already cleared above
    else not cloudMode and CH enabled
        API->>CH: ALTER TABLE website_event DELETE WHERE website_id IN (...)
        API->>CH: ALTER TABLE event_data DELETE ...
        API->>CH: ALTER TABLE session_data DELETE ...
        API->>CH: ALTER TABLE session_replay DELETE ...
        API->>CH: ALTER TABLE website_revenue DELETE ...
        API->>CH: ALTER TABLE website_event_stats_hourly DELETE ...
        API->>CH: ALTER TABLE event_data_pivot DELETE ...
        CH-->>API: mutations queued (async in CH)
    end
    API-->>API: return result

Loading

_{Reviews (1): Last reviewed commit: "fix: clean up ClickHouse analytics rows ..." | Re-trigger Greptile}

[anvme]

Hey @mikecao I hope you merge this