Skip to content

Prevent late platform action writes from recreating deleted Actions databases - #19776

Open
jvcalderon wants to merge 10 commits into
feature/18586-actions-read-sdkfrom
feature/18586-race-condition
Open

jvcalderon wants to merge 10 commits into
feature/18586-actions-read-sdkfrom
feature/18586-race-condition

Conversation

@jvcalderon

@jvcalderon jvcalderon commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes the deletion race raised in #19724 review: an in-flight platform action event or a queued session-index job could recreate a workspace's dedicated Actions database after the workspace was deleted, because CouchDB creates missing databases on write.

The fix pairs a lock with an existence check. Either one alone still leaves a race.

  • Workspace-scoped lock: new LockName.PLATFORM_ACTIONS_WORKSPACE, keyed by the prod workspace ID as a system lock. Dev and prod writers, workspace deletion and tenant deletion all contend for the same key, whatever tenant is in context.
  • Writers: action-event persistence and session-index updates only write while holding the lock and only if the dev workspace database still exists. Prod may legitimately be absent for an unpublished workspace. Late events are discarded without enqueueing an index job. Late index jobs complete without writing, so Bull does not retry them. The existing per-session lock now runs inside the workspace lock.
  • Deletion: full workspace deletion destroys the dev and Actions databases while holding the lock. Tenant deletion groups databases per workspace and destroys that workspace's Actions and workspace databases under its lock. Unpublish is unchanged and still preserves Actions.

Scope and known limitations:

  • Actions writes are now serialized per workspace. Critical sections are limited to an existence check plus a single write.
  • Tenant deletion still ignores failures when destroying workspace databases, as it did before this PR. If a dev database destroy fails silently, a later writer can still recreate Actions. Propagating those failures changes general tenant deletion behaviour and is left out of this PR.
  • This coordinates lifecycle ordering. It is not a transaction: partial deletion failures and distributed-lock failure semantics are not solved.

Tests cover both orderings (writer first and deletion first), late events, late session jobs (action and lifecycle-only), dev/prod lock identity, per-workspace lock isolation, tenant deletion running only inside each workspace's lock, and Actions preservation on unpublish.

Addresses

Launchcontrol

Prevents activity logs from reappearing after a workspace or tenant is deleted.


Summary by cubic

Fixes the race where late platform action writes, queued session-index jobs, or view creation could recreate a deleted workspace's Actions database, because CouchDB creates missing databases on write.

Bug Fixes:

  • Writers, session-index jobs, and view creation now hold a workspace-scoped lock and only act while the dev workspace database still exists; late events are discarded, late jobs complete without writing, and view creation fails with a 404.
  • Locks auto-extend and block, so a slow write keeps deletion queued behind it instead of dropping the event when contention exceeds the old retry budget.
  • Workspace and tenant deletion destroy the dev and Actions databases while holding the same lock, keyed by the prod workspace ID so dev, prod, and deletion paths all contend.
  • Tenant deletion still ignores failures when destroying workspace databases; this coordinates lifecycle ordering but is not a transaction.

Tests cover both orderings (writer vs. deletion), late events and jobs, lock renewal past the original TTL, dev/prod lock identity, per-workspace isolation, view creation during deletion, and Actions preservation on unpublish.

Written for commit dc1e72e. Summary will update on new commits.

Review in cubic

@jvcalderon
jvcalderon marked this pull request as ready for review September 30, 2026 14:35
@jvcalderon jvcalderon self-assigned this Sep 30, 2026
@jvcalderon
jvcalderon added this pull request to stack #19741 September 30, 2026 14:39

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 14 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread packages/worker/src/sdk/tenants/tests/tenants.spec.ts Outdated
Comment thread packages/backend-core/src/events/processors/platformActions/tests/db.spec.ts Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant