Skip to content

Drive user index engine from dedicated management stream with transactional writes  #5439

@qodo-code-review

Description

@qodo-code-review

Description

  • Avoid expensive log scan during startup to build user index management event index, which blocks custom indexes from working on large databases
  • Eliminate extra $all subscription cost when custom indexes are not in use
  • Enable atomic writes of index management events to both the individual index stream and a dedicated management stream
  • Simplify event state management by extracting index identity from events instead of stream names

Deliverables

  • Multi-stream writes: Extend SystemClient to support transactional writes to multiple streams with per-stream expected versions
  • Management stream: Create $all-indexes stream to collect all user index management events, written transactionally with individual index streams
  • Event store decorator: Implement UserIndexEventStore to intercept and duplicate index management events to the management stream
  • Subscription refactor: Update UserIndexEngineSubscription to subscribe to management stream instead of building index from bootstrap
  • Optional services: Make secondary indexing services optional with graceful error handling when disabled
  • State simplification: Remove MultiEntityState abstraction and extract index names from events instead of stream names

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions