feat: DynamoDB EventStore adapter

[PaulGrimshaw]

Summary

Implement a DynamoDB-backed EventStore adapter (@dcb-es/event-store-dynamodb) that is fully DCB-compliant.

Approach

Pessimistic fine-grained locks + batch commit record.

• Fine-grained locks on (eventType, tagValue) pairs — unrelated writes never conflict
• Atomic global sequence counter for ordering
• Batch commit record for crash-safe atomic visibility of large appends
• Small appends (≤~40 events) use a single TransactWriteItems — zero lock overhead
• Large appends use lock-based flow with BatchWriteItem — no size limit

Full technical design: packages/event-store-dynamodb/README.md

Key Design Decisions

• Why not optimistic watermarks? Type-level watermarks cause false conflicts between unrelated entities of the same type. Fine-grained watermarks blow the 100-item TransactWriteItems limit on large batches.
• Why pessimistic locks? Fine-grained locks match the natural conflict boundary (same entity, same event type). In practice, conflicts are rare — most lock acquisitions succeed immediately. No transaction size limits on event writes.
• Crash safety: Every crash scenario is safe. Partial writes are invisible (PENDING batch). Stale locks expire via TTL.

Scope

• Single-table DynamoDB schema + GSI design
• Lock manager (acquire/release with TTL, deadlock prevention via sorted acquisition)
• Append flow: small (transactional) and large (lock-based + batch commit)
• Read flow: type/tag queries via GSIs, batch status filtering with caching
• Sequence counter (atomic increment, range reservation)
• Stale batch / lock cleanup strategy
• Integration tests (localstack or DynamoDB local)
• Tag query GSI strategy (denormalized index items vs alternatives)

Open Questions

• Tag query GSI strategy — depends on expected query patterns and tag cardinality
• Lock TTL value (likely 30-60s)
• Query.all() optimisation to avoid global lock serialisation

[PaulGrimshaw]

Implementation Plan

Work is decomposed into 3 sequential chunks, each building on the previous:

Chunk	Issue	Scope
1	#42	Read path + append without condition (sequence counter, BatchWriteItem, query routing)
2	#43	Append with condition — transactional path (TransactWriteItems for small appends)
3	#44	Lock-based append for large batches + concurrency safety (batch-linked locks, batch commit lifecycle)

Prerequisite: #40 (opaque SequencePosition) — in progress on feature/opaque-seq-position worktree. Can rebase when merged.

Design doc: packages/event-store-dynamodb/README.md (committed on feature/dynamo)

Scaffold: Committed and green — package structure, test infrastructure (testcontainers + DynamoDB Local), stub implementation, 3 passing validation tests.

[PaulGrimshaw]

Test Strategy

TDD throughout — tests written before implementation, each chunk adds its test suite.

Chunk 1 (#42): Restore query + append-without-condition tests

• Empty store, sequence ordering, multi-event append, metadata round-trip
• All query shapes: type, tag, type+tag, Query.all()
• Read options: backwards, limit, fromSequencePosition
• Multi-QueryItem OR logic with dedup

Chunk 2 (#43): Restore conditional append tests

• Succeed with no prior matching events
• Reject when matching events exceed observed position
• Succeed with unrelated type/tag appended concurrently

Chunk 3 (#44): Restore concurrency + add crash safety tests

• 10 parallel same condition → 1 wins
• 100 parallel no condition → all succeed
• 50 parallel different entities → all succeed
• Batch commit visibility (PENDING invisible, COMMITTED visible)
• Lock steal after batch commit
• Crash recovery (stale PENDING → FAILED)

Infrastructure: testcontainers + amazon/dynamodb-local, isolated table per test suite. Already scaffolded and green.

[PaulGrimshaw]

Reset — fresh start from design spec

All previous implementation branches and sub-issues have been closed.

Current state:

• Single branch: feature/dynamodb-adapter — contains only the technical design README
• Sub-issues feat(dynamodb): read path + append without condition #42, feat(dynamodb): append with condition (transactional path) #43, feat(dynamodb): lock-based append for large batches + concurrency safety #44 closed as duplicates — all work tracked here
• No implementation code — clean slate

Design spec (the README) incorporates all learnings from the previous implementation rounds and architecture reviews:

• Standard database transaction terminology (txn records, fencing tokens, ACTIVE/COMMITTED/ABORTED)
• Lease on transaction record (not per-lock), relative leaseDuration (no clock skew)
• Streaming sub-batch transactions (steps 2–6 repeat per batch)
• Scoped safe checkpointing (no global watermark — projections only block on overlapping ACTIVE transactions)
• Single _TXN partition for efficient recovery queries