Skip to content

Rework session storage: overflow-driven idempotent segments - #27

Merged
lyndonbauto merged 2 commits into
mainfrom
refactor/session-segments-idempotent-append
Jun 22, 2026
Merged

Rework session storage: overflow-driven idempotent segments#27
lyndonbauto merged 2 commits into
mainfrom
refactor/session-segments-idempotent-append

Conversation

@lyndonbauto

Copy link
Copy Markdown
Collaborator

Summary

Replaces the hot-tail-on-session-record + sealed c: chunk model with append-only K_ORDERED segment records (app:user:session:g:NNNNNNNN). Each segment holds one events Map keyed "{ts_micros:020d}:{event_id}".

  • Idempotent append. The map key is a pure function of the event, so a retried/duplicated map_put overwrites the same slot — a timed-out-then-retried append can never duplicate an event.
  • Cheap reads. Key-ordered map → server-side last-N via map_get_by_index_range(-N, N) and after_timestamp via map_get_by_key_range.
  • React, don't predict. The only rollover trigger is a real RecordTooBig; no client-side byte estimation/threshold/flush. cur advances with a cur == N guarded increment so concurrent rollovers converge on the same next index.
  • 1-RTT hot path. No-state-delta append = one operate; an append carrying state = one batch_write coalescing the segment write with session/app/user state writes (multi-scope appends now match single-scope latency).
  • Back-pressure resilience. Because every hot-path write is idempotent, transient back-pressure is retried with bounded jittered backoff: DeviceOverload always (rejected, never applied) and ambiguous TimeoutError where a re-apply is a no-op, including per-record DeviceOverload in the coalesced batch_write. create_session retries DeviceOverload only (POLICY_EXISTS_CREATE put is not timeout-idempotent).

Fixes

  • Heavy-concurrency data loss / unrecoverable RecordTooBig on a single hot session (events were dropped when concurrent appends overran the 1 MiB write-block-size).
  • A lone event exceeding max-record-size now raises a clear RecordTooBig instead of looping.

Breaking

  • Session on-disk layout changed (clean break, no migrator). Sessions written by <= 0.0.2 are not read by this version. Version bumped to 0.1.0.
  • Removed flush_threshold_bytes / huge_event_bytes / max_tail_bytes ctor params, codec.estimate_event_size, keys.chunk_key, and the tbytes / chunks / seq / ts_lo / ts_hi / cidx bins.

Also includes the previously-staged Atomic Session Append tutorial (updated for the segment model).

Test plan

  • Unit suite (no Docker): pytest -m "not aerospike" → 53 passed
  • Session integration suite (testcontainers): pytest tests/test_sessions.py → 43 passed
  • New server-free retry unit tests: tests/test_session_retry.py → 5 passed (overload retried/exhausted, timeout retried/opted-out, other errors propagate)
  • Hot-path benchmark vs patched old model: no regression (faster under concurrency; multi-scope appends up to ~3x faster via batch_write coalescing)

Demonstrate scoped state deltas, bin inspection, and concurrent append_event load.
Replace the hot-tail-on-session-record + sealed `c:` chunk model with
append-only K_ORDERED segment records (`app:user:session:g:NNNNNNNN`), each a
single `events` Map keyed `"{ts_micros:020d}:{event_id}"`. The key is a pure
function of the event, making append idempotent (a retried map_put overwrites
the same slot) and reads cheap (server-side last-N / after_timestamp via map
index/key ranges).

Rollover reacts to a real RecordTooBig (no client-side byte estimation): the
writer bumps `cur` with a `cur == N` guarded increment so concurrent rollovers
converge. The append hot path is 1 RTT — a no-state-delta append is one
operate; an append carrying state is one batch_write coalescing the segment
write with session/app/user state writes.

Because every hot-path write is now idempotent, add bounded jittered retry on
transient back-pressure: DeviceOverload always, ambiguous TimeoutError where a
re-apply is a no-op (including per-record DeviceOverload in the batch).
create_session retries DeviceOverload only (POLICY_EXISTS_CREATE put is not
timeout-idempotent).

Fixes heavy-concurrency data loss / unrecoverable RecordTooBig on a single hot
session. BREAKING on-disk layout (clean break, no migrator); bump to 0.1.0.
@lyndonbauto lyndonbauto self-assigned this Jun 22, 2026
@lyndonbauto
lyndonbauto merged commit cae16a4 into main Jun 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants