Snapshot enrichment store for post-alert rendering

[jfberry]

Summary

Add an on-disk store of fully-resolved enrichment views keyed by delivered message ID, with TTL matched to the alert's TTH and a long-tail age-based safety sweep. This is the substrate for any feature that needs to re-render an alert in a different shape, or take action against it, after the message has already been sent — ephemeral buttons (#109), action buttons (mute, unsubscribe), late-arriving edits, summary digests that reference past alerts, etc.

Design choices

Save the resolved view, not the raw layers, not pre-rendered output

We considered three approaches:

A. Save the resolved per-delivery view. One snapshot per delivered message (per user, per channel), containing the fully-merged LayeredView map plus alert metadata.
B. Save raw layers per event and reconstruct the view at click time, swapping per-user data dynamically.
C. Pre-compute every possible response at fire time and store the rendered bytes.

Recommendation: A.

1. The snapshot is exactly what the user saw — no view-reconstruction logic at click time. Re-rendering is just render(template, view).
2. DTS render latency is low single-digit ms; Discord's 3s interaction window has plenty of headroom.
3. Pre-computing (C) locks operators into the responses they configured when the alert fired. Adding a new button or changing a response template later would require re-firing alerts.
4. Per-event raw-layer storage (B) saves some duplicated bytes across users but costs significant complexity for the reconstruction path; click-rate workloads don't justify the optimisation.
5. Per-delivery snapshots scale storage with audience size, but each record is small and self-contained. The simpler model wins.

Per-delivery, not per-event

Each delivered message gets its own snapshot. A raid that fans out to 100 DMs + 5 channels = 105 snapshots. Channels-as-humans (the existing Poracle model where channels are humans rows with type=discord:channel) means "per delivered destination" and "per snapshot" are the same statement.

Trade-off: more snapshots than a per-event store, but each is self-contained and trivial to read. The simplicity is worth the storage.

Snapshot record structure

type Snapshot struct {
    // Identity
    MessageID  string  // pogreb key
    Target     string  // human_id (DM user or channel)
    TargetType string  // "dm" / "channel" / "webhook" — affects which buttons are valid
    CreatedAt  int64
    ExpiresAt  int64   // TTH-derived; background sweep handles overruns

    // Alert metadata — the "out-of-band" fields actions need
    AlertType    string  // "monster" / "raid" / "incident" — source webhook type
    TemplateType string  // resolved template type ("raid" / "rsvpChanges" / "incident")

    // Template identity — both forms, so consumers can either re-render the
    // exact entry the user saw OR re-resolve against the original intent.
    // When these differ, the selection chain fell back from the requested id.
    TemplateRequested string  // the literal value from the tracking rule (may be empty = "use default")
    TemplateSelected  string  // the DTS entry id actually picked by the 6-priority selection chain

    Language string
    Platform string  // "discord" / "telegram"

    // Every tracking rule that fired for this delivery
    TrackingUIDs []int64

    // Geographic context (for area-scoped actions)
    MatchedAreas []string

    // The fully-resolved LayeredView this user actually saw
    View map[string]any
}

Reasoning on each non-obvious piece:

• View is the resolved merged map, not raw layers. Re-rendering on click is just render(template, view). If a click handler needs clicker-specific data (channel-message "directions from me" pressed by someone other than the original target), compute it fresh from the clicker's stored location and merge — no need to retain the raw layer split in the snapshot.
• TrackingUIDs []int64 is what "mute this rule" and "unsubscribe from this rule" buttons need. A pokemon delivery can match multiple rules in one go (basic IV + PVP great + PVP ultra). The full list lets a button offer "mute the IV rule" vs "unsub from all PVP rules" vs "unsub from this species entirely."
• AlertType vs TemplateType split — same distinction we landed in raid-rsvp for tracker semantics. For action buttons it matters: a mute action reads AlertType="raid" to know which tracking table to touch, not the resolved template name "rsvpChanges".
• TemplateRequested and TemplateSelected together. The pair captures both intent and outcome. TemplateRequested is the literal value from the tracking rule (often empty, meaning "use the configured default"); TemplateSelected is the entry id the selection chain ultimately picked. If they differ, fallback happened — useful for diagnostics ("why didn't this user see template X?") and for "re-render with original intent" buttons that might prefer to redo the selection chain rather than pin to the historic outcome. The pair (TemplateType, TemplateSelected, Platform, Language) locates an entry unambiguously in currently-loaded DTS for exact re-render; if the entry is gone, the selection chain runs as a graceful fallback.
• TargetType is needed by Interactive buttons on alert messages with ephemeral responses #109's button visibility rules — some buttons (unsubscribe, user-scoped mute) only make sense on DM deliveries. Snapshot carries this so click-time visibility checks don't need a humans table lookup.
• No raw webhook fields stored separately. The view already has them via alias fallback. Revisit if a concrete need surfaces.
• MatchedAreas earns its place because area-scoped actions ("mute this area for an hour") are plausible and recovering area names from the view via alias is awkward.

Implementation note: threading TemplateRequested

The render pool currently resolves the template via the selection chain and forwards the resolved entry downstream — the requested template id (from the tracking rule) is consumed and not propagated. Capturing it for the snapshot is a small refactor in processor/cmd/processor/render.go: pass the requested id through alongside the resolved entry so the sender can include it in the snapshot write.

Storage stack

• pogreb for on-disk persistence — separate instance from geocache, same dependency.
• No in-memory hot tier in v1. pogreb reads are fast and button click rates are human-paced. Add ttlcache layer later if metrics ever show contention.
• Key: target:messageID (mirrors MessageTracker's convention).
• Value: JSON-serialised Snapshot struct.

We already use pogreb in internal/geocoding/cache.go. No new dependency.

The 6-priority template selection chain and the rest of the render pipeline are unchanged; this is a new sidecar store.

Lifecycle

• Write: at delivery success, in the sender after the message goes on the wire. Atomic point — if the bot dies before save, that one message's buttons won't work. Soft failure; ephemeral "this alert has expired" response covers it. Write errors (disk full, pogreb corruption) are non-fatal: log a warn, continue. Snapshot writes never block delivery.
• Read: on button interaction (or whatever consumer arrives next).
• Delete (normal): at ExpiresAt. Piggyback on MessageTracker's clean-deletion callback when applicable so a deleted message also drops its snapshot.
• Delete (safety sweep): background sweep on a long interval deletes any snapshot older than configured max_age (default 7 days, configurable). This catches orphaned entries from crash/restart timing where the TTL expiry path didn't run. Expected steady state never hits this — it's only ever cleaning up missed expiries.

Edits and re-snapshotting

Message edits write a new snapshot that overwrites the previous one for the same MessageID. The render pipeline for an edit resolves whatever template the edit calls for (e.g. an RSVP update transitions from the raid template to the rsvpChanges template), and that template's buttons + view become the new snapshot.

Implications:

• Snapshots are always "what the user currently sees" — never stale due to edits.
• The button set attached to a message can change between edits (the compact rsvpChanges template can declare a different set than the full raid template).
• The render path must emit the components block on every edit call, because Discord's edit semantics replace components by default. The existing edit code in delivery/discord.go needs to thread components through edits, not just initial sends.
• Reply chains (raid → rsvpChanges) work the same way: each delivery (original send and each subsequent edit) writes its own snapshot.

This collapses the "snapshot staleness on edited messages" problem entirely — the design says edits are full re-renders, so the snapshot stays accurate by construction.

Storage cost — back-of-envelope

• Per snapshot ≈ 10 KB (resolved view + metadata).
• 100k events/day × avg 3 destinations = 300k snapshots/day.
• Average normal TTL ~30 min → peak in-flight ~6k entries × 10 KB ≈ 60 MB working set.
• 7-day safety-net retention only applies to orphaned entries that escaped normal expiry — not the steady-state churn. Expected disk footprint is bounded by the working set plus a small orphan tail.

pogreb's compaction handles the churn shape comfortably (similar to MessageTracker's clean-deletion entries).

Format and versioning

• Format: JSON via encoding/json. Simple, debuggable, plays nicely with pogreb's []byte values. Revisit msgpack/gob if size becomes a problem.
• Schema version: include a version int at the top of the serialised record. On read, if the version is newer than the binary supports → drop the snapshot and respond ephemerally "alert has expired" (graceful no-op). On older version → either upgrade in-place or drop, depending on the change.

Configuration

[snapshots]
enabled = false        # default OFF; operators opt in
# path           = "config/.cache/snapshots/"
# max_age        = "7d"
# sweep_interval = "1h"

The store is opt-in, defaulting to off. Two reasons:

1. Disk usage isn't free — even a ~60 MB working set is something operators should agree to before we start writing it. Sneaky disk usage is a bad default.
2. Buttons require snapshots. With enabled = false, the render path skips the components block entirely — no buttons attached to any message, regardless of what the DTS says. Operators who don't want buttons don't pay the snapshot cost.

The path, retention, and sweep cadence have sensible defaults; operators only touch them for tuning.

Interaction with buttons

The button-rendering code (in #109) checks the snapshot config at startup: if enabled = false, buttons declared in DTS are silently skipped during alert render. The DTS still loads without error — operators can keep button definitions in their templates and turn them on later by flipping the snapshot knob. This avoids load-time errors that would force operators to strip button declarations to disable the feature.

Metrics

For operator visibility:

• poracle_snapshot_writes_total{result="ok|fail"} — counter of write attempts.
• poracle_snapshot_reads_total{result="hit|miss|stale"} — counter of read attempts (miss = no entry; stale = version too new).
• poracle_snapshot_store_bytes — gauge of approximate disk usage.
• poracle_snapshot_sweep_deletions_total — counter of safety-sweep deletions (should be near-zero in healthy operation).
• poracle_snapshot_age_seconds — histogram of snapshot age at delete time.

Snapshot inspection API

For debugging — an admin-protected endpoint exposes the stored snapshot for a given message:

GET /api/snapshots/<messageID>

• Requires x-poracle-secret (existing API auth).
• Returns the Snapshot struct as JSON.
• 404 if no snapshot exists for that messageID.

No list endpoint in v1 — operators with debugging needs know the messageID from logs or Discord.

Open questions

• Pogreb instance separation. Lean toward a dedicated instance — keeps geocache's working set independent and lets the safety sweep run without affecting geocode operations.
• max_age default. 7 days as a starting point. Long enough that operators investigating "why didn't this button work yesterday" can find the snapshot; short enough that orphans don't accumulate. Configurable for operators with different audit needs.
• Sweep cadence. Hourly is probably fine — the sweep just walks pogreb's key space looking at ExpiresAt. Not on the hot path.
• Channel vs DM View content. Channels have no per-user enrichment (no distance/bearing). DMs do. The View field shape is identical either way; channel snapshots just have those fields empty. Document, don't special-case.

Out of scope

• Specific consumers (buttons, edits) — those are Interactive buttons on alert messages with ephemeral responses #109 and follow-ups.
• Cross-process replication / clustering — single-process semantics only.
• In-memory hot tier — defer until metrics justify.

Dependencies

None — foundation issue. Consumers: #109 (buttons), #110 (TOML format for the buttons that consume this).