Skip to content

feat(notifications): dead-letter undeliverable emails with delivery counters - #1792

Open
bilhokista wants to merge 5 commits into
Arena1X:mainfrom
bilhokista:feat/1603-email-dead-letter
Open

feat(notifications): dead-letter undeliverable emails with delivery counters#1792
bilhokista wants to merge 5 commits into
Arena1X:mainfrom
bilhokista:feat/1603-email-dead-letter

Conversation

@bilhokista

Copy link
Copy Markdown

Closes #1603.

What was already there, and what was not

Retry with capped exponential backoff already exists: EmailService.deliverEmailWithRetry retries transient failures (network errors, 5xx, 429) with base * 4^attempt and ±20% jitter, and throws immediately on a 4xx. I did not touch that policy.

What was missing is everything after it gives up. processQueue shifts a message off the in-memory queue, and if delivery fails it catches the error, logs one line, and returns — the message is gone, with nothing left to inspect or replay. There were also no counters.

The change

Dead-letter store. New entity DeadLetteredEmail (dead_lettered_emails) plus migration 1788100000000-CreateDeadLetteredEmails.ts. It records the recipient, subject, both rendered bodies, the failure message, the attempt count, and when the message was first queued. The rendered body is stored so a message can be replayed exactly as composed, without re-running template rendering that may since have changed.

reason is an enum with two values, because the operational response differs:

  • permanent — the provider gave a verdict (bad address, revoked key). Resending unchanged will fail again.
  • retries_exhausted — the send never got a verdict. Worth replaying once the provider is healthy.

Indexed on message_id, on created_at, and on (reason, created_at) for the obvious triage query: everything worth replaying, newest first.

Failure type. deliverEmailWithRetry now throws EmailDeliveryFailure, carrying the reason and attempt count so the caller does not have to re-derive them. It preserves the underlying message, so the existing assertion rejects.toThrow(permanentError) still holds.

Counters. getDeliveryCounters() returns { sent, retried, deadLettered } by value, ready for a metrics endpoint. Every dead-letter is logged at error with the counters inline.

Failure of the failure path. deadLetter() never rethrows. If the database write fails, it logs that the message is lost and queue processing continues — a dead-letter write must not take down delivery for every message behind it. That case is tested.

Tests

Five new cases in email.service.spec.ts, driving the real queue timer with fake timers:

  • transient failure retries then succeeds — nothing dead-lettered, counters {sent: 1, retried: 1, deadLettered: 0}
  • 4xx is dead-lettered after exactly one attempt, with reason: permanent and the row contents asserted
  • exhausted budget is dead-lettered with reason: retries_exhausted, attempts: 3
  • a failing dead-letter write does not escape processQueue
  • getDeliveryCounters() hands out a copy, not the live object

Both existing TestingModule setups gained the new repository provider, so the current suite still compiles and runs unchanged.

Honest note on verification: I could not run the backend Jest suite locally (no full workspace install). Every changed file was parsed with the TypeScript compiler API, and email.service.ts was transpiled and executed standalone against a stubbed Nest/TypeORM surface — all 15 behavioural assertions passed, covering the same four scenarios above against the real compiled service rather than a re-implementation. CI is the real check on the Nest-level suite.

One thing worth a maintainer's eye: the dead-letter table has no retention policy. For a high-volume outage it will grow. I left that out rather than guessing at a TTL — happy to add a pruning job if you want one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CrfEY1tvXrbeMDAUzxfuk7

@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
insight-arena-4rll Ready Ready Preview Sep 10, 2026 1:45pm UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Backend] — Notification Delivery Retry with Dead-Letter Queue

1 participant