Contact Center — Production Readiness (Consolidated & Revised)

Status: Retained record for this PR. This document supersedes and consolidates the earlier planning documents (PLAN.md, PLAN-2-SINGLE-NODE-COMPLETION.md, PLAN-3-PRODUCTION-SINGLE-NODE-DISTRIBUTED.md, R0-BASELINE.md), the pure release-tracking *.v1.json governance ledgers, and the .playwright-mcp/ capture directory — all of which were planning scaffolding removed in the final cleanup task (§7) so the PR is merge-ready. It is now the sole surviving plan document.

Revision intent. The prior plans were reviewed in depth from an Orchard Core / ASP.NET Core engineering standpoint, then cross-reviewed by two independent expert models (GPT-5.6 and Opus-5) who each verified the claims against the actual source. A large amount of genuinely valuable correctness, distribution, and data-lifecycle work was already completed and is retained (§3). A second, open set was found to be either (a) over-engineered relative to a first production release of an Orchard Core module, (b) reinventing capabilities Orchard Core already provides, or (c) exotic telephony features that belong to a later track — those are de-scoped with rationale (§5, §6). The remaining work (§4) is the minimum honest set that makes the module production-ready while extending Orchard Core rather than working around it. Where the reviewers found the code cannot honestly support a broad claim, we deliberately narrow scope and advertise capabilities honestly instead of building heavy machinery — that is the anti-over-engineering path and the production-honest one.

1. Mission & scope

Ship a production-supported Contact Center on a single active application process running the full distributed contract: PostgreSQL, Redis distributed locking, and the Redis SignalR backplane are mandatory, validated at startup, and probed by health checks. This is a voice ACD release (inbound voice, manual/preview dial, transfer, conference, supervision).

Single active process (deliberate scope decision). Exactly one application process owns the ARI/telephony application at a time. The provider-application ownership and channel-ready coordination are process-local today (they do not use distributed locking), so overlapping deployments (two processes subscribed to the same ARI app simultaneously) are not supported and must be prohibited operationally. Deployment uses a stop-old-then-start-new handover; this is a bounded maintenance interruption for the telephony control path, not zero-downtime — during the handover window new inbound call admission depends on the upstream PBX/carrier's own retry/queueing (documented in D3), and we make no zero-downtime claim absent a tested external buffering contract. Growing to 2–4 concurrently active nodes requires distributed ARI ownership + cross-process channel-ready coordination, which is Track B (§6). The database/Redis/backplane layers are already distributed-correct on one node; this decision is only about the telephony-application singleton.

Recording scope (deliberate decision). v1 supports mono (mixed) bridge recording, documented as mono with no per-leg diarization or talk-over analytics. Dual-channel/stereo recording is Track B (§6). Because recording is in scope, recording/media erasure (C1) is release-blocking. C1 delivers recording/media erasure, which is a component of GDPR Art. 17 / CCPA handling — it is not a general cross-entity subject-erasure feature (subject discovery/fan-out is Track B). Audio-dependent capabilities (recording, monitoring, whisper, barge) are advertised only per §4 item G1 (prove-or-disable), and the base voice path itself is release-gated by G1.

In scope: correctness of the provider-event authority, work-state and live-call topology, data lifecycle/retention + recording/media erasure, distribution correctness on one active process, observability with a minimum operational signal set (including functional Redis/lock/backplane probes), a credible minimum ACD metric set, an accessible agent desktop, security/compliance close-out, and honest operator documentation.

Out of scope (see §6, Track B): concurrently-active multi-node, multi-region, chat/email/SMS channels, predictive dialing, WFM/QM, skill-proficiency/bullseye routing, and advanced media (stereo recording, MOS telemetry, codec negotiation, PBX failover, STIR/SHAKEN, DTMF/IVR, music-on-hold, call park).

2. Guiding principles (extend Orchard Core; do not reinvent)

These principles govern every remaining item and every code review in this PR:

Extend the framework, don't rebuild it. Prefer Orchard Core and ASP.NET Core primitives over bespoke equivalents:
- Config export → Deployment steps; config import → Recipes. Never hand-roll an import/export format. (Done in W11.5 via the generic configuration-catalog kit; all future config must follow it. This also means the bespoke preview export/reset service is retired — §5.)
- Health checks → ASP.NET Core IHealthCheck + AddHealthChecks() (already used idiomatically). We do not add a bespoke aggregate endpoint and do not take a base dependency on OrchardCore.HealthChecks — see A2 for why a hard dependency would be a security/availability regression and why the aggregate route is documentation-only.
- Background work → IBackgroundTask; distributed coordination → IDistributedLock / ILock; caching → ISignal / Orchard cache; settings → site settings drivers; navigation → INavigationProvider; permissions → IPermissionProvider; tenant teardown → IModularTenantEvents.RemovingAsync(ShellRemovingContext).
- Data → YesSql sessions, indexes, and IDataMigration. No raw ADO, no second ORM. Never parallelize operations over a single YesSql ISession; external/media I/O runs outside the YesSql transaction in fresh tenant scopes.
Reuse in-repo patterns. Catalog CRUD → CatalogManager<T> / ICatalogEntryHandler<T>; deferred external work → ShellScope.AddDeferredTask; durable side effects → the existing Contact Center outbox; scoped units of work → IContactCenterScopeExecutor; tenant-scoped blob cleanup → the *BlobContainerTenantEvents pattern (e.g. AIDocumentBlobContainerTenantEvents, DncRegistryBlobContainerTenantEvents).
Right-sized rigor. Every change ships with a focused regression test that fails if the behavior regresses. We do not require per-gate versioned "test specifications", ledger-evidence meta-tests, or governance JSON — that process overhead is removed (§5).
Fail closed, degrade predictably. Production topology is validated at startup; missing Redis/PostgreSQL is Unhealthy with an actionable message, not a silent semantic change. Checked-in development credentials are rejected in Production.
Honest capability advertising. A capability is enabled only when a provider implements it and it is verified (test or documented proof). Unimplemented surface is deleted, not advertised. Implemented capabilities whose only unproven dependency is the deployment's media path (Monitor/Whisper/Barge) stay advertised, but the deployment audio proof is a release gate (G1): in Production, unacknowledged audio verification fails closed (readiness withheld), and Recording — where the unproven-audio risk compounds with the C1 media-lifecycle risk — is off by default until proven.

3. Retained foundation (already completed — do not redo)

The following completed work is sound and is kept. It represents the bulk of the production-readiness effort and is only summarized here (full history lived in the superseded ledgers, which §7 removes):

Truth & containment: single-node-distributed topology profile + startup validation (fail-closed), CI gate repair, supply-chain (NuGet audit, SBOM, secret scan, license inventory, hermetic offline build), replacement of non-executing tests, and W0.7 containment fixes.
One provider-event authority: CrestApps.OrchardCore.Telephony.Core provider-neutral ingress (lease, dedupe, ordering, retry); single ingest with two projections (Telephony history + Contact Center); one call-state vocabulary with real hangup causes; capability-split ITelephonyProvider; hardened Telephony store; provider-leakage removed from neutral contracts.
State authority & data lifecycle: ContactCenterWorkState extracted off OmnichannelActivity; live call topology (legs/bridge/participants/consult); retention across all high-volume tables; predicate-led hot-path indexes with PostgreSQL EXPLAIN budgets; migration-safety (batched backfills, additive-only enforcement, N-1 rolling-upgrade harness, event/document upcasters).
Feature graph & Orchard idiom: headless/admin feature split with a runtime closure proof; real ContactCenter.Admin; analytics capability degradation; feature-dependency audit; recipes + deployment steps for all Contact Center/CRM configuration (generic catalog kit); bounded voice-webhook body; write-path validation moved out of display drivers into handlers; public-API baseline approval; agent-workspace N+1 removed.
Residual hygiene: terminal-event teardown gating; hub cancellation convention; metric hot-row → append-and-rollup; agent-session heartbeat isolation; reservation critical-section reduction; no Elasticsearch in any correctness path (enforced).
Config validation (W18): IValidateOptions<T> + ValidateOnStart for Contact Center/Telephony/Asterisk options; production refuses invalid retention/health/topology configuration. (The KnownDevelopmentValues digest register that shipped with W18 is deleted — see A1; the mandatory Production rejection of checked-in dev credentials is retained, inlined at both call sites.)
Agent desktop accessibility (W6): ARIA/live-region/keyboard/degraded-state work on the agent workspace.
Minimum ACD metrics (W9): report timezone, interval buckets, service level %, hold time, occupancy, RONA auto-not-ready, supervisor force actions, ASA-from-queue-entry.

4. Remaining work — checklist tracker

Each item lists why it adds real value, the Orchard Core-aligned approach, and a testable exit criterion. Items are grouped, not strictly ordered; within a group they are independent unless noted. Every removal that touches a public type must regenerate the W11.8 PublicApiApprovalTests baseline as part of its exit.

Group A — De-overengineering & Orchard Core alignment (do first)

A1. Remove KnownDevelopmentValues entirely; inline mandatory Production credential rejection at both call sites. Value: the class (SHA-256 digest register, placeholder tables, asset-rescanning test) is over-engineered machinery for a v1 module and the user mandated its removal — but the direct rejection of the repository-published ARI/SIP/TURN defaults in Production is a real runtime security control that must survive. Approach: delete KnownDevelopmentValues.cs and KnownDevelopmentValuesTests.cs. Inline a small, explicit rejection of the known checked-in dev secrets/placeholders directly at both consumers: DefaultAsteriskOptionsValidator (startup/options validation) and AsteriskSoftPhoneRegistrationConfigContributor (the runtime TURN-secret guard that degrades to STUN-only ICE — this covers tenant-configured settings that IValidateOptions/ValidateOnStart never sees). No shared static register, no digests, no rescan test. Exit: the class and its test are gone; both call sites still reject the published defaults in Production (covered by a focused test at each site); strict build clean; config-validation suite green.
A2. Keep the idiomatic health-check registration; add functional dependency probes; do not add bespoke aggregate plumbing. Value: the module already registers all checks via AddHealthChecks().AddCheck<T>(name, tags) — the idiomatic mechanism; there is no parallel subsystem to remove. One real gap: §1 promises live Redis/lock/backplane probing but the current registrations contain no functional Redis connectivity or distributed-lock-acquisition probe. A hard base dependency on OrchardCore.HealthChecks is also wrong: that module unconditionally maps its single, predicate-less aggregate route when enabled, so it cannot express the readiness-vs-dependency split and, with details on, would leak DependencyTag info anonymously; adding our own second /health/aggregate would just be the bespoke plumbing this principle forbids. Approach:
- Add functional dependency checks: a short-timeout Redis connectivity probe, a distributed-lock acquire/release probe, and a backplane pub/sub round-trip probe (publish→receive on a dedicated channel within a bounded timeout — Redis/lock connectivity alone does not prove SignalR pub/sub works), all registered with DependencyTag (alerting), not ReadyTag (readiness stays node-local) — this makes §1's live-probe claim true.
- Keep the tag-filtered node-readiness route, the authorized dependency-diagnostics route, host-level UseContactCenterProcessLiveness, and SharedHealthCheckEndpointGuard.
- Do not add a companion aggregate feature. If an operator separately enables OrchardCore.HealthChecks, document that its route is an aggregate diagnostic (guarded by SharedHealthCheckEndpointGuard), not a load-balancer probe. Exit: base feature does not hard-depend on OrchardCore.HealthChecks; functional Redis connectivity, distributed-lock, and backplane pub/sub round-trip probes exist and are all DependencyTag; a test proves no dependency check participates in readiness; the "aggregate ≠ probe" note is in the docs; strict build clean.
A3. Remove the dead generic provider-webhook ingress (was W16.9). Value: IProviderVoiceWebhookAdapter / HmacProviderVoiceWebhookAdapterBase / ProviderVoiceWebhookProcessor have no concrete implementation; both shipping PBXs ingest through the normalized IProviderVoiceEventService seam, so ProcessAsync returns UnknownProvider for every request — an authenticated public endpoint nothing can satisfy is a liability. Approach: delete the unused contract, base class, processor, endpoint mapping, and their test fakes. Exit: the endpoint and types are gone; no shipping ingress path is affected; activation/feature suites pass; PublicApiApprovalTests baseline regenerated.
A4. Delete the advertised-but-unimplemented supervisor take-over surface, and close the class with a test (was W12.2). Value: MonitorMode.TakeOver, the TakeOver capability bit (1 << 12), and its ContactCenterMonitoringService entry advertise a feature no provider implements. TakeOver is the last MonitorMode member and the capability flag is an explicit value, so removal is ordinal-safe and does not disturb persisted bitmasks. Approach: remove the enum value, capability bit, and mode entry; add a ~30-line unit test asserting every bit in ContactCenterVoiceProviderCapabilities is either declared by an in-tree provider or explicitly listed as reserved (this enforces the §2.5 honesty principle generically, replacing the removed governance gate). Exit: the symbol is gone; the capability-coverage test passes; monitoring tests updated; PublicApiApprovalTests baseline regenerated.
A5. Retire the bespoke preview export/reset service in favour of Deployment/Recipes (promoted from §5). Value: ContactCenterPreviewMaintenanceService is not a reliable backup — it exports before quiescing, its "receipt" hashes row counts not content, it has no import/restore, and its config export duplicates Orchard Deployment/Recipes. Retiring it removes a false recoverability guarantee and a public API surface. This is real deletion + removal of a destructive admin action, so it is tracked here, not buried in §5. Approach: delete the entire Maintenance feature — the bespoke config-export path (Deployment steps + Recipes are the sanctioned mechanism) and the destructive preview reset (operational-data recovery is a database backup/restore, per E2); delete the public maintenance abstractions (IContactCenterPreviewMaintenanceService, ContactCenterPreviewMaintenanceOptions, ContactCenterPreviewMaintenanceStatus, and the whole Abstractions/Maintenance + Core/Maintenance surface), the controller/views/view-models, admin menu, permission, the service-collection extension, the Feature manifest block, and the dedicated tests. Exit: config export/import goes through Deployment/Recipes only; the destructive preview reset is removed entirely (no bespoke reset ships); public maintenance abstractions removed and PublicApiApprovalTests baseline regenerated; docs updated; strict build clean.

Group B — Correctness & distribution close-out

B1. Start the ARI listener after provider-local validation + ownership; gate Contact Center admission on the topology verdict (was W16.3). Value: the base Asterisk realtime listener also serves standalone Telephony, so it must not depend on Contact Center readiness or an HTTP probe. Today it can start before the node is ready. Approach: start the listener after provider-local validation and ARI-application ownership acquisition (in AsteriskRealtimeVoiceTenantEvents); gate the Contact Center admission / event bridges on the topology verdict recorded by ContactCenterTopologyValidator. Readiness observes listener state; it does not control startup. Exit: a test proves the listener opens its WebSocket only after provider-local validation + ownership, and that Contact Center admission is withheld until the topology verdict is healthy.
B2. Telephony command settlement on shutdown, with correct per-path semantics (was W15.5). Value: DefaultTelephonyCommandExecutor links ApplicationStopping into the operation token, which cancels the in-flight provider call — there is no command buffer/queue to "drain". Two distinct command paths exist and must be handled differently: the durable path via ProviderCommandProcessor (recoverable), and synchronous release-critical mutations that call ITelephonyCommandExecutor directly — recording (ContactCenterRecordingService), monitoring (ContactCenterMonitoringService), and transfer (ContactCenterTransferService) — which ProviderCommandProcessor can not durably mark OutcomeUnknown. Approach: on ApplicationStopping, stop admitting new commands. For the durable path, allow bounded settlement and let unresolved commands be marked OutcomeUnknown for ProviderCommandRecoveryBackgroundTask. For the synchronous path, either (preferred) route those release-critical mutations through the durable command state machine, or give them explicit synchronous-command semantics: bounded settlement, an explicit unknown-result surfaced to the caller (never a silent success), and provider-state reconciliation (D2) as the backstop. Do not claim exactly-once application for either path. Exit: a restart with in-flight commands leaves each command in a definite state (durable→OutcomeUnknown; synchronous→explicit unknown to caller) with no silent success; tests cover both paths and assert new commands are rejected during shutdown.
B3. Report-factory mutable-state — prove before refactoring (was W10.8). Value: some scoped report providers hold mutable per-execution fields; a real cross-request/singleton leak would be a data-leak shape, but none was demonstrated. Approach: first write a concurrency test that drives two simultaneous report requests through the suspect provider(s). Only if it reproduces a leak, make the captured state per-invocation. If it does not reproduce, close the item with the passing test as evidence (no broad refactor). Exit: a two-request concurrency test exists and passes; any real leak it exposed is fixed.
B4. Provider-specific idempotency key (was W12.5). Value: Asterisk AsteriskRealtimeVoiceEventMapper.BuildIdempotencyKey hashes the raw payload, so an upstream serialization change silently turns dedupe into double-processing. But ARI events carry no eventId/sequence — a naive (provider, callId, eventType) fallback would suppress legitimately repeated same-type events (hold/unhold, ChannelVarset, DTMF), converting dedupe into event loss (strictly worse). Approach: the key must be per-provider and per-event-type, enumerating discriminators that make two distinct same-type events on one call distinguishable (e.g. include the varset name/value, the target sub-state, or a monotonic field where ARI provides one); where ARI genuinely offers nothing better, pin a stable serialization for the payload hash rather than the raw wire bytes and document the assumption. Exit: dedupe survives a payload re-serialization in a test, and a test proves two legitimately distinct same-type events on one call are both processed.
B5. Historical transfer-string cleanup only — MediaTopologyId is already gone (narrowed from W10.9). Value: verification showed the live CallSession already uses a typed active topology and the MediaTopologyId placeholder no longer exists; only historical transfer strings remain on InteractionTransferHistoryEntry. So this is not a release blocker. Approach: narrow to a compatibility-safe cleanup — ensure neutral consumers read typed transfer targets/outcomes and that historical string fields are clearly typed/labelled; no live-topology remodel. Exit: neutral consumers reference typed topology; historical strings are isolated to the history entry; architecture test passes. (If verification finds nothing to change, close as already-correct with evidence.)
B6. Confirm and harden the single-active-process telephony ownership (new — from the §1 scope decision). Value: AsteriskAriApplicationOwnershipRegistry uses a process-local static dictionary (no distributed lock) and returns success for blank claim inputs; AsteriskAgentChannelReadySignal is tenant-local in-memory. These are correct only under the single-active-process scope. Approach: keep the process-local design (distributed ownership is Track B), but deny + log blank/invalid claim inputs at the claim path (TryClaim returns false so the listener does not start on an unconfigured provider — a denial, not a thrown exception during shell activation), and add a startup log/health note stating the single-active-process constraint. Document the non-overlapping deployment requirement in D3. Exit: blank claims are rejected by a test; the constraint is surfaced at startup and in the runbook.
B7. Background-task drain & lock-margin safety (new — reviewer finding). Value: ReservationExpiryBackgroundTask holds a 60-second lock expiration on a one-minute schedule and does not participate in the feature work-manager drain, so a slow run can overlap itself or a rolling handover. Approach: widen the lock margin relative to the schedule, and make the task participate in work admission/drain (W3.5-style) so it stops cleanly on shutdown. Exit: a test proves no self-overlap at the margin and that the task drains on ApplicationStopping.
B8. DialPad UTC parsing fix (new — reviewer finding). Value: DialPadTelephonyProvider uses culture/time-zone-sensitive DateTimeOffset.TryParse, so provider timestamps can shift by the server's locale. Approach: parse as invariant/UTC (DateTimeOffset.TryParse(..., CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, ...) or the ISO-8601 exact form the provider emits). Exit: a test parses a provider timestamp identically regardless of ambient culture/time zone.

Group C — Data lifecycle & compliance

C1. Recording/media erasure & lifecycle (was W17) — release-blocking because recording is in scope. Scope note: this delivers recording/media erasure, a component of GDPR Art. 17 / CCPA handling — not a general cross-entity subject-erasure feature (subject discovery/fan-out across CRM activities is Track B). All claims are worded as recording/media erasure. Value: recorded audio is currently never deleted by any path: EraseAsync/RecordingErased have no callers, IRecordingMediaStore.DeleteAsync is never invoked, a second RecordingReference lives on CallSession (not just Interaction), the interaction retention path deletes the row that carries RecordingReference on age alone without deleting the media, and the durable AsteriskRecordingIngestService can write media after an erasure request. So deletion is unimplementable and racy today. Approach (extend Orchard Core):
- Add an authorized erasure command/endpoint that calls EraseAsync. In one YesSql unit of work, atomically: clear all references (Interaction and CallSession), write the durable erasure tombstone, and enqueue durable media deletion on the existing Contact Center outbox. Pointer clears + tombstone + outbox message all commit together or not at all.
- AsteriskRecordingIngestService must consult the tombstone and refuse/clean up a late ingest so an event cannot resurrect the recording.
- Perform the actual provider/local/pluggable-store deletion idempotently in a fresh scope outside the YesSql transaction (external I/O never inside the tx).
- Extend the existing retention background task: the interaction retention path must enqueue media deletion before deleting the row and must exclude records whose per-record RecordingLegalHold is set (today IsSubjectToLegalHold only widens the time floor and ignores the per-record flag — a real bug that both orphans media and deletes held records).
- Handle tenant decommission via IModularTenantEvents.RemovingAsync(ShellRemovingContext) following the *BlobContainerTenantEvents pattern. Because a removed tenant's outbox/background task will not keep running, tenant removal must fail closed on cleanup failure (block/raise) or persist the cleanup in a host-level mechanism that survives shell deletion — never silently orphan media on tenant removal.
- Record a deletion receipt in the Orchard audit trail for human-visible confirmation — but audit is not the sole durable receipt (its category/retention can be disabled); the durable proof is the outbox completion + tombstone. Exit: integration tests prove (1) after erasure the encrypted media bytes, transcript, both pointers, and any pending ingest job are gone, and the pointer clears + tombstone + outbox enqueue commit in a single atomic unit (crash-between test); (2) the retention path deletes expired media and refuses to delete records under RecordingLegalHold; (3) a late ingest after erasure is rejected by the tombstone; (4) tenant removal either completes media cleanup or fails closed (test both); (5) confirmed deletion (not merely request acceptance) is audited. No general "subject erasure / GDPR-complete" claim appears in code or docs.
C2. Column-size correction — treat as a real widening migration (from W16.7). Value: a couple of index columns are mis-sized (nvarchar(261), nvarchar(128)), risking truncation. Correction: widening an existing column is not an "additive-only" no-op — it alters an existing column and must be handled as a schema change with the N-1 rolling-upgrade harness. Approach: a migration that widens the columns, verified by the existing N-1 harness so old and new nodes agree during rollout. Exit: fresh and upgraded tenants produce identical, correctly-sized schemas; N-1 harness green. Note: moving the heartbeat timestamp out of the session document (the other half of W16.7) stays deferred (§6) — schema churn without a proven symptom now that the heartbeat write is isolated (W16.5).

Group D — Operability

D1. Bounded event channel with real backpressure + saturation metric (was W12.11; W12.13 folded in and de-scoped). Value: the realtime channel uses a magic 1000; AsteriskRealtimeVoiceListener declares FullMode.Wait but writes with TryWrite, so on saturation it drops the connection and reconnects. Approach: move the bound to a validated option; switch the write to await Writer.WriteAsync(...) with a bounded wait (real backpressure absorbed by the OS socket buffer); emit one saturation metric/alert on the first wait (not on drop); only if the wait times out, reconnect and run coalesced reconciliation after the old backlog drains. We do not build a separate durable pre-buffer event journal (W12.13). Honest limitations to document: reconciliation is pointer-driven (it reconciles calls we already know about, capped at 200/invocation, and no-ops when the reconciliation distributed lock is already held), so it restores current state but cannot reconstruct every intermediate hold/resume transition, and a dropped StasisStart (a call never learned about) is not recovered. (Optional, cheap: adopt ARI channel-list on reconnect to recover unknown channels — this is W12.6's core idea; pull forward only if small.) Exit: the bound is configuration-driven and validated; a saturation test asserts WriteAsync backpressure is applied and the metric increments on first wait; the documented limitation is written in §6/operator docs; no "zero loss" language remains.
D2. Reconciliation proves it reads live ARI (from W12.4). Value: "reconciliation" that reads local state is not reconciliation. Approach: add a contract test proving the Asterisk ITelephonyCallStateProvider queries live ARI (using the recorded-cassette harness from W5.4), and that the explicit call-state transition table (from W1.3) rejects illegal edges. Exit: a cassette test fails if the provider stops querying ARI. (The state-machine table itself is done; this is the reconciliation-liveness proof only.)
D3. Minimum operational signals + handover/rollback runbook (was W15.4 + reviewer finding). Value: ContactCenterDiagnostics currently exposes only two outbox counters; operators cannot see the health of the live system, and the deployment claim needs a written, rehearsed procedure. Approach: (a) add a minimum operational signal set to the existing telemetry: ARI connectivity/reconnect count, event-channel saturation (from D1), outbox age & dead-letter count, active/queued call counts, and the Redis/lock probe results (from A2) — reusing the observability seam already in place, no new framework. (b) Document the stop-old-then-start-new handover honestly as a bounded maintenance interruption of the telephony control path (not zero-downtime): state the single-active-process constraint (B6), that overlapping deployments are prohibited, that inbound admission during the window relies on the upstream PBX/carrier retry/queue behaviour, and the canary + rollback procedure for the single-node-distributed profile, referencing the drain and N-1 guarantees. Exit: the new metrics are emitted and covered by a test; the runbook exists, is linked from the deployment docs, states the non-overlapping-deployment requirement and the bounded-interruption reality (no zero-downtime claim), explicitly states what happens to live calls during the handover gap (channels sitting in ARI Stasis when the last subscriber disconnects — held, dropped, or dialplan fall-through), and the docs site builds.

Group E — Review & release

E1. Omnichannel/CRM layer review (was W13). Value: Contact Center depends on the CRM layer for the universal work item, but that layer was reviewed less deeply. Approach: run a focused review (authorization boundaries, migration safety, concurrency/CAS, retention, PII, background schedulers, tenant isolation); also evaluate signed origination markers (W12.12) here — if the current channel-variable ownership inference is cheaply forgeable and the HMAC fix is small, pull it forward, else record Asterisk/dialplan as a documented trusted boundary. Fix or explicitly accept each finding with a dated rationale. Exit: review findings closed or accepted in writing; no open high-severity CRM finding remains.
E2. Operator documentation truthful & complete. Value: README/architecture/deployment docs must match the shipped code. Approach: module READMEs, the single-node-distributed reference deployment (incl. the single-active-process constraint), the recording-is-mono statement, the emergency-calling scope statement + operator warning, and the data-residency statement — all in src/CrestApps.Docs, verified against code. Exit: docs build; internal links resolve; claims match code (covered by the §7 docs-review task).

Group F — Optional refactors (recommended, not release-blocking)

These improve maintainability but are not required for a correct release. Implement if time allows; otherwise carry into normal maintenance.

F1. Decompose VoiceContactCenterCallRouter (23 ctor deps) into an ordered, testable routing pipeline (was W10.3).
F2. Collapse remaining catalog-CRUD controller duplication onto the generic base introduced in W11.5 (was W10.4).
F3. Decompose the two remaining mega-files (AsteriskContactCenterVoiceProvider, EnterpriseInteractionReportProvider) (was W10.7).

Group G — Honest capability verification

G1. Prove the base voice path at deployment; default-disable only Recording (was W5 audio/restart/capacity). Value: AsteriskContactCenterVoiceProvider advertises Recording | Monitor | Whisper | Barge, and the snoop/bridge implementations exist and are unit/cassette-tested — what is unproven is the end-to-end WebRTC audio path (certs/TURN/ICE), which is a property of a deployment, not of the capability code. §1 lists supervision as in-scope, so blanket-disabling Monitor/Whisper/Barge would under-advertise implemented features (the opposite honesty error). But inbound/outbound/transfer/conference all ride the same audio path, so it must be proven once. Approach (right-sized, but a real gate):
- Base-voice deployment/release gate (fail-closed in Production): a successful reference execution of the audio proof — direct-ICE and forced-TURN media established, plus a restart/drain + dependency-failure check and a measured capacity floor — must be performed against the reference topology and its evidence recorded before a deployment is declared production-supported. Live WebRTC E2E is not required in CI; use the existing recorded-cassette/signaling harness (W5.4) for the CI-provable portion and require the live media proof as a documented deployment acceptance step (D3/E2) with captured evidence. In Production, until the operator acknowledges the audio-verification step, the node fails closed (readiness withheld), not merely warns; outside Production a startup warning suffices. Do not hide implemented supervision capabilities.
- Recording default-off only: because the unproven-audio risk compounds with the C1 media-lifecycle/erasure risk, ship Recording off by default, enabled after the same proof passes for a deployment. Monitor/Whisper/Barge remain advertised (they are implemented and in-scope). Exit: the CI-provable signaling portion is exercised by the cassette harness; the live direct-ICE + forced-TURN + restart/failure + capacity-floor proof procedure and its evidence template are in src/CrestApps.Docs and referenced as a base-voice deployment acceptance gate; a successful reference execution against the reference topology has completed and its captured evidence is retained (linked from the docs); in Production an unacknowledged audio verification withholds readiness (test proves fail-closed), non-Production emits a warning; Recording is off by default and a test asserts that; Monitor/Whisper/Barge stay advertised. Progress (code + docs landed; box stays open): the fail-closed readiness check, AudioVerificationAcknowledged ⇒ AudioVerificationEvidenceReference config validator (ValidateOnStart), production/non-production startup warning, RecordingEnabled default flip to false, acceptance procedure + evidence template + validated-settings doc, and their tests are all landed and green (gate:pr_ci.yml#build_test). The box remains unchecked because the one sub-criterion the CI cannot produce is the live reference execution itself — direct-ICE + forced-TURN + restart/drain + capacity-floor against a real Asterisk/coturn/browser topology (the scaffolded AsteriskBrowserAudioE2ETests.BrowserToAsteriskWebRtcAudio_WithDirectIceAndForcedTurn_VerifiesReceivedToneFrequencies is [Skip] for exactly this reason). That run and its retained evidence are the outstanding operator step (the live audio proofs and the capacity certification), which stay open.

5. De-scoped as over-engineering or process overhead (removed / not doing)

Removed with rationale. These do not add real value proportional to their cost for a first production release of an Orchard Core module:

Per-gate versioned test specifications and the gate-ledger-evidence meta-test that parsed plan documents and enforced gate: annotations on every checkbox. Right-sized regression tests per change replace this; the generic capability-coverage test (A4) preserves the one behavioral rule worth keeping (no capability without an implementation). (The plan-parsing governance tests and JSON ledgers were removed in §7 cleanup (T1).)
KnownDevelopmentValues SHA-256 digest register + asset-rescanning test — deleted entirely (A1); the mandatory Production rejection of checked-in credentials is inlined directly at both call sites with no shared static register.
Durable pre-buffer event journal with "zero lifecycle loss under saturation" (old W12.13) — folded into D1 (bounded backpressure + coalesced reconciliation) with the loss limitation documented honestly. A separate durable journal is Track B rigor.
Bespoke Contact Center preview export/reset service (ContactCenterPreviewMaintenanceService) — retired in favour of Orchard Deployment/Recipes; because this involves public-API removal and a behavior change to a destructive admin action, it is a tracked checklist item (A5), not merely a de-scope. Rationale: it exports before quiescing, its "receipt" hashes row counts not content, it has no import/restore, and its config export duplicates Deployment/Recipes. Operational record/media backup is the datastore/provider's responsibility, documented in E2.
Pure release-tracking JSON ledgers and their plan-parsing meta-tests as committed artifacts — the pr-test-control-matrix.v1.json P0/P1 finding→gate ledger, the r0b-harness-dependency-ledger.v1.json, service-objectives.v1.json, and the gate-ledger-evidence / control-matrix / R0b-harness meta-tests that only parsed those documents — were plan-tracking scaffolding, removed in §7. Not removed: the runtime/architecture-contract ledgers that an ordinary (non-plan-parsing) test actually consumes — support-matrix.v1.json (the topology-profile mirror asserted identical to ContactCenterTopologyProfiles), feature-dependency-violations.v1.json (the architecture-characterization ledger ContactCenterFeatureDependencyArchitectureTests reads), and feature-lifecycle-contracts.v1.json (the lifecycle contract ContactCenterFeatureLifecycleTests reads) — these encode real behavioral contracts and stay. The behaviors that matter (topology validation, no-Elasticsearch, feature-dependency correctness) remain enforced by those ordinary tests.

6. Deferred to Track B (post-release; explicitly not started)

Real gaps, but none blocks a credible single-active-process voice-ACD release; starting them now multiplies surface over the current base. Each ships as a documented known limitation with an operator note:

Concurrently-active multi-node (distributed ARI ownership + cross-process channel-ready coordination): the deferred core of old W3.1/W3.2. v1 is single-active-process (§1, B6); running 2–4 concurrently-active nodes requires a distributed ownership lock and cross-process channel-ready signalling. Documented as unsupported for v1.
Open channel model (old W10.6): InteractionChannel stays a closed enum for v1. Consequence to document: because W11.8 locked the public-API baseline, adding chat/email/SMS channels later is a post-GA breaking API change. Accepted for a voice-only v1; stated explicitly so it is a conscious trade, not a silent one.
Media/telephony depth: dual-channel/stereo recording (old W12.1 — v1 is mono, documented as mono with no diarization); codec negotiation & SRTP/DTLS enforcement & TURN credential lifetime (W12.3, minus any trivial hardening pulled forward in E1); orphan bridge/channel sweeper beyond existing reconciliation + optional ARI channel-list adoption (W12.6); PBX failover across multiple ARI endpoints (W12.7 — external PBX/SBC HA is the operator's responsibility); media-quality telemetry MOS/jitter/loss (W12.8); DTMF/IVR, music-on-hold, call park, CLI presentation (W12.9 — direct-DID queue routing only in v1); STIR/SHAKEN passthrough (W12.10 — carrier/SBC responsibility, regional limits documented).
Signed origination markers (W12.12): evaluated in E1 → accepted as a documented trusted boundary. The CRESTAPPS_ORIGINATED channel marker is forgeable in theory but is not the security boundary; tenant-unique ARI application ownership + store-driven call bindings are the real guards, so Asterisk/dialplan is a trusted boundary. The signed-HMAC marker stays deferred (the fix is not small and adds no boundary the ARI-app ownership does not already provide).
Product-wide accessibility & localization beyond the agent desktop (W14): admin/supervisor/report a11y and full i18n. Agent desktop (W6) + server-side localized display values are done; whole-product axe coverage is Track B.
Move agent-session heartbeat timestamp out of the session document (W16.7 second half).
Multi-region, chat/email/SMS channels, predictive dialing, WFM/QM, skill proficiency/bullseye routing, virtual hold.

7. Final tasks (run last, in order)

T1. Clean up planning scaffolding so the PR is merge-ready. Remove .playwright-mcp/; remove the superseded plan docs (PLAN.md, PLAN-2-SINGLE-NODE-COMPLETION.md, PLAN-3-PRODUCTION-SINGLE-NODE-DISTRIBUTED.md, R0-BASELINE.md) and the pure release-tracking ledgers (pr-test-control-matrix.v1.json, r0b-harness-dependency-ledger.v1.json, service-objectives.v1.json); remove the test/CI that only exists to parse those documents (the ledger-evidence, PR-test-control-matrix, and R0b-harness meta-tests) and the docs pages that only describe them (pr-test-control-matrix.md, service-objectives.md); update .github/copilot-instructions.md to drop references to the removed files. Keep the runtime/architecture-contract ledgers an ordinary test consumes — support-matrix.v1.json, feature-dependency-violations.v1.json, feature-lifecycle-contracts.v1.json — and their consuming tests. Keep this PRODUCTION-READINESS.md as the short retained record (drop its transitional ledger-authority marker once the meta-tests are gone). Exit: no plan-tracking scaffolding remains; the retained runtime-contract ledgers and their tests still pass; build + tests green after removals.
T2. PR-wide implementation deep-dive. Review every changed file in this PR: correct implementation, reuse of pre-existing design patterns, extension of Orchard Core features where possible, and clean code per .editorconfig and copilot-instructions.md. Fix what falls short. Exit: review complete; strict Release build with 0 warnings; full unit suite green.
T3. Documentation review. Deep-dive the src/CrestApps.Docs pages: ensure they align exactly with the shipped code, are internally consistent, and all links resolve. Update the changelog file matching VersionPrefix. Exit: docs site builds; link check passes; claims match code.
T4. Aspire host smoke test. Run src/Startup/CrestApps.Aspire.AppHost locally and confirm it starts with no failure. Exit: the AppHost boots cleanly.

Progress log

Append a one-line entry per completed item (item id, commit, build/test result). This replaces the previous multi-thousand-line evidence ledger.

Plan finalized & cross-reviewed (GO/GO). Consolidated from PLAN/PLAN-2/PLAN-3/R0. Independently reviewed by GPT-5.6 and Opus-5; both returned GO with no remaining blockers after three revision rounds (health-check integration corrected to functional Redis/lock/backplane DependencyTag probes with no bespoke aggregate; C1 hardened to atomic recording/media erasure with tombstone + retention legal-hold fix + fail-closed tenant removal; single-active-process scope decision with multi-node deferred to Track B; G1 base-voice audio proof made a fail-closed Production release gate; KnownDevelopmentValues deleted).
A1 done (commit 0e79b59b). Deleted KnownDevelopmentValues + its test; inlined explicit dev-credential rejection at both call sites (DefaultAsteriskOptionsValidator, AsteriskSoftPhoneRegistrationConfigContributor). Independent GPT-5.6 review: GO (added exact TURN-secret validator test per its non-blocking note). Asterisk module builds 0 warnings; 9 focused tests pass.
A2 done. Added three functional DependencyTag probes without any bespoke aggregate: ContactCenterDistributedLockHealthCheck (acquire/release a dedicated probe lock), ContactCenterRedisConnectivityHealthCheck (bounded ConnectAsync+PingAsync), and ContactCenterBackplaneHealthCheck (bounded pub/sub round-trip on an invocation-unique, tenant-qualified channel). All resolve IRedisService optionally and report Healthy-when-absent (topology validator, not the probe, decides whether Redis is required). No hard dependency on OrchardCore.HealthChecks. Backplane check allowlisted in the architecture guard (legit ISubscriber need). Independent GPT-5.6 review: initial NO-GO on 3 blockers (arch-guard, non-unique unsubscribe channel, unbounded ops) then a 4th (subscription leak on subscribe-timeout) — all fixed; final GO. 10 focused tests pass (incl. a Moq timeout-cleanup regression test); module builds 0 warnings.
A3/A4/A5 done. Dead/dishonest-surface removal in one landing: A3 deleted the never-implemented generic provider-webhook ingress (IProviderVoiceWebhookAdapter, HmacProviderVoiceWebhookAdapterBase, ProviderVoiceWebhookProcessor, endpoint + models + 2 test fakes; DI/endpoint mapping removed from Startup). A4 removed MonitorMode.TakeOver, the TakeOver capability bit (1 << 12), its ContactCenterMonitoringService entry and ResolveCapability arm, and the supervisor-dashboard take-over control; added ContactCenterVoiceProviderCapabilityCoverageTests asserting every capability bit is provider-declared or explicitly reserved (replaces the removed governance gate). A5 deleted the entire Maintenance feature (Abstractions/Maintenance + Core/Maintenance surface, service/controller/views/view-models/admin-menu/permission, manifest Feature block, ManagePreviewData permission, startup) — config export/import now flows through Deployment/Recipes only and the destructive preview reset is removed entirely (operational recovery = DB backup/restore per E2). Regenerated PublicApiApprovalTests baselines for ContactCenter.Abstractions + .Core; applied a minimal governance-ledger patch (removed the take-over/webhook/maintenance references now, full release-ledger removal deferred to T1) and cleaned all docs stale refs. Independent GPT-5.6 review: NO-GO on stale-reference residue → all fixed and under final confirmatory claude-opus-5 review (this line is updated to the recorded verdict only once that review returns GO). Governance (113), PublicApi/capability/monitoring (143), headless-closure (3), lifecycle (18), support-matrix + feature-dependency-architecture suites all green.
A2 follow-up correctness fix (folded into the A3/A4/A5 landing). The original A2 commit left four architecture-governance tests red because the Redis probes took an optional IRedisService — the exact pattern the pinned ContactCenterOptionalDependencyTests invariant forbids (an optional injected collaborator silently changes behaviour when its owning feature is off). Reconciled to the established feature-owns-its-dependency pattern (identical to how the Voice feature owns the provider-ingress check): made IRedisService mandatory on ContactCenterRedisConnectivityHealthCheck and ContactCenterBackplaneHealthCheck, moved all three distributed-dependency probes (distributed-lock, Redis-connectivity, backplane) out of the base feature into a new AddContactCenterRedisHealthChecks() registered by a [RequireFeatures("OrchardCore.Redis")] startup. The base-feature health-check set is now identical to pre-A2, so the readiness/dependency-split contract tests pass unchanged; added a registration test proving the three probes are Redis-gated and dependency-tagged, and a guard test proving the base feature registers none of them; dropped the two now-impossible "healthy when Redis absent" probe tests; corrected the options-discovery floor after A5's options-type removal. Full ContactCenter suite 1477 pass; feature-activation 55 pass.
B1 done (commit f1e02f43). The listener-start ordering (provider-local validation via HasRequiredConfiguration + ARI ownership via IAsteriskAriApplicationGate.TryAcquire, both in AsteriskRealtimeVoiceTenantEvents.ResolveListeners) and Contact Center admission gating (IContactCenterFeatureWorkManager.TryEnter → ContactCenterTopologyState.IsAdmissible, fail-closed) were already implemented; the admission half was already covered by three ContactCenterFeatureLifecycleTests. Added the missing listener-side proof: introduced an IAsteriskRealtimeVoiceListener seam (concrete now implements it; DI registers by interface; IAsyncDisposable disposal preserved) so AsteriskRealtimeVoiceTenantEventsTests can assert the WebSocket opens only after both validation and ownership pass, and that the ownership gate is not even consulted when provider-local validation fails. Independent GPT-5.6 review: GO. Asterisk module builds 0 warnings; Telephony+ContactCenter 1967 pass (1 skipped); feature-activation 55 pass (2 skipped).
B2 done (commit 5dcb4085). Investigation confirmed the durable path (ProviderCommandProcessor) already settles OutcomeUnknown on timeout/cancellation, and the monitoring/transfer synchronous services already return explicit Unknown on Timeout/OCE — the only gap was ContactCenterRecordingService collapsing unknown outcomes to a silent false. Added TelephonyCommandNotAdmittedException : OperationCanceledException, thrown by DefaultTelephonyCommandExecutor at entry when ApplicationStopping is already requested (provider never contacted ⇒ definite non-application); deriving from OCE lets existing monitoring/transfer catches degrade to Unknown while precise callers catch it first for a definite Failure. Changed IContactCenterRecordingService Start/Pause/Resume/Stop to return a tri-state RecordingCommandResult (mirrors TransferResult): NotAdmitted/guard/governance → Failure; timeout/in-flight OCE/unconfirmed provider outcome → Unknown; confirmed change → Success. Added executor + recording shutdown tests and regenerated the Telephony.Abstractions and ContactCenter.Core public-API baselines. Independent GPT-5.6 review: GO. ContactCenter.Core + Telephony build 0 warnings; ContactCenter+Telephony+PublicApi 2004 pass (1 skipped); feature-activation 55 pass (2 skipped).
B3 done. Investigation found the only report provider with mutable per-execution instance fields is EnterpriseInteractionReportProvider (_agentUserNames, _absentFeatureIds), both re-captured from injected tenant-scoped services at the start of every RunAsync and independent of the per-call ReportContext. No leak is reachable: the provider is registered AddScoped<IReport> so concurrent requests never share an instance, and reuse re-captures both fields each call. Closed with a test-only proof and no production change (plan's "prove before refactoring; do not refactor if no defect" directive): added EnterpriseInteractionReportConcurrencyTests with (a) ConfigureServices_RegistersEveryReportProviderWithScopedLifetime — runs the real AnalyticsStartup.ConfigureServices and asserts every IReport descriptor is ServiceLifetime.Scoped, so a future singleton misregistration (the only way concurrent requests could share an instance) fails the build; and (b) RunAsync_WhenOneInstanceIsReusedAcrossRequests_DoesNotCarryStaleCapturedState — a real-SQLite behavioural test flipping capability + agent state between two runs on one instance and asserting no stale voice columns or agent names carry over. Independent GPT-5.6 review: initial NO-GO (the first draft's two-separate-instance "overlap" test and gate could not observe a field leak) → replaced with the scoped-lifetime registration assertion + kept the reuse test → re-review GO. 2 focused tests pass; full Reports suite 184 pass.
B4 done. AsteriskRealtimeVoiceEventMapper.BuildIdempotencyKey hashed the raw payload string, so a whitespace/property-order or numeric-token change from an Asterisk upgrade or a re-serializing proxy would silently defeat dedupe and double-process. ARI events carry no eventId/sequence, so the key is now built from a canonical form of the payload: object properties ordered by ordinal name, arrays preserved in order, numeric tokens normalized (Int64/UInt64 exact, else shortest round-trippable finite double, else raw fallback), SHA-256 hashed and prefixed {provider}:{eventType}:. Re-serialized redeliveries of one event dedupe; two genuinely distinct same-type events on one call (differing in timestamp/varset value/sub-state) keep distinct keys and are both processed — avoiding the event-loss failure of a coarse (provider, callId, type) key. Added mapper tests for property-order re-serialization, numeric re-serialization (16 vs 1.6e1), two distinct same-type hold events on one call, and the key prefix. Independent GPT-5.6 review: initial NO-GO (numeric tokens not canonicalized) → added Int64/UInt64/double numeric normalization + a numeric re-serialization test → re-review GO. Asterisk module builds 0 warnings; Telephony+Asterisk suite 640 pass (1 skipped).
B5 done. Closed as already-correct with a regression guard (plan's escape hatch). Verified MediaTopologyId is absent from all production src/**/*.cs; the live CallSession topology is fully typed (Bridge, ConsultCall, CallLeg, CallRelationship, MonitorSession; ConsultCall.TargetType is the typed InteractionTransferTargetType); the only historical transfer strings are the audit-only InteractionTransferHistoryEntry.TargetType/.Result, which no neutral consumer parses back into typed values (report providers only read TransferHistory.Count or group the raw strings for display). Kept the two fields as string (retyping would change the persisted YesSql JSON of existing interactions — the plan forbids a compatibility break). Clarified the two fields' XML docs as historical text snapshots that are never re-parsed, and added ContactCenterTransferTopologyTypingArchitectureTests (live consult topology uses the typed enum; no live topology model exposes MediaTopologyId or a string TargetType; MediaTopologyId stays absent from production source). Independent GPT-5.6 review: GO. 3 focused tests pass. Note for T2: the distributed-test harness ContactCenterStoreTestHarness still declares a stale, larger CallSessionIndex schema (extra MediaTopologyId/ConferenceId/SupervisorAgentId/SupervisorLegId/RecordingId/AgentSessionId/DurableCommandId columns not on the current 11-column production index) — test infra only; fold the harness resync into the T2 PR-wide deep-dive.
B6 done (b4dc0adb): AsteriskAriApplicationOwnershipRegistry.TryClaim now fails closed — blank baseUrl/applicationName/tenantName/ownershipToken are denied and logged (was silently granting). Injected ILogger; updated interface contract doc. Added a startup information log stating the single-active-process ownership constraint when the Asterisk listener starts. Documented the non-overlapping deployment requirement in runbooks.md (new "Asterisk single-active-process listener ownership" subsection under Provider failure). Updated all registry/gate test call sites for the new ctor and rewrote the blank-claim test to assert denial + added a blank-tenant-name test. 41 ownership/gate tests pass. gpt-5.6-sol rubber-duck: NO-GO (runbook lost ## Node failure heading + misleading unique-per-tenant guidance) → fixed → GO.
B7 (2025): ReservationExpiryBackgroundTask now bounds each run to a 90s wall-clock budget (IClock between-op checks + hard CancelAfter on a linked CTS) < the 120s lock, participates in the Routing work-admission drain (TryEnter + registered ContactCenterFeatureWorkLifecycleParticipant), propagates shutdown cancellation, and ActivityReservationService.ExpireDueAsync observes cancellation between per-item lock acquisitions. Contract ledger → implemented-r3. gpt-5.6-sol GO. Commit c94ec207.
B8 (2025): DialPadTelephonyProvider.ReadDateTimeOffset parses with CultureInfo.InvariantCulture + AssumeUniversal | AdjustToUniversal so offset-less provider timestamps are UTC (not server-local); helper made internal for cross-culture unit tests. gpt-5.6-sol GO. Commit 6c4aa4d4.
C1 done (commit f3f466ff; ledger W17). Recording/media erasure & lifecycle proven end-to-end: authenticated RecordingErasureEndpoint → EraseAsync clears Interaction + CallSession pointers, stamps the tombstone, and enqueues durable outbox media deletion in one YesSql unit of work; RecordingMediaDeletionHandler deletes encrypted bytes idempotently in a fresh scope and writes an audit-trail receipt; RecordingErasureGuard reads the tombstone through a fresh child scope so AsteriskRecordingIngestService refuses/cleans up a late ingest (deterministic-key delete even across a pre-persist crash); retention enqueues media deletion before row delete, excludes indexed RecordingLegalHold rows (per-record veto backstop), and runs each batch atomically — a failed batch is discarded with ISession.ResetAsync so no partial event-without-outbox can commit and orphan media; tenant decommission purges media via ISupportsTenantMediaPurge/RecordingMediaTenantEvents, fail-closed. gpt-5.6-sol rubber-duck: first NO-GO on 6 findings, second NO-GO on 3 (retention partial-commit, mirror-only pointer, legal-hold TOCTOU) — partial-commit fixed with the atomic batch discard, the other two validated unreachable (capture-time-only hold stamping; mirror always seeded from the interaction reference) → GO. ContactCenter + PublicApi 1,411 pass / 0 fail; ContactCenter + Asterisk modules build 0 warnings; docs site builds clean.
C2 done (commit 3d5865ed). Widened CallSessionIndex.ProviderCallId 128→256 and the derived ProviderCallClaimKey 261→385 (= ProviderName 128 + sep 1 + ProviderCallId 256; 770 bytes < SQL Server's 900-byte key limit) as a real schema change, not an additive no-op. Because SQLite has no ALTER COLUMN, widening runs through a new reusable IndexStringColumnRebuild.WidenAsync add/copy/drop/rename rebuild, dropping the unique claim index + covering index first and recreating them after. B1: SQL-Server-only pre-drop of the auto-named default constraint before DropColumn (name read from sys.default_constraints); the additive-migration guard gains a raw-SQL in-place-rebuild contract entry with an (anonymous) restored-object sentinel scoped strictly to raw-SQL entries. B2: tolerant per-index DropIndex unrolled to one call per literally-named index (MySQL has no DROP INDEX IF EXISTS). B3: real-PostgreSQL migration test proves both columns reach the wider length, a seeded value survives, and the claim-key UQ still rejects a duplicate (SQLSTATE 23505 + constraint name) — contact_center_operations_gates.yml#redis-backplane-two-node / release_ci.yml#distributed_test. N1: copy step resumes (early-return when temp column absent). N3: duplicate-claim-key preflight before recreating the UQ index aborts with repair guidance if a duplicate slipped into the transient index-absent window on an autocommitting engine; SQLite N-1 harness + additive guard + uniqueness throw-branch test via pr_ci.yml#build_test. SQL Server/MySQL are construction-correct-only (not exercised in the Postgres-only CI). Also fixed pre-existing CS0105 duplicate-usings blocking strict warnaserror builds. Two independent claude-opus-5 rubber-duck reviews → GO (blockers were documentation accuracy: corrected the reject-vs-truncate failure mode and the in-place-rebuild exception to the additive-only audit). Strict warnaserror builds of all 5 touched projects 0 warnings; 86 additive-guard + 16 uniqueness + 118 targeted + 156 Architecture + 3 Postgres migration tests pass; docs build clean. Heartbeat-relocation half of W16.7 stays deferred.
D1 done. Bounded real-time event channel with genuine backpressure + a saturation metric for the Asterisk voice listener. The listener already used Channel.CreateBounded(FullMode.Wait); the fix moved the write path to AsteriskRealtimeIngestionWriter (await Writer.WriteAsync with a bounded RealtimeEventBackpressureTimeout wait, default 5s) and emits asterisk.realtime.ingestion.saturated (meter CrestApps.OrchardCore.Asterisk) on the first wait of a saturation episode (episode reset only when the buffer fully drains). Buffer capacity is a validated option (RealtimeEventBufferCapacity, default 1000, upper-bounded by AsteriskConstants.MaxRealtimeEventBufferCapacity = 100_000; both > 0 and <= cap validators in Startup). On backpressure timeout the listener reconnects and runs the existing coalesced, pointer-driven reconciliation. Post-disconnect drain extracted to AsteriskRealtimeIngestionDrainer — progress-based and hard-bounded by max(backpressure timeout, 30s) so a healthy-but-slow dispatcher finishes while a stalled one is cancelled and abandoned (worst case ~30s offline, not tens of minutes). Reconnect backoff decays via a Stopwatch measuring receive uptime captured before the finally-drain and started after reconciliation, so drain/reconcile time never masquerades as healthy uptime. B1 (Asterisk precondition): client backpressure only reaches the provider as real flow control when Asterisk's [general] websocket_write_timeout exceeds the configured backpressure window; the sample ari.conf sets it to 10000 (ms) > 5s. Docs (production-support.md, runbooks.md, changelog v2.0.0) reworded to state the precondition, the bounded-drain caveat (a dispatch wedged in non-cancellable tenant-scope work can still delay reconnect), and that the stream is not lossless under any config. Three independent claude-opus-5 rubber-duck reviews → GO (rounds 2 and 3 fixed reviewer-found defects: backoff stopwatch had spanned the drain; the progress-based drain initially had no overall budget); all four final non-blocking suggestions (budget-exhaustion test, reconcile-excluded uptime, per-cause abandon log, doc precision) were also landed. Asterisk + Tests strict warnaserror builds 0 warnings; 687 targeted tests pass (1 skip, the WebRTC E2E) incl. 5 drainer + 6 writer + options-rejection tests via pr_ci.yml#build_test; docs build clean (only the pre-existing #backup-and-restore anchor). W12.13 durable pre-buffer journal stays out of scope (Track B).
D2 done. Reconciliation-liveness proof: the Asterisk ITelephonyCallStateProvider (AsteriskTelephonyProviderBase.GetCallStateAsync) is the surface reconciliation depends on (ProviderCallStateSynchronizationService/TelephonyInteractionSynchronizationService dispatch on a runtime is ITelephonyCallStateProvider check), so a new cassette-backed contract test AsteriskCallStateReconciliationContractTests drives the production provider against the pinned Asterisk release's recorded ARI channel (Cassettes/Asterisk/22.10.1/rest/responses.json, GET channels/{channelId}) and asserts it issues a live GET channels/{callId} and parses the recorded payload into TelephonyCallLookupResult (Found, CallState.Connected from "Up", From/To from the recorded caller/connected numbers); a second test proves a live 404 reconciles to Found=false; a third pins that AsteriskTelephonyProvider still implements ITelephonyCallStateProvider so reconciliation cannot silently skip ARI. Added a reusable AsteriskContractCassettes.TryReadRecordedRestResponse(...) accessor. The channelId/state/numbers are all read from the recorded body, so an Asterisk upgrade that changes the channel shape also fails here. Mutation-verified by the independent claude-opus-5 reviewer: short-circuiting GetCallStateAsync to return the exact cassette values without issuing the ARI GET makes both liveness tests FAIL (empty request collection) — proving the exit criterion "a cassette test fails if the provider stops querying ARI." The transition-table half (illegal-edge rejection via CallSessionLifecycle.CanTransition/CallSession.TransitionTo→InvalidStateTransitionException) was already covered by AggregateLifecycleTableTests/AggregateStateTransitionTests and re-verified. claude-opus-5 rubber-duck: GO (landed NB-1 interface-pin fact, NB-2 direction comment, S-3 doc wording). Tests strict warnaserror build 0 warnings; 3 new + full Telephony/ContactCenter 2029 pass (1 skip, WebRTC E2E) via pr_ci.yml#build_test; docs build clean (only pre-existing #backup-and-restore anchor). Also documented in production-support.md that restored state is read from live ARI and pinned by this contract test.
D3 done. Minimum operational signals + voice handover/rollback runbook. Signals: two new Asterisk meter counters (asterisk.realtime.connected, asterisk.realtime.reconnect_attempted, meter CrestApps.OrchardCore.Asterisk, tagged provider) wired into AsteriskRealtimeVoiceListener (connected recorded after ConnectAsync succeeds; reconnect recorded before each backoff Task.Delay), plus two store-count "gauge" health checks surfacing live counts as HealthCheckResult.Data — contactcenter-active-calls (active_calls, ICallSessionStore.CountActiveAsync = EndedUtc == null) registered by the base Area feature, and contactcenter-queue-backlog (queued_interactions, IQueueItemStore.CountAllWaitingAsync = Status == Waiting) registered by the Queues feature via a new AddContactCenterQueuesHealthChecks(). Counts are surfaced as health-data (not ObservableGauge metrics) deliberately: a gauge callback is synchronous with no ambient tenant scope and a process-global count is wrong across tenants/nodes, whereas a health check runs in tenant scope with an async body; both stay Healthy at any count (Unhealthy only on store read failure). The ownership split is required — a check must be registered by the feature owning all its store dependencies (dep chain Queues→Availability→Agents→Area) or it throws on a tenant that enables the registering feature without the owning one. Runbook: new ## Voice listener handover and rollback section in runbooks.md (linked from Rolling + Blue-green and from production-support.md), stating the single-active-process constraint, the prohibited overlapping deployment, the bounded-interruption reality (no zero-downtime claim), and explicitly what happens to live calls during the gap: bridged media keeps flowing while control pauses; channels parked in Stasis() with no dialplan continuation are stranded; and new inbound calls whose StasisStart was missed are not recoverable by reconciliation (store-driven, no ARI channel enumeration) — mitigation is a dialplan continuation + upstream carrier retry. Three independent claude-opus-5 rubber-duck reviews → GO: round 1 NO-GO caught (B1) unregenerated PublicApi baselines (branch red), (B2) a false "reconciled on reconnect" claim for missed-StasisStart calls, and (B3) a wrong active_calls recovery direction (it stays flat through the gap and only ever drops toward the true live count post-sweep) — all fixed; NB1 (provider tag is per-node/process, not per-tenant), NB3 (queue-backlog test now runs the real QueueItemIndexMigrations so Status is asserted against the shipped Column<QueueItemStatus>), and NB4 (deep link from the deployment doc) landed. The reviewer's NB2 mutation-gap point was then closed for the reconnect call site with a deterministic, non-flaky AsteriskRealtimeVoiceListenerReconnectMetricTests (real listener against a closed loopback port → ConnectAsync fails immediately → increment fires before any backoff timer; 3× repeat green, ~90ms). Asterisk + Tests strict warnaserror builds 0 warnings; new metric/health/endpoint/reconnect tests + full Telephony/ContactCenter 2039 pass (1 skip, WebRTC E2E) via pr_ci.yml#build_test; docs build clean (only the pre-existing #backup-and-restore anchor).
E1 done. Omnichannel/CRM layer review (was W13, PLAN-3 §W13) across ~287 .cs files / 5 areas — authorization & tenant isolation, migration safety, concurrency/CAS, retention/PII, background schedulers — plus the W12.12 origination-marker evaluation. Real defects fixed in AutomatedActivitiesProcessorBackgroundTask (S1): (1) pagination skip — the page query combined a moving DocumentId > cursor keyset with an increasing .Skip(iterationCount++ * _batchSize) OFFSET, advancing the window twice per batch and silently skipping every other page of due activities; replaced with pure keyset pagination (removed Skip/iterationCount). (2) cross-node double-send — the loop committed only once at end-of-run under LockExpiration = 90_000; on multi-node a run that outlived the 90s lease let another node re-acquire the distributed lock and reprocess still-uncommitted activities, and SmsOmnichannelProcessor.StartAsync sends with no idempotency key before flipping Status = AwaitingCustomerAnswer, so re-processing = duplicate SMS to real contacts. Fix: commit per batch; a wall-clock budget that stops a run at 60% of the lease so a still-running node ends well before its lease can lapse (OrchardCore does not cancel DoWorkAsync when the lease elapses, so per-batch commit alone does not stop an alive node from leap-frogging a peer — the budget is what does), enforced per item in both the expiry and send loops (not just per batch) so worst case is the 6-min budget plus a single in-flight send (≤ the ~100 s HTTP timeout) ≈ 7.7 min, still comfortably inside the 10-min lease; lease raised 90_000 → 600_000, deliberately above the 5-min schedule interval so it is not designed to expire exactly when the next tick fires; bounded work per invocation (_maxActivitiesPerInvocation = 1000). Honest bound: this does not make the SMS send idempotent — it means a crashed/killed node re-sends at most the single uncommitted in-flight batch (≤_batchSize), not the whole backlog, and an alive node never hands an uncommitted backlog to a peer. After the first per-batch commit YesSql keeps _save = true for the session, so a later throw commits the already-flushed in-flight batch instead of rolling it back — the desired behaviour here (sent messages stay marked). The lease is 10 min against a 5-min schedule, so a crashed node's lock is held for up to two ticks before another node retries its in-flight batch — an accepted trade to keep the lease safely above run duration. The catch also excludes OperationCanceledException, so a tenant shutdown/recycle during a send is not logged as a failure and does not consume a ProcessingAttempts slot. True per-item idempotency (an idempotency key on the send) remains follow-up S-1; the residual duplicate window is specifically a post-send throw (status flip / prompt persistence) after SmsOmnichannelProcessor.StartAsync has already sent, now bounded to at most _maxAttempts rather than unbounded. (3) cap-induced permanent stall (regression caught in round-1 review, B1) — with the new per-invocation cap, the pre-existing no-op failure path (a caught StartAsync exception left the activity NotStated/due, Attempts never incremented) would let one permanently failing activity (e.g. a misconfigured AI profile) occupy a slot on every invocation and starve all healthy outbound; the catch now transitions the failure on a separate internal ProcessingAttempts counter (a document-only field, not the routing-owned, indexed, report-and-UI-surfaced Attempts that the contact-center work-state projector overwrites) — it increments ProcessingAttempts, reschedules with linear backoff, and marks Status = Failed at _maxAttempts — so a failing activity always leaves the due set and the counter can never be reset out from under it. (4) expiry head-of-line stall (NB-2) — ExpireNoResponseActivitiesAsync took an unordered page and continued no-timeout rows with no state change, so ≥_batchSize such rows would permanently block expiry; it now restricts the query to the subject content types whose flow actually defines a no-response timeout (computed once per run from the flow settings) and keyset-pages within them, so no-timeout conversations never enter the candidate set at all — no per-row flow lookup and, crucially, no sentinel written into the user-visible, sorted, filtered ScheduledUtc field. One pre-existing edge is accepted (not introduced here): because no-timeout flows never stamp a deadline, if an operator later adds a timeout to such a flow, its in-flight AwaitingCustomerAnswer conversations carry a stale past ScheduledUtc and would be mass-expired on the next tick until the customer next replies — identical to the pre-E1 behaviour, tracked as a follow-up (arm the deadline from ModifiedUtc when ScheduledUtc predates the status transition). Tests (AutomatedActivitiesProcessorBackgroundTaskTests): a 250-activity case proving every due activity is processed exactly once (teeth-verified: fails at 150/250 with the Skip regression) and a two-invocation failure case proving a throwing activity is attempted once and not re-attempted the next tick (teeth-verified: 200 vs 150 without the failure transition). Inbound webhook hardening (NB-3/NB-4): TwilioWebhookEndpoint built its signed URL from the raw request scheme/host and omitted the path base — silently Forbid()ing genuine deliveries behind a TLS-terminating proxy — and used signature.First() (empty header → 500, not 403); it now reuses the Event Grid endpoint's tested IsRequestValid/GetExternalRequestUrl helper (honours site.BaseUrl + path base, signature.ToString()), so both inbound SMS paths share the one validator covered by TwilioEventGridEndpointSignatureTests. Dismissed as governed (F1/F2): every migration Drop in OmnichannelActivityIndexMigrations.UpdateFrom3Async (2) and OmnichannelContactsMigrations (6) is registered in MigrationAdditiveOnlyGuardTests with MigrationContractJustification.NeverReleased — the guard verifies that against real git release tags (columns/tables existed only in the unreleased 2.0.0 line). Accepted with dated rationale (W12.12, 2026-07): the CRESTAPPS_ORIGINATED channel marker is forgeable in theory but is not the security boundary — tenant-unique ARI application ownership + store-driven call bindings are the real guards; Asterisk/dialplan is recorded as a documented trusted boundary and the signed-HMAC marker stays deferred. Accepted with dated rationale (PII P1–P3, 2026-07): plaintext SMS body/from/to at rest, no automated CRM retention purge, and no per-contact subject-erasure are deferred for the unreleased product — at-rest encryption is a datastore/deployment concern, and E2 must state explicitly that OmnichannelMessage.Content/CustomerAddress/ServiceAddress are stored plaintext (recording media is separately advertised as encrypted at rest, so silence here would mislead); per-contact subject erasure is recorded as a GA blocker (a legal obligation at general availability), not open-ended post-GA. No open high-severity CRM finding remains. Independent claude-opus-5 rubber-duck review (three adversarial rounds) → GO — round 1 caught B1 the cap-induced stall, B2 the over-claimed ≤batch bound, and B3 the count-cap-≠-time-budget error; round 2 caught B4 (the failure counter reused the routing-owned Attempts field, which the work-state projector overwrites — resetting the retry ladder and corrupting reports), B5 (the wall-clock budget was enforced only per batch, so a full page could still run past the deadline), and B6 (the no-timeout deferral wrote a year-3026 sentinel into the user-visible ScheduledUtc); all fixed and re-verified — B4 by repo-wide grep and the post-reload test assertion that the routing-owned Attempts stays 1 while ProcessingAttempts records the failure, B5 by the per-item budget placed before the cursor advance in both loops, and B6 by an empirical probe confirming zero ScheduledUtc writes and exactly the right expiry set. Recorded verdict: GO. Two pre-existing, unchanged schema follow-ups the reviewer noted are tracked separately: the SubjectContentType index column is WithLength(26) (the pre-filter is now correctness-load-bearing on it — widen toward 255 like ContactContentType), and no SQL index matches the processor's query shape (Status, InteractionType, ScheduledUtc, Channel/SubjectContentType, DocumentId), a scan-plus-filter given the 12k/hr target. Omnichannel.Managements + Sms + Tests strict warnaserror builds 0 warnings; new tests + full Modules.Omnichannel suite pass via pr_ci.yml#build_test; docs build clean (only the pre-existing #backup-and-restore anchor).
E2 done. Operator documentation truthful & complete (PLAN-3 §E2), a docs-only truthfulness pass over src/CrestApps.Docs verifying every operator claim against shipped code. Five scope items delivered: (1) data-residency/plaintext disclosure — new ## Data at rest and privacy section in omnichannel/management.md states OmnichannelMessage.Content/CustomerAddress/ServiceAddress are stored plaintext (addresses also projected plaintext into OmnichannelMessageIndex), contrasts the separately-advertised encrypted recording media store, and records no-automated-subject-erasure as a GA blocker (closes the E1 P1–P3 carry-over); (2) recording-is-mono — refined the contact-center/index.md "Encrypted media store and secure ingest" paragraph to conversation-bridge mixed mono WAV, with monitor/whisper NOT captured (separate supervisor bridge) and barge IS captured (joins the recorded conversation bridge), diarization still needed; (3) single-node-distributed reference deployment — added the node-census clause to contact-center/production-support.md clarifying the single-active-process constraint is not runtime-verified (ContactCenterTopologyEvaluator never reads Min/MaximumApplicationNodes) and is only mitigated by the distributed lock serializing the critical sections that take it, not by a count check; (4) emergency-calling scope + operator warning — rewrote the contact-center/voice-routing.md :::danger Not an emergency-calling service admonition to disclose that ExternalDestinationPolicy denial is PARTIAL (server-side first-dial/orchestrated-transfer only) and the agent soft-phone bypasses it entirely (TelephonyHub.DialAsync→AsteriskTelephonyProviderBase.DialAsync validates only non-empty To), so operators must block emergency/premium ranges at the Asterisk dialplan / SIP trunk; a bare short code is refused earlier by the E.164 form and minimum-length gate, not the emergency check; (5) new ## Data residency table in production-support.md enumerating every system that holds/processes customer data — tenant SQL DB (plaintext content), recording media store, Asterisk host's transient unencrypted source file, Redis backplane (in-transit call-state payloads + lock keys, not persisted state), third-party SMS/email providers (Twilio, ACS — email+SMS, no ACS voice module), third-party voice provider (DialPad, agent-device-native so it holds ALL call media and Orchard never bridges it), and the AI completion provider (receives inbound SMS bodies via SmsOmnichannelEventHandler→_aICompletionService.CompleteAsync). Also fixed a pre-existing broken #backup-and-restore anchor and added a governance cross-ref bullet stating a Contact Center Interaction is deleted outright at retention (ContactCenterRetentionPolicyBase.cs:145 DeleteAsync) while OmnichannelMessage has no retention window. Changelog v2.0.0.md bullet updated to carry these operator-facing truths. Independent claude-opus-5 rubber-duck review (three adversarial rounds) → GO — round 1 caught B1 (emergency admonition contradicted by the unfiltered soft-phone path — my "no emergency-origination path" claim was an overclaim), B2 (approved-destination transfer catalog has zero production consumers — claim over-strong), B3 (no actual residency statement delivered); round 2 caught B4 (a new false "addresses anonymized on retention" bullet — Anonymize is enum/catalog-only with zero implementation, retention DELETES the row), B5 (residency table mislabeled ACS as voice and omitted DialPad + the AI provider), B6 (catalog still presented as an operative control despite the orchestrated transfer path having no agent-facing caller); all fixed and re-verified against code, plus NB1–NB7 landed (mono/barge precision, E.164-form gate, "mitigated by" vs "upheld by", Redis in-transit wording, orchestrated-vs-soft-phone transfer reality). Recorded verdict: GO. Pre-existing unimplemented "Anonymize" claim in the per-entity governance table noted but deliberately left untouched (out of E2-added scope, tracked separately). Docs build clean via pr_ci.yml#build_test (zero broken anchors).
T1 done. Planning-scaffolding cleanup (§7 T1). Removed .playwright-mcp/ (30 files), the four superseded plan docs (PLAN.md, PLAN-2-SINGLE-NODE-COMPLETION.md, PLAN-3-PRODUCTION-SINGLE-NODE-DISTRIBUTED.md, R0-BASELINE.md), the three pure release-tracking ledgers (pr-test-control-matrix.v1.json, r0b-harness-dependency-ledger.v1.json, service-objectives.v1.json), the plan-parsing meta-tests (the Governance ledger-evidence test + WorkflowJobCatalog, and the ContactCenter PR-test-control-matrix / R0b-harness / service-objectives tests), and the two docs pages that only described them (pr-test-control-matrix.md, service-objectives.md). Kept the runtime/architecture-contract ledgers that ordinary tests consume — support-matrix.v1.json, feature-dependency-violations.v1.json, feature-lifecycle-contracts.v1.json — and excised only the one orphaned, vacuous EveryKnownViolation_OwnsAControlMatrixGate method (empty knownViolations) from the kept ContactCenterFeatureDependencyArchitectureTests. Updated .github/copilot-instructions.md and the docs (index / production-support / public-api-surface / runbooks + changelog v2.0.0.md) to drop references to the removed files, and added .playwright-mcp/ to .gitignore. This also removes the LedgerEvidenceTests.AuthoritativeLedger_BacksEveryCompletedItemWithARealCiJob meta-test that was the sole red gate on PR #501 (the W13 review-type gate that structurally cannot name a CI job), turning CI green. Full main test project 3733 passed / 0 failed / 1 skipped; strict warnaserror build 0 warnings; docs build clean (gate:pr_ci.yml#build_test). Independent claude-opus-5 rubber-duck review → GO.
F1 done. Decomposed the 718-line VoiceContactCenterCallRouter (§4 Group F) into three focused, single-responsibility services with no behavior change: (1) InboundVoiceCallProcessor (IInboundVoiceCallProcessor) owns inbound routing — distributed-lock + work-lease orchestration, flow/contact/queue resolution, activity + interaction creation, and terminalization; (2) VoiceQueueOfferService (IVoiceQueueOfferService) owns OfferNextAsync (reserve next agent for a queue and offer the queued call); (3) VoiceContactCenterCallRouter is now a thin 114-line facade implementing both public interfaces (IVoiceContactCenterCallRouter + IInboundVoiceService), delegating inbound→processor and offer→offer-service and keeping outbound in place. All method bodies moved verbatim; the shared IContactCenterFeatureWorkManager singleton preserves the nested lease semantics of the inbound→offer path. Three static architecture/retention guards were truthfully repointed to the file that now carries the moved code (InboundVoiceCallProcessor.cs): the scope-executor guard (ContactCenterFeatureDependencyArchitectureTests), the _settlementWriters["Interaction"] retention guard (ContactCenterRetentionCoverageTests), and the aggregate-lifecycle scanned-files spot-check (AggregateLifecycleArchitectureTests). No public-API baseline change (the concrete router is not in any *.approved.txt; only the two untouched interfaces are). Full main test project 3733 passed / 0 failed / 1 skipped; strict warnaserror build 0 warnings (gate:pr_ci.yml#build_test). Independent claude-opus-5 rubber-duck review → GO (verbatim relocation confirmed method-by-method, lease/lock pairing unchanged, all three guards verified truthfully enforced; the one clarity suggestion — disambiguating the new offer service from the pre-existing IQueuedVoiceWorkOfferService — was applied by naming it IVoiceQueueOfferService).
F2 done. Collapsed 7 structurally-identical catalog-CRUD admin controllers (Queues, QueueGroups, EntryPoints, DialerProfiles, AgentStateReasonCodes, BusinessHoursCalendars, Skills) onto a new generic base ContactCenterCatalogController<TModel> where TModel : CatalogItem, INameAwareModel, new() with no behavior change — net −699 lines (610 insertions / 1309 deletions). The base owns all list/filter/create/edit/delete orchestration once (auth Forbid, pager + Options.Search round-trip, SummaryAdmin display, CatalogEntryValidation.ValidateAsync before the ModelState check, NotFound on missing id, success notifications); each derived controller keeps only the thin routing shell — [Admin("url","route-name")] + [Feature] + [HttpPost]/[ActionName]/[FormValueRequired] attributes, its specific IXxxManager/IStringLocalizer<TController>/IHtmlLocalizer<TController> ctor wiring, and the literal S["…"]/H["…"] label overrides (kept literal so localization extraction is unaffected). Routes, route names, permissions (incl. EntryPoints/BusinessHoursCalendars deliberately reusing ManageQueues), verbs, localized strings, and the nameof(Model) validation prefix (now typeof(TModel).Name) are all byte-identical to the originals. View resolution is unchanged: base View(vm) resolves the view name from the executing action descriptor and the location from the runtime (derived) controller name, so Views/{Controller}/{Action}.cshtml still binds. AgentEntitlementsController intentionally excluded — it is bespoke (no IDisplayManager editor pipeline, no Delete, custom UserManager binding + duplicate-user validation, writes via _presenceManager.UpdateEntitlementsAsync), so forcing it onto the base would change behavior. Architecture-guard erosion fixed in the same change (review Issue 1): ValidationOwnershipArchitectureTests.EveryConfigurationCatalogEditor_ValidatesThroughTheHandlersBeforeSaving selected editors by IDisplayManager<Entity> and would have passed vacuously for the 7 controllers once their write sites moved into the IDisplayManager<TModel> base (whose TModel is not in the entity set); the guard now follows inheritance — when a derived editor forwards to a base controller it also inspects the base file's members — so removing ValidateAsync from the base fails all 7 (teeth-verified: negative probe produced 7 violations, restored to green). Also tightened the shared H/S fields to private protected to preserve the originals' internal reach (review Issue 2). Strict warnaserror builds 0 warnings (module + tests); full main test project 3733 passed / 0 failed / 1 skipped; ValidationOwnershipArchitectureTests 4/4 (gate:pr_ci.yml#build_test). Independent claude-opus-5 rubber-duck review → GO (mechanically verified via ordered attribute/route/localized-string diffs, per-controller single-permission cardinality, typeof(TModel).Name↔nameof correspondence, a normalized line-for-line diff of the extracted orchestration against the original, no manager-interface member hiding, unambiguous ValidateAsync overload resolution, and zero external references to the changed types; the AgentEntitlements exclusion confirmed correct).
F3 done. Decomposed the two remaining mega-files with no behavior change. (1) EnterpriseInteractionReportProvider (1554 lines): extracted the pure, stateless metric-calculation logic into a new internal static InteractionMetricsCalculator (Reports/Services/) — Aggregate, CalculateQueueServiceLevel, CalculateCombinedQueueServiceLevel, IsInboundOffered, IsAbandoned, GetWaitSeconds, GetWaitUntilEndSeconds, GetTalkSeconds, GetWrapUpSeconds — moved byte-identically (six were private static → public static because using static only imports accessible members; the container is internal, so assembly-visible surface is unchanged), and relocated the two result models InteractionMetrics/QueueServiceLevelMetrics from nested types into top-level internal sealed types under Reports/Models/. The provider keeps all instance-stateful work (RunAsync, every Build* section/document method that depends on S/_agentUserNames/_absentFeatureIds/_capabilityGuard, AgentPerformanceMode, DisplayOrUnknown, ResolveAgentName, CreatePerformanceRow) and gains using static …InteractionMetricsCalculator; so its ~30 bare call sites resolve unchanged; now ~1297 lines. Two test call sites retargeted to InteractionMetricsCalculator.*. (2) AsteriskContactCenterVoiceProvider (2412 lines): this type implements seven segregated voice-capability interfaces but is DI-registered once (Startup.cs:137) and consumers discover capabilities by casting the single instance (provider is IContactCenterVoiceXProvider) across ContactCenter.Core — so it must remain one compiled type. The only behavior-identical decomposition is a partial-class split by capability: internal sealed class → internal sealed partial class, main file retains usings/fields/ctor/props/DialAsync/ConnectToAgentAsync + shared private helpers + the sole class-closing brace (down to ~775 lines), and five new partials carry one capability group each — .Recording.cs (213), .Transfer.cs (320), .Conference.cs (325), .AttendedTransfer.cs (293, incl. nested ResolvedConsultContext), .Monitoring.cs (565). Member multiset preserved exactly (64 → 64, none lost/duplicated/split); interface list untouched so all capability casts are unaffected. The 11-line using block is repeated per partial for consistency (repo does not enforce unused-using removal; CI -warnaserror build is 0 warnings). Asterisk + test-project strict warnaserror builds 0/0; full main test project 3733 passed / 0 failed / 1 skipped (gate:pr_ci.yml#build_test). Independent claude-opus-5 rubber-duck review → GO (re-derived every claim: all 11 report blocks + both models byte-identical to HEAD; provider diff is exactly +using static plus two pure deletions; using static resolves with no shadowing incl. the GetWaitUntilEndSeconds method-group conversions; Asterisk class-body line multiset and 64→64 member set verified, 56/56 contiguous member blocks matched verbatim against HEAD, single class decl + single column-0 brace per file, ResolvedConsultContext moved wholesale, one deliberate cross-partial helper call; reviewer reran CI-exact Release -warnaserror builds 0/0 and ~ContactCenter 1466/0 + ~Asterisk 444/0/1).
T3 done. Documentation review (§7 T3). Deep-dived src/CrestApps.Docs: confirmed the shipped Contact Center pages (base-voice acceptance/production-support.md, recording ship-disabled default, supervision capability wording) match the code as reconciled in G1/E2, the changelog file docs/changelog/v2.0.0.md matches VersionPrefix 2.0.0 and already carries the G1 + DI-cycle-fix bullets. The Group F work (F1/F2/F3) is internal maintainability refactoring with no public-behavior, configuration, or API surface change, so no doc/changelog update was required for it. Docusaurus production build succeeded with no broken-link warnings (gate:pr_ci.yml docs job / validate-docs).
T4 done. Aspire host smoke test (§7 T4). Ran src/Startup/CrestApps.Aspire.AppHost (dotnet run -c Release) against a live Docker daemon: the distributed application booted cleanly ("Distributed application started", dashboard on :17260, zero startup errors), provisioned its infrastructure — custom crestapps/asterisk-webrtc image built and reached healthy, plus Coturn and Redis Up — and the OrchardCoreCMS project started and served HTTP 200 on https://localhost:5001. The Ollama resource stayed in Created because .WithGPUSupport() requires GPU passthrough that Docker Desktop on macOS does not provide; this is an environment limitation and is intentionally non-blocking (the CMS does not WaitFor Ollama). AppHost also compiles as part of the full-solution strict -warnaserror Release build (0 warnings). Containers were torn down cleanly on shutdown.
T2 done. PR-wide implementation deep-dive (§7 T2). Confirmed the mechanical exit criteria: full-solution strict -warnaserror Release build (RunAnalyzers=true) 0 warnings / 0 errors and the full unit suite 3733 passed / 0 failed / 1 skipped. For the review portion, ran an independent claude-opus-5 review (three parallel area reviewers — Core orchestration, ContactCenter module, Asterisk/Telephony — plus the reviewer's own programmatic cross-checks) over the ~1015 changed production .cs files. Verdict GO: nothing blocking, nothing should-fix. Independently verified invariants included all 24 YesSql indexes matching their migration columns, all 24 feature-ids declared in Manifest.cs, all 12 permissions registered, all 15 minimal-API endpoints authorized (the two anonymous ones are signature-verified webhooks), fenced/optimistic-concurrency outbox claims, bounded idempotency keys, post-commit ShellScope.AddDeferredTask dispatch, fail-closed topology admission, and the single shared ExternalDestinationPolicy (strict E.164) enforced at every dial decision point; layer boundary holds (Interaction carries no disposition). Convention scans across all changed files: DateTime.UtcNow=0, async void=0, non-literal S[...]=0. Fixed the two surfaced nits (T2 mandates fixing what falls short): (1) the canonical provider-name column width was inconsistent (128 in InteractionIndex/CallSessionIndex, 100 in ProviderCommandIndex/ProviderWebhookInboxMessageIndex) — promoted a single ContactCenterConstants.ProviderNameLength = 128 and referenced it from all four migrations (safe pre-release CreateAsync alignment; regenerated the ContactCenter.Abstractions public-API baseline for the new constant); (2) AsteriskResolvedSettings.PjsipRealtimeConnectionString carried ciphertext while its sibling secrets were unprotected in the same initializers — now unprotected at both tenant-settings construction sites (AsteriskTelephonyProvider via UnprotectPassword, AsteriskSoftPhoneRegistrationConfigContributor via Unprotect), matching the credential store's own handling; verified the value is never emitted to the browser and resolved is never persisted, so no plaintext-at-rest or leak regression. Both helpers are null/empty-safe. Strict module + test builds 0/0; full suite re-run 3733/0/1 (gate:pr_ci.yml#build_test).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Contact Center — Production Readiness (Consolidated & Revised)

1. Mission & scope

2. Guiding principles (extend Orchard Core; do not reinvent)

3. Retained foundation (already completed — do not redo)

4. Remaining work — checklist tracker

Group A — De-overengineering & Orchard Core alignment (do first)

Group B — Correctness & distribution close-out

Group C — Data lifecycle & compliance

Group D — Operability

Group E — Review & release

Group F — Optional refactors (recommended, not release-blocking)

Group G — Honest capability verification

5. De-scoped as over-engineering or process overhead (removed / not doing)

6. Deferred to Track B (post-release; explicitly not started)

7. Final tasks (run last, in order)

Progress log

Uh oh!

FilesExpand file tree

PRODUCTION-READINESS.md

Latest commit

History

PRODUCTION-READINESS.md

File metadata and controls

Contact Center — Production Readiness (Consolidated & Revised)

1. Mission & scope

2. Guiding principles (extend Orchard Core; do not reinvent)

3. Retained foundation (already completed — do not redo)

4. Remaining work — checklist tracker

Group A — De-overengineering & Orchard Core alignment (do first)

Group B — Correctness & distribution close-out

Group C — Data lifecycle & compliance

Group D — Operability

Group E — Review & release

Group F — Optional refactors (recommended, not release-blocking)

Group G — Honest capability verification

5. De-scoped as over-engineering or process overhead (removed / not doing)

6. Deferred to Track B (post-release; explicitly not started)

7. Final tasks (run last, in order)

Progress log