Fix caller transaction graph handoff

Track the originating PostgreSQL transaction while the worker waits for a newly started graph. Use durable, bounded probes so long commits resume, rollbacks fail cleanly, transient database errors retry, and wait history remains bounded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Author: Haider Zahid

CHANGELOG.md

@@ -12,6 +12,7 @@ Pre-1.0 note: while `pg_durable` is in major version `0`, minor releases may inc
 
 ### Fixed
 
+- **Caller-transaction handoff:** `df.start()` now tracks the originating transaction until it commits or aborts, so legal caller transactions lasting more than five seconds no longer leave a `pending` `df.instances` row paired with a failed engine execution. Graph admission uses durable backoff and bounded-history compaction rather than holding a worker connection while it waits.
 - **Worker connection role names:** catalog role names are now passed verbatim when opening workflow connections, preventing quote-wrapped names from being reinterpreted as a different role.
 - **Restricted HTTP transport (#342):** restricted allow-list builds now require HTTPS so credentials and request bodies cannot be sent over plaintext HTTP; development-only `http-allow-all` builds continue to permit HTTP.
 

USER_GUIDE.md

@@ -169,6 +169,13 @@ runs in. It changes nothing about the durable function that gets started:
 | `'caller'` (default) | Joins the caller's transaction; a `ROLLBACK` discards the durable function. |
 | `'new'` | Runs in its own transaction on a separate session; **survives a rollback of the caller's transaction**. |
 
+The caller transaction may remain open for an arbitrary amount of time after
+`df.start()` returns. The worker follows that transaction's outcome without
+holding an execution connection: it begins the workflow after commit and
+terminates the engine record without executing SQL after rollback. Rolling back
+only the savepoint that contains `df.start()` is also treated as a rollback even
+if the enclosing transaction later commits.
+
 `'new'` provides the same rollback-survival outcome as Oracle autonomous
 transactions and `REQUIRES_NEW` propagation for **asynchronously started work**.
 It is not a synchronous autonomous routine: only the durable launch has

docs/ARCHITECTURE.md

@@ -452,6 +452,19 @@ pub async fn execute(
 }
 ```
 
+`df.start()` commits the duroxide start independently while its `df.instances`
+and `df.nodes` writes remain in the caller's transaction. New orchestration
+inputs therefore carry the originating top-level transaction ID. Graph
+admission uses a single-shot probe: load immediately when the graph is visible,
+otherwise inspect `pg_xact_status()`, then wait with deterministic durable
+timers while the transaction is in progress. An abort terminates the engine
+record without executing SQL; a committed transaction whose graph is still
+absent identifies a savepoint rollback. The wait periodically uses
+`continue_as_new` to bound replay history and never holds a management
+connection between probes. Historical orchestration inputs omit the transaction
+ID and continue scheduling the original load activity with its original input,
+preserving in-flight replay compatibility.
+
 ### Node Execution
 
 Internal node handlers return `NodeResult`, a `Result` whose error arm is a typed
@@ -809,4 +822,3 @@ SELECT df.start(
 2. **Phase 2 (Execution)**: Background worker's duroxide runtime picks up the orchestration. `LoadFunctionGraph` activity loads the graph. Orchestration walks the graph, scheduling activities for each step. Results flow between nodes via `$variable` substitution. Loops use `continue_as_new` for durability.
 
 The key insight is that **graph construction is synchronous** (in user transaction) while **execution is asynchronous and durable** (in background worker via duroxide replay).
-

docs/E2E_TESTING.md

@@ -72,6 +72,7 @@ The test suite is organized into 23 files. Files `01`–`09` open with `SET SESS
 | `15_rls.sql` | RLS on `df.instances` / `df.nodes` / `df.vars` — per-user visibility, cross-user cancel/signal denied, column-level UPDATE, superuser bypass, per-user variable isolation |
 | `16_heartbeat.sql` | Worker heartbeat liveness — `df._worker_epoch.last_seen_at` advances over time |
 | `52_node_id_collision_across_instances.sql` | Cross-instance node-ID collision — two instances own the same 8-hex node id; asserts composite-PK coexistence, that `(instance_id, id)` addresses exactly one row, `df.result()` is instance-scoped, and a scoped `update_node_status`-style UPDATE affects exactly one row (issue #129) |
+| `68_long_caller_transaction.sql` | Caller-transaction handoff beyond five seconds, transient graph-probe failure, whole rollback, and savepoint rollback |
 
 ### Build-Phase Specific
 

docs/upgrade-testing.md

@@ -203,6 +203,15 @@ gate, so they never need to be added to the exclude list.
 Each schema-changing PR should add a section here documenting what changed,
 what the upgrade script handles, and any backward compatibility considerations.
 
+### v0.2.6 → v0.2.7
+
+#### Transaction-aware graph admission
+- **Runtime change (no DDL):** New caller-mode starts include the top-level PostgreSQL transaction ID in the root orchestration input. A versioned single-shot activity probes graph visibility and `pg_xact_status()`; the deterministic orchestration waits with capped backoff and periodically `continue_as_new`s to bound replay history.
+- **Rollback behavior:** A whole-transaction abort fails the df-less engine record without executing SQL. A committed origin transaction with no visible graph is reported distinctly as a likely savepoint rollback. Transient graph/pool errors return a retry state rather than terminally failing the orchestration.
+- **Replay compatibility:** Historical `FunctionInput` payloads deserialize with no origin transaction ID and schedule the original `pg_durable::activity::load-function-graph` activity with the same raw instance-ID input. Existing in-flight history therefore retains its operation name, order, and input bytes. The new activity name and input shape are used only for starts created by the new binary.
+- **Scenario A/B2 considerations:** No extension schema or persisted `df` data changes; no upgrade DDL is required.
+- **Scenario B1 considerations:** The new binary uses PostgreSQL's built-in `pg_current_xact_id()` / `pg_xact_status()` functions and existing `df.instances` / `df.nodes` columns, all available across the supported v0.2.2+ provider line. New starts work against old extension schemas without runtime schema detection.
+
 ### v0.2.5 → v0.2.6
 
 #### Remove `df.ensure_durofut()`