Add a new df.with_http_options helper.

Several of my tasks require passing additional options to `df.http` and
`df.http_multipart`. Unfortunately, adding new parameters to these
functions is Hard. The normal approach (for example taken in #377 with
the `df.loop`) is to keep the old function around, renamed (but with the
same wrapper name). This is fine (and even avoids issues when
`ALTER EXTENSION UPDATE` is not run), but `df.http` and
`df.http_multipart` are functions that will likely have a bunch
of `GRANT`s (and those grants are semantically meaningful to
`pg_durable` beyond our ability to call the functions).

If we try to capture and re-issue[^1] those grants from an extension, PG
will record the grants as coming from the extension update script, and
assume it doesn't need to provide them in pg_dump, so then the
pg_restore won't have them, meaning logical restore and/or PG upgrades
will be broken.

The AI suggested the right approach was some catalog feng shui but that
it would take a while to engineer. I asked on the PostgreSQL discord,
and one of the PG committers (rhass) told me that this (trying to copy
grants from one function to another) was something that you should
never do, and to just have users reissue the grants on update.

So... instead of that, we just add a combinator function that
manipulates the durofut JSON directly to add the options. For
example, you'd use it like:

```sql
df.with_http_options(
    df.http(...),
    '{"options": "here"}'::jsonb
);
```

This admittedly is less ergonomic than adding an `options =>` parameter
for `df.http`, but... well, yeah.

If we want, we could make this into an operator, e.g. allowing
`df.http(...) <some-operator> '{"options": "here"}'` or something like
that? I don't have strong feelings.

[^1]: This is ignoring the fact that the permissions may be different
    now, for example even ignoring the next issue this won't work quite
    right if a delegator of a grant became a superuser.

Author: thomcc-work

USER_GUIDE.md

@@ -701,6 +701,24 @@ df.http(
 ) RETURNS TEXT                    -- JSON response object
 ```
 
+### df.with_http_options() Function
+
+`df.with_http_options(fut TEXT, options JSONB) RETURNS TEXT` is the entry point for
+HTTP modifiers beyond the arguments passed to `df.http` and `df.http_multipart`.
+
+```sql
+df.with_http_options(df.http('https://api.github.com/', 'GET'), '{}'::jsonb)
+    |=> 'response'
+```
+
+In this version, only SQL `NULL` and an empty object (`{}`) are accepted as
+`options`; no option keys are supported yet. Other JSON values, including JSON
+`null`, are rejected. Empty options return the input text byte-for-byte.
+
+The input must be a single `HTTP` or `HTTP_MULTIPART` node, optionally named with
+`|=>`. SQL nodes and compound graphs are rejected, so apply the helper before
+combining nodes. It does not execute a request or change HTTP permissions.
+
 ### Response Format
 
 HTTP calls return a JSON object with full response details:

docs/api-reference.md

@@ -367,6 +367,31 @@ Returns the same envelope as `df.http()`.
 
 ---
 
+### df.with_http_options(fut, options)
+
+HTTP-specific modifier entry point. Returns the JSON-encoded TEXT node for use in
+a workflow, not an HTTP response. Neither existing HTTP function changes signature.
+
+| Parameter | Type | Auto-wrap | Description |
+|-----------|------|-----------|-------------|
+| `fut` | TEXT | ❌ Literal | A single `HTTP` or `HTTP_MULTIPART` node, optionally named with `\|=>` |
+| `options` | JSONB | ❌ Literal | SQL `NULL` or an empty object (`'{}'`) only in this version |
+
+```sql
+df.with_http_options(df.http('https://api.github.com/', 'GET'), '{}'::jsonb)
+  |=> 'response'
+```
+
+No option keys are supported yet. Unknown keys, non-object JSON values (including
+JSON `null`), malformed nodes, SQL nodes, and compound graphs raise an error.
+SQL `NULL` and `{}` return the original node text byte-for-byte, preserving its
+config and result name. Apply the helper to each HTTP node before combining nodes.
+It neither resolves secrets nor grants HTTP access; activity-time permission and
+network checks still apply. Existing installations need `ALTER EXTENSION pg_durable
+UPDATE` to use this new helper, but not to keep using the original HTTP functions.
+
+---
+
 ## Control Functions
 
 ### df.start(fut [, label] [, database] [, transaction_mode])

docs/http-security.md

@@ -117,6 +117,12 @@ superusers always return `true`; regular roles return `true` only when an
 explicit `GRANT EXECUTE ON FUNCTION df.http(text, text, text, jsonb, integer) TO <role>` (or a role that
 inherits one) is in effect.
 
+`df.with_http_options(text,jsonb)` is a node modifier, not a network operation.
+Like other combinators, it uses ordinary `df` schema access and default PUBLIC
+`EXECUTE`. Wrapping a hand-crafted HTTP node does not bypass the activity's
+privilege check. No option keys are supported in this version; SQL `NULL` and
+`{}` preserve the original node text.
+
 ### 3.3 Managing access
 
 HTTP access is **opt-in** and separate from general `df` access.

docs/upgrade-testing.md

@@ -203,7 +203,9 @@ gate, so they never need to be added to the exclude list.
 Each schema-changing PR should add a section here documenting what changed,
 what the upgrade script handles, and any backward compatibility considerations.
 
-### 0.2.8
+### v0.2.7 → v0.2.8
+
+#### Loop failure continuation
 
 - `sql/pg_durable--0.2.7--0.2.8.sql` renames `df.loop(text, text)` to
   `df._loop_legacy(text, text)`, preserving its function OID and dependent
@@ -233,6 +235,12 @@ what the upgrade script handles, and any backward compatibility considerations.
   binary, which continues toward the higher backstop instead. Drain such
   long-running loops before upgrade when continuity is required.
 
+#### Add `df.with_http_options()`
+- **DDL change:** Adds `df.with_http_options(fut text, options jsonb) RETURNS text`. The input must be a single `HTTP` or `HTTP_MULTIPART` node. No option keys are supported yet: SQL `NULL` and `{}` return the original node text byte-for-byte; other values and unsupported keys raise an error.
+- **Upgrade script:** [sql/pg_durable--0.2.7--0.2.8.sql](../sql/pg_durable--0.2.7--0.2.8.sql) adds this helper without replacing the existing HTTP functions. The new helper uses the same schema-access and default PUBLIC `EXECUTE` model as other combinators; it does not grant HTTP access.
+- **Scenario A considerations:** The added function matches pgrx-generated fresh-install SQL, including argument names, null handling and the `with_http_options_wrapper` C symbol.
+- **Scenario B1 considerations:** The new helper remains absent until `ALTER EXTENSION UPDATE`. The new `.so` exports `with_http_options_wrapper`; existing HTTP function signatures, C symbols, OIDs and ACLs are unchanged.
+
 ### v0.2.6 → v0.2.7
 
 #### Transaction-aware graph admission

scripts/test-upgrade.sh

@@ -778,6 +778,17 @@ test_b1_conditional_loop() {
     assert_sql_contains "SELECT df.loop('SELECT 1', 'SELECT false');" '"node_type":"LOOP"'
 }
 
+test_b1_http_construction() {
+    assert_sql_contains "SELECT df.http('https://api.github.com/');" '"node_type":"HTTP"' &&
+    assert_sql_equals \
+        "SELECT (df.http('https://api.github.com/', 'GET', NULL, NULL, 7)::jsonb->>'query')::jsonb->>'timeout_seconds';" \
+        "7"
+}
+
+test_b1_http_options_absent() {
+    assert_sql_equals "SELECT to_regprocedure('df.with_http_options(text,jsonb)') IS NULL;" "t"
+}
+
 test_b1_dsl_chain() {
     assert_sql_contains "SELECT df.sql('SELECT 1') ~> df.sql('SELECT 2');" '"node_type":"THEN"'
 }
@@ -909,6 +920,10 @@ else
         run_test "B1 [v${B1_VERSION}]: df.version()" test_b1_version
         run_test "B1 [v${B1_VERSION}]: df.sql() construction" test_b1_dsl_construction
         run_test "B1 [v${B1_VERSION}]: df.loop(body, condition)" test_b1_conditional_loop
+        run_test "B1 [v${B1_VERSION}]: df.http() construction" test_b1_http_construction
+        if ! version_ge "$B1_VERSION" "0.2.8"; then
+            run_test "B1 [v${B1_VERSION}]: new HTTP options helper remains absent" test_b1_http_options_absent
+        fi
         run_test "B1 [v${B1_VERSION}]: DSL chain (~>)" test_b1_dsl_chain
         run_test "B1 [v${B1_VERSION}]: conditional operators (?>/!>)" test_b1_conditional_operators
         run_test "B1 [v${B1_VERSION}]: df.start()/wait_for_completion()" test_b1_start_and_complete
@@ -1030,13 +1045,63 @@ test_b2_grant_usage_after_upgrade() {
     run_sql_capture "DROP OWNED BY ${probe_role}; DROP ROLE IF EXISTS ${probe_role};" >/dev/null 2>&1 || true
 }
 
+test_b2_http_api_after_upgrade() {
+    create_extension_at_version "$PREV_VERSION"
+
+    local output
+    output=$(run_sql_capture "
+        CREATE ROLE durable_b2_http_probe;
+        GRANT EXECUTE ON FUNCTION df.http(text,text,text,jsonb,integer),
+            df.http_multipart(text,text,jsonb,jsonb,integer)
+            TO durable_b2_http_probe WITH GRANT OPTION;
+
+        CREATE TEMP TABLE http_api_before AS
+            SELECT oid, proacl FROM pg_proc
+            WHERE oid IN (
+                'df.http(text,text,text,jsonb,integer)'::regprocedure,
+                'df.http_multipart(text,text,jsonb,jsonb,integer)'::regprocedure
+            );
+        CREATE TEMP VIEW http_calls_before AS
+            SELECT df.http('https://api.github.com/') AS http_node,
+                df.http_multipart('https://api.github.com/',
+                    parts => '[{\"name\":\"field\",\"data_b64\":\"aGk=\"}]'::jsonb) AS multipart_node;
+
+        ALTER EXTENSION pg_durable UPDATE TO '${CURRENT_VERSION}';
+
+        DO \$verify\$
+        BEGIN
+            IF (SELECT count(*) FROM http_api_before) <> 2 OR EXISTS (
+                SELECT 1 FROM http_api_before AS previous
+                LEFT JOIN pg_proc AS current ON current.oid = previous.oid
+                WHERE current.oid IS NULL OR current.proacl IS DISTINCT FROM previous.proacl
+            ) THEN
+                RAISE EXCEPTION 'HTTP function OIDs or ACLs changed during upgrade';
+            END IF;
+            IF NOT EXISTS (
+                SELECT 1 FROM http_calls_before
+                WHERE http_node::jsonb->>'node_type' = 'HTTP'
+                  AND multipart_node::jsonb->>'node_type' = 'HTTP_MULTIPART'
+                  AND df.with_http_options(http_node, '{}'::jsonb) = http_node
+                  AND df.with_http_options(multipart_node, NULL) = multipart_node
+            ) THEN
+                RAISE EXCEPTION 'Legacy HTTP calls or the additive helper failed after upgrade';
+            END IF;
+        END
+        \$verify\$;
+
+        DROP OWNED BY durable_b2_http_probe;
+        DROP ROLE durable_b2_http_probe;
+    ") || { echo "$output"; return 1; }
+}
+
 if [ "$HAS_COMPAT_PREV" = true ]; then
     run_test "B2: Pre-upgrade data survives ALTER EXTENSION UPDATE" test_b2_data_survives_upgrade
     run_test "B2: Pre-upgrade instance remains queryable" test_b2_pre_upgrade_instance_after_upgrade
     run_test "B2: In-flight work completes after upgrade" test_b2_inflight_work_after_upgrade
     run_test "B2: Loop dependency and unified API survive upgrade" test_b2_loop_dependency_survives_upgrade
     run_test "B2: New data and execution after upgrade" test_b2_new_data_after_upgrade
     run_test "B2: df.grant_usage() works and df.debug_connection() is gone after upgrade" test_b2_grant_usage_after_upgrade
+    run_test "B2: HTTP OIDs, grants and dependent views survive upgrade" test_b2_http_api_after_upgrade
 fi
 
 # ============================================================================

sql/pg_durable--0.2.7--0.2.8.sql

@@ -15,3 +15,11 @@ CREATE FUNCTION df."loop"(
 ) RETURNS TEXT
 LANGUAGE c
 AS 'MODULE_PATHNAME', 'loop_with_policy_wrapper';
+
+-- HTTP options are additive; existing function ABIs, OIDs and ACLs stay unchanged.
+CREATE FUNCTION df."with_http_options"(
+    "fut" TEXT,
+    "options" jsonb
+) RETURNS TEXT
+LANGUAGE c
+AS 'MODULE_PATHNAME', 'with_http_options_wrapper';

src/dsl.rs

@@ -487,6 +487,52 @@ pub fn race(a: &str, b: &str) -> String {
     .to_json()
 }
 
+/// Applies HTTP options to a single HTTP or HTTP_MULTIPART node.
+#[pg_extern(schema = "df")]
+pub fn with_http_options(fut: &str, options: Option<pgrx::JsonB>) -> String {
+    use std::{collections::HashSet, sync::LazyLock};
+    static ALLOWED_KEYS: LazyLock<HashSet<&'static str>> = {
+        LazyLock::new(|| {
+            HashSet::from_iter([
+                // Intentionally empty for now.
+            ])
+        })
+    };
+    let node = Durofut::try_from_json(fut).unwrap_or_else(|_| {
+        pgrx::error!("df.with_http_options(): expected an HTTP or HTTP_MULTIPART node")
+    });
+
+    if !matches!(node.node_type.as_str(), "HTTP" | "HTTP_MULTIPART")
+        || node.left_node.is_some()
+        || node.right_node.is_some()
+        || node.condition_node.is_some()
+        || !node.extra_nodes.is_empty()
+    {
+        pgrx::error!("df.with_http_options(): expected a single HTTP or HTTP_MULTIPART node");
+    }
+
+    let config = node
+        .query
+        .as_deref()
+        .and_then(|query| serde_json::from_str::<serde_json::Value>(query).ok());
+    if !config.as_ref().is_some_and(serde_json::Value::is_object) {
+        pgrx::error!("df.with_http_options(): HTTP node config must be a JSON object");
+    }
+
+    if let Some(options) = options {
+        let Some(map) = options.0.as_object() else {
+            pgrx::error!("df.with_http_options(): options must be a JSON object");
+        };
+        for key in map.keys() {
+            if !ALLOWED_KEYS.contains(key.as_str()) {
+                pgrx::error!("df.with_http_options(): unrecognised option '{key}'.");
+            }
+        }
+    }
+
+    fut.to_string()
+}
+
 /// Creates an HTTP request node.
 /// Makes an HTTP request to the specified URL and returns the response.
 ///