Add a new df.with_http_options helper.

Several of my tasks require passing additional options to
`df.http` and `df.http_multipart`. Unfortunately, adding new
parameters to these functions is not possible without breaking
users who don't run `ALTER EXTENSION UPDATE`, even if those
parameters use `pgrx::default!`. This is a new combinator
function that manipulates the durofut JSON directly to add the
options, for example, you'd use it like:

```sql
df.with_http_options(
    df.http(...),
    '{"options": "here"}'::jsonb
);
```

This admittedly is less ergonomic than adding an `options =>`
parameter for `df.http`, which is what my first pass at this
did (see thomcc-work@79e26003
for my abandoned work along that approch).

If we want, we could make this into an operator, e.g. allowing
`df.http(...) <some-operator> '{"options": "here"}'` or
something like that. I don't have strong feelings.

Author: thomcc-work

USER_GUIDE.md

@@ -696,11 +696,31 @@ df.http(
     url TEXT,                     -- Required: endpoint URL
     method TEXT DEFAULT 'POST',   -- GET, POST, PUT, DELETE, PATCH
     body TEXT DEFAULT NULL,       -- Request body (JSON)
-    headers JSONB DEFAULT '{}',   -- Custom headers
+    headers JSONB DEFAULT NULL,   -- Custom headers
     timeout_seconds INT DEFAULT 30
-) RETURNS TEXT                    -- JSON response object
+) RETURNS TEXT                    -- JSON-encoded workflow node
 ```
 
+### df.with_http_options() Function
+
+`df.with_http_options(fut TEXT, options JSONB) RETURNS TEXT` is the entry point for
+HTTP modifiers without changing either HTTP constructor's signature.
+
+```sql
+df.with_http_options(df.http('https://api.github.com/', 'GET'), '{}'::jsonb)
+    |=> 'response'
+```
+
+In this version, only SQL `NULL` and an empty object (`{}`) are accepted as
+`options`; no option keys are supported yet. Other JSON values, including JSON
+`null`, are rejected. Empty options return the input text byte-for-byte.
+
+The input must be a single `HTTP` or `HTTP_MULTIPART` node, optionally named with
+`|=>`. SQL nodes and compound graphs are rejected, so apply the helper before
+combining nodes. It does not execute a request or change HTTP permissions.
+An existing installation needs `ALTER EXTENSION pg_durable UPDATE` to add this
+helper; the original five-argument HTTP functions remain usable without it.
+
 ### Response Format
 
 HTTP calls return a JSON object with full response details:
@@ -961,10 +981,10 @@ It needs the same `include_http => true` grant as `df.http()`.
 df.http_multipart(
     url TEXT,                     -- Required: endpoint URL
     method TEXT DEFAULT 'POST',
-    parts JSONB DEFAULT '[]',     -- Array of part objects
-    headers JSONB DEFAULT '{}',
+    parts JSONB DEFAULT NULL,     -- Required: non-empty array of part objects
+    headers JSONB DEFAULT NULL,
     timeout_seconds INT DEFAULT 30
-) RETURNS TEXT                    -- Same JSON envelope as df.http()
+) RETURNS TEXT                    -- JSON-encoded workflow node
 ```
 
 Each part is an object with `name` and `data_b64`, optionally `filename` (which makes it a

docs/api-reference.md

@@ -326,6 +326,31 @@ Returns the same envelope as `df.http()`.
 
 ---
 
+### df.with_http_options(fut, options)
+
+HTTP-specific modifier entry point. Returns the JSON-encoded TEXT node for use in
+a workflow, not an HTTP response. Neither existing HTTP function changes signature.
+
+| Parameter | Type | Auto-wrap | Description |
+|-----------|------|-----------|-------------|
+| `fut` | TEXT | ❌ Literal | A single `HTTP` or `HTTP_MULTIPART` node, optionally named with `\|=>` |
+| `options` | JSONB | ❌ Literal | SQL `NULL` or an empty object (`'{}'`) only in this version |
+
+```sql
+df.with_http_options(df.http('https://api.github.com/', 'GET'), '{}'::jsonb)
+  |=> 'response'
+```
+
+No option keys are supported yet. Unknown keys, non-object JSON values (including
+JSON `null`), malformed nodes, SQL nodes, and compound graphs raise an error.
+SQL `NULL` and `{}` return the original node text byte-for-byte, preserving its
+config and result name. Apply the helper to each HTTP node before combining nodes.
+It neither resolves secrets nor grants HTTP access; activity-time permission and
+network checks still apply. Existing installations need `ALTER EXTENSION pg_durable
+UPDATE` to use this new helper, but not to keep using the original HTTP functions.
+
+---
+
 ## Control Functions
 
 ### df.start(fut [, label] [, database] [, transaction_mode])

docs/http-security.md

@@ -107,16 +107,31 @@ the node fails immediately.
 `execute_http` runs the following check before any network activity:
 
 ```sql
-SELECT has_function_privilege($submitted_by::regrole,
-    'df.http(text,text,text,jsonb,integer)'::regprocedure,
-    'EXECUTE')
+SELECT CASE
+           WHEN p.oid IS NULL THEN 'absent'
+           WHEN has_function_privilege($submitted_by::regrole, p.oid, 'EXECUTE') THEN 'allowed'
+           ELSE 'denied'
+       END
+FROM (SELECT to_regprocedure('df.http(text,text,text,jsonb,integer)') AS oid) p
 ```
 
+The original five-argument signature remains the privilege identity on both old
+and new schemas. `to_regprocedure()` lets a missing function fail with an explicit
+error instead of raising during lookup. Only `'allowed'` permits the request;
+`'absent'` and `'denied'` both fail the node. Multipart requests use the same check
+against `df.http_multipart(text,text,jsonb,jsonb,integer)`.
+
 `has_function_privilege` honours PostgreSQL's standard privilege model:
 superusers always return `true`; regular roles return `true` only when an
 explicit `GRANT EXECUTE ON FUNCTION df.http(text, text, text, jsonb, integer) TO <role>` (or a role that
 inherits one) is in effect.
 
+`df.with_http_options(text,jsonb)` is a node modifier, not a network operation.
+Like other combinators, it uses ordinary `df` schema access and default PUBLIC
+`EXECUTE`. Wrapping a hand-crafted HTTP node does not bypass the activity's
+privilege check. No option keys are supported in this version; SQL `NULL` and
+`{}` preserve the original node text.
+
 ### 3.3 Managing access
 
 HTTP access is **opt-in** and separate from general `df` access.

docs/upgrade-testing.md

@@ -203,6 +203,15 @@ gate, so they never need to be added to the exclude list.
 Each schema-changing PR should add a section here documenting what changed,
 what the upgrade script handles, and any backward compatibility considerations.
 
+### v0.2.7 → v0.2.8
+
+#### Add `df.with_http_options()` without changing the HTTP ABI
+- **DDL change:** Adds `df.with_http_options(fut text, options jsonb) RETURNS text`. The input must be a single `HTTP` or `HTTP_MULTIPART` node. No option keys are supported yet: SQL `NULL` and `{}` return the original node text byte-for-byte; other values and unsupported keys raise an error.
+- **Upgrade script:** [sql/pg_durable--0.2.7--0.2.8.sql](../sql/pg_durable--0.2.7--0.2.8.sql) adds only this helper. The existing HTTP functions, their grant/revoke helpers, OIDs, ACLs, and dependent objects are untouched. The new helper uses the same schema-access and default PUBLIC `EXECUTE` model as other combinators; it does not grant HTTP access.
+- **Scenario A considerations:** The added function matches pgrx-generated fresh-install SQL, including argument names, null handling and the `with_http_options_wrapper` C symbol.
+- **Scenario B1 considerations:** Both legacy HTTP wrappers retain their five-argument ABI. Merely changing the activity's `to_regprocedure()` lookup cannot make a six-argument C wrapper callable through a five-argument catalog entry. The B1 tests now invoke `df.http()` on every supported old schema (v0.2.2+) and `df.http_multipart()` where present (v0.2.5+), using both defaulted and explicit arguments without an extension update. The new helper remains absent until `ALTER EXTENSION UPDATE`, while existing HTTP construction continues to work. Missing catalog functions are still denied explicitly by the activity checks.
+- **Scenario B2 considerations:** No node-config or history-format changes. The upgrade tests verify that legacy HTTP OIDs and ACLs (including `WITH GRANT OPTION`) survive, that dependent views still work, and that the newly added helper preserves their node text.
+
 ### v0.2.6 → v0.2.7
 
 #### Transaction-aware graph admission

scripts/test-upgrade.sh

@@ -774,6 +774,26 @@ test_b1_dsl_construction() {
     assert_sql_contains "SELECT df.sql('SELECT 1');" '"node_type":"SQL"'
 }
 
+test_b1_http_construction() {
+    assert_sql_contains "SELECT df.http('https://api.github.com/');" '"node_type":"HTTP"' &&
+    assert_sql_equals \
+        "SELECT (df.http('https://api.github.com/', 'GET', NULL, NULL, 7)::jsonb->>'query')::jsonb->>'timeout_seconds';" \
+        "7"
+}
+
+test_b1_multipart_construction() {
+    assert_sql_contains \
+        "SELECT df.http_multipart('https://api.github.com/', parts => '[{\"name\":\"field\",\"data_b64\":\"aGk=\"}]'::jsonb);" \
+        '"node_type":"HTTP_MULTIPART"' &&
+    assert_sql_equals \
+        "SELECT (df.http_multipart('https://api.github.com/', 'POST', '[{\"name\":\"field\",\"data_b64\":\"aGk=\"}]'::jsonb, NULL, 7)::jsonb->>'query')::jsonb->>'timeout_seconds';" \
+        "7"
+}
+
+test_b1_http_options_absent() {
+    assert_sql_equals "SELECT to_regprocedure('df.with_http_options(text,jsonb)') IS NULL;" "t"
+}
+
 test_b1_dsl_chain() {
     assert_sql_contains "SELECT df.sql('SELECT 1') ~> df.sql('SELECT 2');" '"node_type":"THEN"'
 }
@@ -904,6 +924,13 @@ else
         run_test "B1 [v${B1_VERSION}]: df.getvar()" test_b1_getvar
         run_test "B1 [v${B1_VERSION}]: df.version()" test_b1_version
         run_test "B1 [v${B1_VERSION}]: df.sql() construction" test_b1_dsl_construction
+        run_test "B1 [v${B1_VERSION}]: df.http() legacy ABI and defaults" test_b1_http_construction
+        if version_ge "$B1_VERSION" "0.2.5"; then
+            run_test "B1 [v${B1_VERSION}]: df.http_multipart() legacy ABI and defaults" test_b1_multipart_construction
+        fi
+        if ! version_ge "$B1_VERSION" "0.2.8"; then
+            run_test "B1 [v${B1_VERSION}]: new HTTP options helper remains absent" test_b1_http_options_absent
+        fi
         run_test "B1 [v${B1_VERSION}]: DSL chain (~>)" test_b1_dsl_chain
         run_test "B1 [v${B1_VERSION}]: conditional operators (?>/!>)" test_b1_conditional_operators
         run_test "B1 [v${B1_VERSION}]: df.start()/wait_for_completion()" test_b1_start_and_complete
@@ -1017,12 +1044,62 @@ test_b2_grant_usage_after_upgrade() {
     run_sql_capture "DROP OWNED BY ${probe_role}; DROP ROLE IF EXISTS ${probe_role};" >/dev/null 2>&1 || true
 }
 
+test_b2_http_api_after_upgrade() {
+    create_extension_at_version "$PREV_VERSION"
+
+    local output
+    output=$(run_sql_capture "
+        CREATE ROLE durable_b2_http_probe;
+        GRANT EXECUTE ON FUNCTION df.http(text,text,text,jsonb,integer),
+            df.http_multipart(text,text,jsonb,jsonb,integer)
+            TO durable_b2_http_probe WITH GRANT OPTION;
+
+        CREATE TEMP TABLE http_api_before AS
+            SELECT oid, proacl FROM pg_proc
+            WHERE oid IN (
+                'df.http(text,text,text,jsonb,integer)'::regprocedure,
+                'df.http_multipart(text,text,jsonb,jsonb,integer)'::regprocedure
+            );
+        CREATE TEMP VIEW http_calls_before AS
+            SELECT df.http('https://api.github.com/') AS http_node,
+                df.http_multipart('https://api.github.com/',
+                    parts => '[{\"name\":\"field\",\"data_b64\":\"aGk=\"}]'::jsonb) AS multipart_node;
+
+        ALTER EXTENSION pg_durable UPDATE TO '${CURRENT_VERSION}';
+
+        DO \$verify\$
+        BEGIN
+            IF (SELECT count(*) FROM http_api_before) <> 2 OR EXISTS (
+                SELECT 1 FROM http_api_before AS previous
+                LEFT JOIN pg_proc AS current ON current.oid = previous.oid
+                WHERE current.oid IS NULL OR current.proacl IS DISTINCT FROM previous.proacl
+            ) THEN
+                RAISE EXCEPTION 'HTTP function OIDs or ACLs changed during upgrade';
+            END IF;
+            IF NOT EXISTS (
+                SELECT 1 FROM http_calls_before
+                WHERE http_node::jsonb->>'node_type' = 'HTTP'
+                  AND multipart_node::jsonb->>'node_type' = 'HTTP_MULTIPART'
+                  AND df.with_http_options(http_node, '{}'::jsonb) = http_node
+                  AND df.with_http_options(multipart_node, NULL) = multipart_node
+            ) THEN
+                RAISE EXCEPTION 'Legacy HTTP calls or the additive helper failed after upgrade';
+            END IF;
+        END
+        \$verify\$;
+
+        DROP OWNED BY durable_b2_http_probe;
+        DROP ROLE durable_b2_http_probe;
+    ") || { echo "$output"; return 1; }
+}
+
 if [ "$HAS_COMPAT_PREV" = true ]; then
     run_test "B2: Pre-upgrade data survives ALTER EXTENSION UPDATE" test_b2_data_survives_upgrade
     run_test "B2: Pre-upgrade instance remains queryable" test_b2_pre_upgrade_instance_after_upgrade
     run_test "B2: In-flight work completes after upgrade" test_b2_inflight_work_after_upgrade
     run_test "B2: New data and execution after upgrade" test_b2_new_data_after_upgrade
     run_test "B2: df.grant_usage() works and df.debug_connection() is gone after upgrade" test_b2_grant_usage_after_upgrade
+    run_test "B2: HTTP OIDs, grants and dependent views survive upgrade" test_b2_http_api_after_upgrade
 fi
 
 # ============================================================================

sql/pg_durable--0.2.7--0.2.8.sql

@@ -6,5 +6,10 @@
 -- See docs/upgrade-testing.md for the upgrade-script and backward-compatibility
 -- requirements (Scenario A / B1 / B2).
 --
--- No schema changes yet for 0.2.8. Add DDL below as the 0.2.8 cycle lands
--- extension-schema changes.
+-- HTTP options are additive; existing function ABIs, OIDs and ACLs stay unchanged.
+CREATE FUNCTION df."with_http_options"(
+    "fut" TEXT,
+    "options" jsonb
+) RETURNS TEXT
+LANGUAGE c
+AS 'MODULE_PATHNAME', 'with_http_options_wrapper';

src/activities/execute_http.rs

@@ -28,19 +28,30 @@ pub const NAME: &str = "pg_durable::activity::execute-http";
 /// This closes the bypass path where a user crafts a raw Durofut JSON and
 /// passes it directly to `df.start()`, inserting an HTTP node without going
 /// through the DSL guard in `df.http()`.
+///
+/// A missing catalog function is denied explicitly instead of raising on lookup.
 async fn check_http_privilege(pool: &PgPool, submitted_by: &str) -> Result<(), String> {
-    let has_priv: Option<bool> = sqlx::query_scalar(
-        "SELECT has_function_privilege($1::regrole, \
-             'df.http(text,text,text,jsonb,integer)'::regprocedure, \
-             'EXECUTE')",
+    let verdict: Option<String> = sqlx::query_scalar(
+        "SELECT CASE \
+             WHEN p.oid IS NULL THEN 'absent' \
+             WHEN pg_catalog.has_function_privilege($1::regrole, p.oid, 'EXECUTE') THEN 'allowed' \
+             ELSE 'denied' \
+         END \
+         FROM (SELECT \
+             pg_catalog.to_regprocedure('df.http(text,text,text,jsonb,integer)') \
+             AS oid) p",
     )
     .bind(submitted_by)
     .fetch_optional(pool)
     .await
     .map_err(|e| format!("HTTP privilege check failed for role '{submitted_by}': {e}"))?;
 
-    match has_priv {
-        Some(true) => Ok(()),
+    match verdict.as_deref() {
+        Some("allowed") => Ok(()),
+        Some("absent") => Err(format!(
+            "Blocked: df.http() is not installed in this database, so the HTTP privilege \
+             of role '{submitted_by}' cannot be verified."
+        )),
         _ => Err(format!(
             "Blocked: role '{submitted_by}' does not have EXECUTE privilege on df.http(). \
              Grant EXECUTE ON FUNCTION df.http(text,text,text,jsonb,integer) TO {submitted_by} to allow HTTP requests."

src/activities/execute_multipart.rs

@@ -32,19 +32,29 @@ pub const NAME: &str = "pg_durable::activity::execute-multipart";
 /// Mirrors `execute_http::check_http_privilege` — closes the bypass path where
 /// a user crafts a raw Durofut JSON and passes it directly to `df.start()`,
 /// inserting an HTTP_MULTIPART node without going through the DSL guard.
+/// A missing catalog function is denied explicitly instead of raising on lookup.
 async fn check_multipart_privilege(pool: &PgPool, submitted_by: &str) -> Result<(), String> {
-    let has_priv: Option<bool> = sqlx::query_scalar(
-        "SELECT has_function_privilege($1::regrole, \
-             'df.http_multipart(text,text,jsonb,jsonb,integer)'::regprocedure, \
-             'EXECUTE')",
+    let verdict: Option<String> = sqlx::query_scalar(
+        "SELECT CASE \
+             WHEN p.oid IS NULL THEN 'absent' \
+             WHEN pg_catalog.has_function_privilege($1::regrole, p.oid, 'EXECUTE') THEN 'allowed' \
+             ELSE 'denied' \
+         END \
+         FROM (SELECT \
+             pg_catalog.to_regprocedure('df.http_multipart(text,text,jsonb,jsonb,integer)') \
+             AS oid) p",
     )
     .bind(submitted_by)
     .fetch_optional(pool)
     .await
     .map_err(|e| format!("HTTP privilege check failed for role '{submitted_by}': {e}"))?;
 
-    match has_priv {
-        Some(true) => Ok(()),
+    match verdict.as_deref() {
+        Some("allowed") => Ok(()),
+        Some("absent") => Err(format!(
+            "Blocked: df.http_multipart() is not installed in this database, so the multipart \
+             HTTP privilege of role '{submitted_by}' cannot be verified."
+        )),
         _ => Err(format!(
             "Blocked: role '{submitted_by}' does not have EXECUTE privilege on df.http_multipart(). \
              Grant EXECUTE ON FUNCTION df.http_multipart(text,text,jsonb,jsonb,integer) TO {submitted_by} to allow multipart HTTP requests."

src/dsl.rs

@@ -459,6 +459,46 @@ pub fn race(a: &str, b: &str) -> String {
     .to_json()
 }
 
+/// Applies HTTP options to a single HTTP or HTTP_MULTIPART node.
+#[pg_extern(schema = "df")]
+pub fn with_http_options(fut: &str, options: Option<pgrx::JsonB>) -> String {
+    let node = Durofut::try_from_json(fut).unwrap_or_else(|_| {
+        pgrx::error!("df.with_http_options(): expected an HTTP or HTTP_MULTIPART node")
+    });
+
+    if !matches!(node.node_type.as_str(), "HTTP" | "HTTP_MULTIPART")
+        || node.left_node.is_some()
+        || node.right_node.is_some()
+        || node.condition_node.is_some()
+        || !node.extra_nodes.is_empty()
+    {
+        pgrx::error!("df.with_http_options(): expected a single HTTP or HTTP_MULTIPART node");
+    }
+
+    let config = node
+        .query
+        .as_deref()
+        .and_then(|query| serde_json::from_str::<serde_json::Value>(query).ok());
+    if !config.as_ref().is_some_and(serde_json::Value::is_object) {
+        pgrx::error!("df.with_http_options(): HTTP node config must be a JSON object");
+    }
+
+    if let Some(options) = options {
+        let Some(map) = options.0.as_object() else {
+            pgrx::error!("df.with_http_options(): options must be a JSON object");
+        };
+
+        if let Some(key) = map.keys().next() {
+            pgrx::error!(
+                "df.with_http_options(): unrecognised option '{key}'. \
+                 No options are supported in this version."
+            );
+        }
+    }
+
+    fut.to_string()
+}
+
 /// Creates an HTTP request node.
 /// Makes an HTTP request to the specified URL and returns the response.
 ///