Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Pre-1.0 note: while `pg_durable` is in major version `0`, minor releases may inc
`continue_on_failure` syntax is experimental and may change in future
releases.

- **`pg_durable.http_allowed_domains` (#375):** a restart-only GUC that replaces the HTTP and multipart domain allow-list with exact hostnames and `*.domain` patterns. Existing build-dependent defaults are preserved; an explicit empty list denies all domains in restricted builds. Other HTTP feature gates and safeguards are unchanged.

### Changed

- **Loop lifetime:** raises the loop-iteration backstop from 100,000 to
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Tagged releases publish Debian packages for PostgreSQL 17 and 18 on amd64 from t

Tagged releases also publish a ready-to-run Docker image (`linux/amd64`) for PostgreSQL 17 and 18 to GitHub Container Registry: `ghcr.io/microsoft/pg_durable`. The image installs the released Debian package on top of the official `postgres` image. Each release publishes immutable `X.Y.Z-pg<major>` and `vX.Y.Z-pg<major>` tags (for example `0.2.2-pg17`, `0.2.2-pg18`); the highest stable release additionally updates the floating `pg<major>` tags, and the default major (`pg17`) also updates `latest`. The PG major version is part of every tag so multiple PostgreSQL versions can be published alongside each other. Browse all published images and tags at <https://github.com/microsoft/pg_durable/pkgs/container/pg_durable>.

> **Warning:** The published Docker image is intended for **evaluating and learning pg_durable only — do not use it in production.** It enables superuser durable instances for a frictionless out-of-the-box demo. Its HTTP egress policy is whatever the released Debian package was built with (`http-allow-azure-domains` Azure domains only). Multi-arch (`linux/arm64`) images are not published yet; they will follow once arm64 Debian packages are available.
> **Warning:** The published Docker image is intended for **evaluating and learning pg_durable only — do not use it in production.** It enables superuser durable instances for a frictionless out-of-the-box demo. Its HTTP egress policy uses the released Debian package's `http-allow-azure-domains` tier, defaulting to Azure service subdomains and `api.github.com`. See [HTTP allowed domains](USER_GUIDE.md#http-allowed-domains) for configuration in v0.2.8+. Multi-arch (`linux/arm64`) images are not published yet; they will follow once arm64 Debian packages are available.

Run the published image — PostgreSQL 17 and 18 can run side by side on different host ports:

Expand Down
70 changes: 60 additions & 10 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,11 @@ pg_durable is a PostgreSQL extension that brings durable, fault-tolerant functio
14. [Monitoring](#monitoring)
15. [User Isolation & Privileges](#user-isolation--privileges)
16. [Connection Host](#connection-host)
17. [Connection Limits](#connection-limits)
18. [Troubleshooting](#troubleshooting)
19. [Quick Reference Card](#quick-reference-card)
20. [Appendix: Test Data Setup](#appendix-test-data-setup)
17. [HTTP Allowed Domains](#http-allowed-domains)
18. [Connection Limits](#connection-limits)
19. [Troubleshooting](#troubleshooting)
20. [Quick Reference Card](#quick-reference-card)
21. [Appendix: Test Data Setup](#appendix-test-data-setup)

---

Expand Down Expand Up @@ -2168,12 +2169,24 @@ If the user who submitted a function is dropped **before execution**:

#### HTTP Requests

HTTP requests (`df.http()`) currently execute with the **background worker's privileges**, not the submitting user's privileges:

- All users can make HTTP requests to the same endpoints
- No user-specific URL allowlists

**Security model:** For pg_durable's built-in `df.http()` activity, outbound HTTP is controlled by compile-time Cargo features and is off by default. When enabled, a hardcoded SSRF IP blocklist and domain allow-list are enforced — all `df.http()` requests to private/reserved IP ranges are blocked and only approved Azure service domains are permitted (e.g. `*.blob.core.windows.net`, `*.openai.azure.com`). These `df.http()` restrictions cannot be bypassed by any database user, including superusers. They do not restrict arbitrary SQL functions, user-defined functions, or third-party Postgres extensions that a workflow role can execute from SQL nodes; administrators must manage extension installation, function privileges, and network egress separately. See `docs/http-security.md` for the full security model and feature flag reference.
`df.http()` and `df.http_multipart()` share a server-wide destination policy.
Permission to use each function is checked for the submitting role, but there
are no per-role domain allowlists.

**Security model:** Outbound HTTP availability and its security tier are
controlled by compile-time Cargo features; HTTP is off when no HTTP feature is
enabled. Restricted builds enforce a hardcoded SSRF IP blocklist and the
[HTTP domain allow-list](#http-allowed-domains), which defaults to Azure service
subdomains and `api.github.com`. Administrators can replace the domain list
with `pg_durable.http_allowed_domains` and restart PostgreSQL. This does not
relax the other restrictions or exempt superuser requests. Only the
development-only `http-allow-all` build bypasses domain and IP restrictions.

These protections apply to the built-in HTTP activities, not arbitrary SQL
functions, user-defined functions, or third-party Postgres extensions that a
workflow role can execute from SQL nodes. Administrators must manage extension
installation, function privileges, and network egress separately. See
[HTTP security](docs/http-security.md) for the full security model.

**Future:** Per-user HTTP isolation and URL allowlists are planned.

Expand Down Expand Up @@ -2361,6 +2374,43 @@ This postmaster setting requires a PostgreSQL restart. When it is empty or unset

---

## HTTP Allowed Domains

Since v0.2.8, administrators can replace the destination allow-list for
`df.http()` and `df.http_multipart()` in restricted HTTP builds:

```ini
# postgresql.conf
pg_durable.http_allowed_domains = 'api.github.com, *.blob.core.windows.net'
```

This server-wide **Postmaster-context** setting requires a PostgreSQL restart.
It can also be configured through an authorized `ALTER SYSTEM SET`; a reload
alone does not apply it. All users can inspect the active list with
`SHOW pg_durable.http_allowed_domains`, but sessions and roles cannot override it.

Use comma-separated hostnames. `api.example.com` matches only that host;
`*.example.com` matches subdomains at any depth, but not `example.com` itself.
Whitespace around entries is ignored, matching is case-insensitive, and
internationalized hostnames can use UTF-8 or ASCII/Punycode. Do not include
URLs, ports, IP addresses, or trailing dots.

**The configured list replaces all defaults.** Without an override,
`http-allow-azure-domains` permits the existing Azure service subdomains and
`api.github.com`; `http-allow-test-domains` also permits `httpbingo.org`.
An empty list (`''`) denies every domain in restricted builds. Malformed lists
are rejected as a whole; an invalid startup value prevents PostgreSQL from
starting rather than silently restoring defaults.

This setting does not enable HTTP in a build without HTTP support, and
`http-allow-all` continues to bypass it even when it is empty. The HTTPS
requirement, IP blocklist, proxy and redirect restrictions, and HTTP function
privileges are unchanged. After restart, pending requests and retries use the
new list. See [HTTP security](docs/http-security.md#5-layer-2-endpoint-allow-list)
for the full syntax and default domain list.

---

## Connection Limits

pg_durable uses multiple PostgreSQL connections for different purposes. Four GUCs let you control the connection budget to match your deployment's resources.
Expand Down
44 changes: 44 additions & 0 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,10 @@ df.wait_for_signal('approval', 3600) -- 1 hour timeout

Makes an HTTP request.

In restricted builds, the destination must be permitted by
[`pg_durable.http_allowed_domains`](#pg_durablehttp_allowed_domains).
The same policy applies to `df.http_multipart()`.

| Parameter | Type | Auto-wrap | Description |
|-----------|------|-----------|-------------|
| `url` | TEXT | ❌ Literal | Request URL (supports `$var` substitution) |
Expand Down Expand Up @@ -734,6 +738,46 @@ These settings are configured via `ALTER SYSTEM SET` or `postgresql.conf`. See e

---

### pg_durable.http_allowed_domains

The complete destination allow-list for `df.http()` and `df.http_multipart()`
in restricted builds. Available since v0.2.8.

| Property | Value |
|----------|-------|
| Type | `string` |
| Default | Azure subdomain patterns and `api.github.com` with `http-allow-azure-domains`; also `httpbingo.org` with `http-allow-test-domains`; empty otherwise |
| Context | `POSTMASTER` (requires a PostgreSQL restart, not just a reload) |
| Visibility | All users can read the active value |

```ini
# postgresql.conf
pg_durable.http_allowed_domains = 'api.github.com, *.blob.core.windows.net'
```

Entries are comma-separated exact hostnames or `*.domain` patterns, with
optional surrounding whitespace. A pattern permits subdomains at any depth,
not the apex itself. Matching is case-insensitive and uses IDNA/Punycode
normalization. Use UTF-8 internationalized names or ASCII/Punycode.

An explicit value **replaces all defaults**, including test domains. An empty
or whitespace-only value denies all domains in restricted builds. Malformed
entries reject the whole setting; a malformed startup value prevents server
startup. URLs, ports, IPs, CIDRs, percent escapes, trailing dots, standalone `*`,
and empty entries within a nonempty list are not accepted.

Session, role, and database settings cannot override this policy. An
authorized `ALTER SYSTEM SET` can change the startup configuration, but
PostgreSQL must restart before requests use it.

The GUC does not override Cargo feature gates: HTTP remains disabled in builds
without an HTTP feature, and `http-allow-all` bypasses the list even when it is
empty. Other HTTP safeguards are unchanged. See
[HTTP security](http-security.md#5-layer-2-endpoint-allow-list) for the default
domains and execution-time behavior.

---

### pg_durable.enable_superuser_instances

Controls whether pg_durable allows durable function instances whose `submitted_by` role is a PostgreSQL superuser.
Expand Down
Loading