New Catalog Configuration: Credentials via CREATE USER MAPPING#255
Conversation
789bb11 to
59242ec
Compare
adca49d to
c575a4e
Compare
c575a4e to
34d2a8c
Compare
34d2a8c to
f671ae1
Compare
bec4277 to
171a037
Compare
| /* ProcessUtility handler: protects extension-owned catalog servers */ | ||
| /* ProcessUtility handlers */ | ||
| extern PGDLLEXPORT bool ProtectExtensionCatalogServersHandler(ProcessUtilityParams *processUtilityParams, void *arg); | ||
| extern PGDLLEXPORT bool ScrubIcebergUserMappingHandler(ProcessUtilityParams *processUtilityParams, void *arg); |
There was a problem hiding this comment.
nit: RedactRestCatalogSecretsHandler and need to move to pg_lake_table extension
There was a problem hiding this comment.
RedactRestCatalogUserMappingSecrets
| """Credentials should be resolved from $PGDATA/catalogs.conf when no | ||
| user mapping exists.""" | ||
| catalogs_conf( | ||
| "test_conf_srv.client_id = 'conf-id'\n" |
There was a problem hiding this comment.
we would use conf file only for extension owned catalogs, right? (assuming snowflake ui wont enable it for user catalogs) then might be good also add rest.client_id = '' to the test.
There was a problem hiding this comment.
good point: right now the tests only use user-created server names in catalogs.conf - they're testing a path that won't happen in practice. But the point is to test the path to make sure it works for the "platform owned" catalogs. I assume both extension owned "rest" catalog and platform owned catalog will end up getting credentials from conf file, so will add the test you suggested
| /* catalogs.conf overrides GUCs but not user mapping */ | ||
| char *confClientId = NULL; | ||
| char *confClientSecret = NULL; | ||
| char *confScope = NULL; | ||
|
|
||
| if (ReadCatalogsConfCredentials(serverName, | ||
| &confClientId, &confClientSecret, | ||
| &confScope)) |
There was a problem hiding this comment.
better to move this above usermapping even if we have null checks. (user mapping should be checked the last to override all)
74860c2 to
9b17dc3
Compare
baf1541 to
2c7435f
Compare
ba29db5 to
dfd7536
Compare
dfd7536 to
36b145f
Compare
| } | ||
|
|
||
|
|
||
| #define CATALOGS_CONF_FILENAME "catalogs.conf" |
There was a problem hiding this comment.
it could be nice to move this out of the database directory, such that secrets don't end up in backups. Maybe we need a setting here.
There was a problem hiding this comment.
Makes sense, you mean a GUC_SUPERUSER_ONLY that defaults to $PGDATA/catalogs.conf but can be set to any absolute path
There was a problem hiding this comment.
I am not sure if the default should point to $pgdata. I think it would be better to set it empty by default.
b5192a9 to
3aa4348
Compare
2c7435f to
0b63f97
Compare
| if (def->location < 0) | ||
| continue; | ||
|
|
||
| char *p = (char *) queryString + def->location; |
There was a problem hiding this comment.
minor: would be nice to use currentChar instead of p, much easier to search for
There was a problem hiding this comment.
do these need to be removed?
There was a problem hiding this comment.
Yes, I noted in the PR description that this branch is not yet updated with the latest changes in "Create server" branch.
3aa4348 to
75c3869
Compare
76408f8 to
138facc
Compare
Introduces the iceberg_catalog foreign data wrapper, allowing users to define named REST catalog configurations via CREATE SERVER instead of relying solely on global GUCs. Built-in catalogs (rest, postgres, object_store) continue to work unchanged through backward-compatible short-name resolution. Credentials (client_id, client_secret) are currently stored as server options. USER MAPPING support is in a follow-up PR (#255). How users create REST catalogs ------------------------------ CREATE SERVER my_polaris TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint 'http://polaris:8181', client_id '...', client_secret '...', location_prefix 's3://my-bucket/warehouse'); CREATE TABLE t (a int) USING iceberg WITH (catalog = 'my_polaris'); A server with no options is also valid -- all settings fall back to GUCs. This allows creating a named handle for organizational purposes while relying entirely on the global GUC configuration. Accepted server options ----------------------- The iceberg_catalog_validator accepts the following server-level options (all defined in iceberg_catalog_option_descs[]): rest_endpoint REST catalog base URL (required; non-empty, must have URI scheme) rest_auth_type 'oauth2' (default), 'default' (alias for oauth2), or 'horizon' oauth_endpoint Custom OAuth token endpoint scope OAuth scope (default from GUC: PRINCIPAL_ROLE:ALL) enable_vended_credentials Boolean, default true location_prefix S3/storage prefix, overrides the GUC default catalog_name Catalog name passed to REST API (affects read-only table CREATE TABLE; ignored at runtime for writable tables) client_id OAuth client ID client_secret OAuth client secret Credentials (client_id, client_secret) are currently stored as server options. USER MAPPING support is in a follow-up PR (#255). catalog_name server option -------------------------- The catalog_name option specifies the catalog prefix used in REST API URLs (opts->catalogName). At CREATE TABLE time (read-only tables): if the table does not specify catalog_name, it inherits the server's value (via ResolveRestCatalogOptions) or defaults to get_database_name(MyDatabaseId). The resolved value is baked into the table's ftoptions. At runtime (GetRestCatalogName): - Writable tables always return get_database_name(MyDatabaseId). Server and table catalog_name options are ignored. This prevents a subsequent ALTER SERVER ... ADD/SET catalog_name from silently re-routing an existing writable table. - Read-only tables return the table-level catalog_name from ftoptions. Writable tables cannot be created on a server with catalog_name set -- enforced at CREATE TABLE time. Extension upgrade script ------------------------ The upgrade script creates: 1. iceberg_catalog FDW -- a handler-less FDW with a validator function (iceberg_catalog_validator) that accepts server-level options. 2. GRANT USAGE ON FOREIGN DATA WRAPPER iceberg_catalog TO lake_write -- so non-superusers with lake_write can create catalog servers. 3. Three built-in catalog servers -- pg_lake_postgres_catalog (TYPE 'postgres'), pg_lake_object_store_catalog (TYPE 'object_store'), and pg_lake_rest_catalog (TYPE 'rest'). These are extension-owned structural anchors backed by actual pg_foreign_server rows, but carry no options -- all configuration comes from GUCs. 4. GRANT USAGE ON FOREIGN SERVER ... TO lake_write -- for each built-in server, so non-superusers can create tables against the built-in catalogs. The reserved user-facing catalog names postgres, object_store, and rest are mapped internally to these built-in server names via ResolveCatalogServerName. A pre-flight check in the upgrade script prevents conflicts if a user already has a server with one of the built-in long names. DDL protection (ValidateIcebergCatalogServerDDL) ------------------------------------------------ A single ProcessUtility hook validates all DDL on iceberg_catalog servers, the iceberg_catalog FDW itself, and extension-membership changes that affect any of these objects. The handler is registered in pg_lake_iceberg/src/init.c. Two layers of protection for server DDL: 1. Short reserved names ('postgres', 'object_store', 'rest') -- the user-facing catalog= values. CREATE SERVER and RENAME TO these names are blocked so users can't shadow the built-in catalogs. 2. Built-in long server names (pg_lake_postgres_catalog, etc.) -- the pre-created anchors. Outside of CREATE/ALTER EXTENSION, CREATE/ALTER/RENAME/OWNER on them is blocked entirely so they remain pure structural anchors with all configuration in GUCs. DROP is blocked by PostgreSQL core via extension membership (pg_depend deptype 'e'). Protected operations: CREATE SERVER with reserved short or built-in long name -> blocked CREATE SERVER with TYPE 'postgres' or 'object_store' -> blocked CREATE SERVER without TYPE 'rest' (NULL or non-rest) -> blocked CREATE SERVER with TYPE 'rest' and a non-reserved name -> allowed ALTER SERVER on a built-in server (any change) -> blocked ALTER SERVER RENAME TO on any iceberg_catalog server -> blocked (dependent tables store catalog='<name>' in ftoptions) ALTER SERVER OPTIONS when dependent REST tables exist -> blocked DROP SERVER on a built-in server -> blocked (extension-owned via pg_depend) ALTER EXTENSION ... DROP SERVER/FDW -> blocked (defense-in-depth; requires superuser) ALTER FOREIGN DATA WRAPPER iceberg_catalog -> blocked (defense-in-depth; requires superuser) CREATE FOREIGN TABLE on any iceberg_catalog server -> blocked (FDW has no handler; hints to use CREATE TABLE ... USING iceberg) All other DDL on user-created servers -> allowed All protection checks are skipped during CREATE EXTENSION / ALTER EXTENSION UPDATE. The ALTER EXTENSION DROP and ALTER FOREIGN DATA WRAPPER guards are defense-in-depth against superuser misuse. Single-catalog transaction constraint ------------------------------------- Modifying tables from different REST catalogs in the same transaction is rejected at statement time -- before any Parquet data is written to S3. The first mutation binds the transaction to its REST catalog; any subsequent mutation targeting a different catalog raises ERRCODE_FEATURE_NOT_SUPPORTED immediately. This applies to all mutation entry points: INSERT, UPDATE, DELETE, TRUNCATE, as well as DDL paths (CREATE TABLE / DROP TABLE) which reach the same protection indirectly. Backward compatibility ---------------------- - CREATE TABLE ... WITH (catalog = 'rest') continues to work. The value 'rest' maps internally to the built-in pg_lake_rest_catalog server, whose configuration comes entirely from GUCs. - All existing GUCs remain functional and serve as fallback defaults for both built-in and user-created catalogs. - No changes to postgres or object_store catalog behavior. - A pre-existing CREATE SERVER postgres FOREIGN DATA WRAPPER postgres_fdw does not conflict with pg_lake's built-in postgres catalog, since the user-facing short name 'postgres' maps internally to pg_lake_postgres_catalog. ====================================================================== Implementation internals ====================================================================== Connection resolution --------------------- ResolveRestCatalogOptions(catalog) is the single entry point for building a RestCatalogOptions. GetRestCatalogOptionsForRelation(relationId) reads the table's catalog option and delegates to it. All catalogs -- built-in and user-created -- go through the same resolution path: 1. ResolveRestCatalogOptions(catalog) calls ResolveCatalogServerName(catalog) to map short reserved names ('rest' -> pg_lake_rest_catalog, etc.); user-created server names pass through unchanged. Then delegates to BuildRestCatalogOptionsFromServer(serverName, catalog). 2. BuildRestCatalogOptionsFromServer(serverName, userVisibleCatalog) looks up the server in pg_foreign_server, initializes all fields from GUC defaults via ApplyGUCDefaults, then overrides with any options explicitly set on the server via ApplyServerOptionOverrides. The userVisibleCatalog is stored in opts->catalog so error messages, the token cache key, and the cross-catalog DML check stay in user-facing terms. 3. ValidateRestCatalogOptions errors if rest_endpoint is still unset after both sources. Since ALTER SERVER OPTIONS is blocked on the built-in servers, their option set is always empty and the GUC defaults survive untouched -- achieving the "GUCs-only built-in REST" behavior through a single code path. Option validation ----------------- Server options are defined once in iceberg_catalog_option_descs[] -- the single source of truth for the whitelist (FindCatalogOptionDesc), the user-facing hint (GetValidCatalogOptionsHint), and the option-to-struct applier (ApplyCatalogOptionValue). Adding or removing an option requires changing only this table. FDW option names and auth type values are compared case-insensitively (pg_strcasecmp). Server names remain case-sensitive (PostgreSQL's GetForeignServerByName is case-sensitive), but the reserved built-in names are checked case-insensitively. Per-catalog token cache ----------------------- The old code used a single global token (RestCatalogAccessToken). With multiple catalogs potentially using different credentials, the token cache is now a hash table (RestCatalogTokenCache) keyed by the iceberg_catalog server OID (opts->serverOid). Each catalog gets its own cached OAuth token with independent expiry tracking. The hash table and token strings are allocated in a dedicated RestTokenCacheCtx memory context (under CacheMemoryContext). A syscache invalidation callback (InvalidateRestTokenCache) registered via CacheRegisterSyscacheCallback(FOREIGNSERVEROID, ...) resets the entire token cache on any ALTER SERVER or DROP SERVER. Single-catalog transaction constraint (internals) ------------------------------------------------- BindRelationToXactRestCatalog(relationId) is called from every entry point that can mutate REST-backed iceberg tables: postgresBeginForeignModify(), AddQueryResultToTable(), and postgresExecForeignTruncate(). On the first REST-backed write, it pre-resolves the relation's full RestCatalogOptions and deep-copies them into TopTransactionContext. Subsequent writes must target the same catalog (compared by server OID so case variations like 'rest' and 'REST' correctly collapse to the same built-in server); otherwise ERRCODE_FEATURE_NOT_SUPPORTED is raised immediately. DDL paths (CREATE TABLE / DROP TABLE) reach the same protection via RecordRestCatalogRequestInTx(), which is invoked synchronously at statement time from the utility hook. At XACT_EVENT_COMMIT time, PostAllRestCatalogRequests uses the pre-resolved PgLakeXactRestCatalog->catalogOpts to build URLs and authentication headers, avoiding syscache lookups during commit. Catalog type helpers (pg_lake_engine/src/utils/catalog_type.c) -------------------------------------------------------------- IsRestCatalog(catalog) -- returns true for the literal 'rest' (case-insensitive) or any iceberg_catalog server whose TYPE is 'rest'. IsCatalogOwnedByExtension(catalog) -- returns true for the short reserved names 'rest', 'object_store', or 'postgres'. IsBuiltinCatalogServerName(name) -- returns true for the long built-in server names (pg_lake_rest_catalog, etc.). ResolveCatalogServerName(catalog) -- maps user-facing short names to the corresponding built-in server name; passes user-created names through. Fixes part of #230 --------- Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
1465b4a to
7764166
Compare
7764166 to
94e9630
Compare
| * that case. | ||
| */ | ||
| static void | ||
| RedactUserMappingSecrets(const char *queryString, List *options) |
There was a problem hiding this comment.
I think we should still redact, that part is fine. My problem is only that the current redaction code is too complex and a bit risky. It tries to parse the SQL string literal by hand, find the start and end of the value, handle '', E'', U&'', escapes and so on. That is a lot of fragile string math and we already know it does not cover every case (for example $$ dollar quoting, or a string split over two lines). When the failure mode is "a secret leaks in clear text", best effort parsing is not really what we want.
So instead of parsing the value, let's just overwrite the whole statement. We do not need to find the value at all, we only need to know it is a CREATE/ALTER USER MAPPING on an iceberg_catalog server, and then blank the whole statement text in place. It cannot leak because we never copy any value byte anywhere, we just stamp a marker and pad the rest. It is also bounded to this one statement (stmt_location / stmt_len) so it does not touch other statements in a multi statement string.
The whole thing becomes basically this:
static void
RedactWholeStatement(char *queryString, const PlannedStmt *pstmt)
{
static const char marker[] = "<redacted: USER MAPPING with credentials>";
int start = pstmt->stmt_location > 0 ? pstmt->stmt_location : 0;
int len = pstmt->stmt_len > 0
? pstmt->stmt_len
: (int) strlen(queryString) - start;
int mlen = sizeof(marker) - 1;
if (len <= 0)
return;
if (mlen > len)
mlen = len; /* truncate marker to fit short statements */
memcpy(queryString + start, marker, mlen);
memset(queryString + start + mlen, ' ', len - mlen);
}And the handler just checks the statement type and the server, then calls it:
bool
RedactRestCatalogUserMappingSecrets(ProcessUtilityParams * processUtilityParams,
void *arg)
{
Node *parsetree = processUtilityParams->plannedStmt->utilityStmt;
const char *serverName;
if (IsA(parsetree, CreateUserMappingStmt))
serverName = ((CreateUserMappingStmt *) parsetree)->servername;
else if (IsA(parsetree, AlterUserMappingStmt))
serverName = ((AlterUserMappingStmt *) parsetree)->servername;
else
return false;
if (IsIcebergCatalogServerByName(serverName))
RedactWholeStatement(processUtilityParams->queryString,
processUtilityParams->plannedStmt);
return false;
}This lets us delete RedactUserMappingSecrets and IsRedactableUserMappingSecret and all the quote handling, around 120 lines gone. The one small trade off is that a scope only ALTER USER MAPPING also gets blanked, but scope is not secret and these servers are credential servers anyway, so I think that is fine. If we really want to keep the non secret ALTER visible we can add a tiny check that only redacts when client_id or client_secret is present, but I would keep it simple.
There was a problem hiding this comment.
Really good point, current code also produces definite leak when there are SQL comments between option name and value, among other things.
| static void | ||
| ValidateRestCatalogOptions(const RestCatalogOptions * opts, const char *catalog) | ||
| { | ||
| bool missingSecret; |
There was a problem hiding this comment.
nit: declare where they are used
94e9630 to
0e90338
Compare
Bug: server owner can redirect another user's REST catalog credentialsRiskA foreign server's owner can change the server's The 3.4 upgrade grants Existing guards do not cover this:
ReproductionSetup (as superuser): As As As As Adam now holds Sarah's principal credentials and can speak to the real Polaris on Sarah's behalf. Scope notes
|
Footgun: removing a REST catalog's credentials wedges
|
Well, I found it while testing locally, but asked Claude to write the comment :) |
Reproduced this locally on the current branch — For the fix, it slots right into the existing endpoint gate in if (pg_strcasecmp(def->defname, "rest_endpoint") == 0 &&
ServerHasDependentRestIcebergTable(server->serverid))
ereport(ERROR, ... "has dependent iceberg tables" ...);Two gaps to close there:
Conservative first cut: block the endpoint change if any USER MAPPING exists on the server (don't even bother filtering by owner) — simplest and safe, mirrors how the table gate is coarse. We can refine to "mappings not owned by me" later if it's too strict. |
0e90338 to
730ff8f
Compare
sfc-gh-okalaci
left a comment
There was a problem hiding this comment.
we are getting pretty close
| if (access != OAT_DROP) | ||
| return; | ||
|
|
||
| if (classId == UserMappingRelationId) |
There was a problem hiding this comment.
Small thing about placement: the timing here is fine (OAT_DROP runs before
the row is deleted, so it can abort both DROP USER MAPPING and the cascade
from DROP SERVER), but i think this lives in the wrong layer. This is a pure
iceberg-catalog concern, but it sits in pg_lake_table's table-drop hook and
needs the IsExtensionCreated(PgLakeIceberg) gate + a pg_user_mapping include
to do it. The guard also only fires if pg_lake_table happens to be installed,
which feels wrong since user mappings are an iceberg thing.
Could we instead register a small chained object_access_hook in pg_lake_iceberg
itself and handle it there, next to ValidateIcebergCatalogServerDDL? We already
do this exact pattern in pg_extension_base (extension_ids.c,
base_worker_launcher.c), so there is precedent. Then drop_table.c stays only
about tables. Not blocking, but cleaner.
| * removal during DROP SERVER ... CASCADE. | ||
| */ | ||
| void | ||
| EnsureUserMappingDropAllowed(Oid umOid) |
There was a problem hiding this comment.
The file is ~2500 lines now and this PR adds quite a lot on top (redaction,
the new server/UM ddl guards, the credential resolution rework). It is already
mixing many things: option descriptors, ddl guards, credential resolution,
token cache/auth, http transport, and the actual rest operations.
Maybe we can split a bit so the patch doesnt grow the monolith more? Even just
pulling the ddl guards + redaction into a rest_catalog_ddl.c would help, since
that part is almost all new/touched here and is self contained. The Makefile
already globs src//.c so new files get picked up automatically, only cost is
making a few statics non-static behind a small internal header. Full split
(options / auth / http / ddl / ops) can be a follow up if you prefer to keep
this PR focused.
There was a problem hiding this comment.
Really good point. We first merged the split PR and rebased this one on top of the split: #406
| opts->catalog = pstrdup(userVisibleCatalog); | ||
| ApplyGUCDefaults(opts); | ||
|
|
||
| if (!isBuiltin) |
There was a problem hiding this comment.
it looks a bit weird that we don't do this inside ApplyGUCDefaults.
We first set then remove, instead, setting inside ApplyGUCDefaults should be much safer and robust to refactors. We can pass isBuiltin to ApplyGUCDefaults so that the intent is well documented as well
| static char * | ||
| GetRestCatalogAccessToken(RestCatalogOptions * opts, bool forceRefreshToken) | ||
| { | ||
| RestCatalogTokenCacheKey key; |
There was a problem hiding this comment.
nit: in general, please pass over the code, declarations should be closer to where they are used, not at the top
| if (serverName == NULL || options == NIL) | ||
| return false; | ||
|
|
||
| if (!IsIcebergCatalogServerByName(serverName)) |
There was a problem hiding this comment.
if there is a typo in the server name, the credentials are still leaked. That's perhaps OK.
Without complicating the code, can you see any ways to avoid that?
There was a problem hiding this comment.
Hmmm, valid point. Let me see what we can do here
There was a problem hiding this comment.
We can redact unless proved the server is non-iceberg. I.e. a typo / non-existent server falls into the "could be iceberg" bucket and gets scrubbed.
| CacheRegisterSyscacheCallback(FOREIGNSERVEROID, | ||
| InvalidateRestTokenCache, | ||
| (Datum) 0); | ||
| CacheRegisterSyscacheCallback(USERMAPPINGOID, |
There was a problem hiding this comment.
Hm, should we add assignHook to the GUCs so that we invalidate the tokens when they are changed? Like RestCatalogStringCredAssign hook on the GUCs.
But remember, assign hooks should never throw errors, that could prevent server restarts etc.
I think we should not act if the GUC value have not changed, definitely make sure that as well. I suggest you debug that part, not let Claude do it.
There was a problem hiding this comment.
Fixing this in a follow-up PR since it's not directly related to adding USER MAPPING, but rather a pre-existing gap on main branch.
| if (classId == UserMappingRelationId) | ||
| { | ||
| if (IsExtensionCreated(PgLakeIceberg)) | ||
| EnsureUserMappingDropAllowed(objectId); |
There was a problem hiding this comment.
On the user-mapping drop guard and CASCADE
I think there is a problem with the new guard for the cascade case. In a
DROP SERVER ... CASCADE (and worse, DROP EXTENSION pg_lake_iceberg CASCADE)
the user mapping and the dependent tables are both direct dependents of the
server, and Postgres does not give any strict ordering between them when it
deletes the cascade members. So we cannot rely on the tables being dropped
before the credentials, or the other way around.
That makes the outcome non-deterministic for the same command:
- if the user mapping is deleted first, the table drop tries to resolve
credentials inEnsureXactBoundToRestCatalog(it calls
GetRestCatalogOptionsForRelationevery time) and fails with "no
credentials", OR the new guard atrest_catalog.c:537aborts the cascade
with "cannot drop the credentials". - if the tables are deleted first, the same cascade just works.
So depending only on internal delete order, DROP SERVER CASCADE either
succeeds or errors, and DROP EXTENSION CASCADE can become impossible without
manual cleanup first. That feels bad - we are blocking people from dropping
their own objects.
Idea for a fix
We already deep-copy the credentials into the transaction
(CopyRestCatalogOptions into TopTransactionContext), and the post-commit
DELETE authenticates from that copy, not from the user mapping. So the real
problem is only capturing the credentials before the user mapping row is
gone. Two small things would make order irrelevant:
- Capture once: in
EnsureXactBoundToRestCatalog, after we are bound to a
server, do the same-server check from the relation's server OID instead of
re-resolving credentials on every call. In effect the whole transaction
then uses one credential snapshot from the first write until commit, which I
think is actually better anyway: it is consistent (a mapping change mid-tx
does not switch credentials half way) and it is faster (we skip the repeated
server + user mapping syscache lookups and option building on every
statement). Right now we already reuse the capturedcatalogOptsfor the
actual requests, we are just re-resolving needlessly for the check. - Capture on the user-mapping drop instead of blocking: we already do this on
the table side - on the first write/drop of a table we resolve the
credentials and deep-copy them into the transaction
(CopyRestCatalogOptionsintoTopTransactionContext), and the post-commit
DELETE authenticates from that copy, not from the live mapping. So instead of
ereport, in the user-mappingOAT_DROPpath we do the same thing here: if
the server has dependent writable REST tables, bind the transaction to the
catalog (i.e. take that same transaction-local copy of the credentials, while
the mapping row still exists) and let the drop proceed. It is just a
transaction-local snapshot, nothing is persisted and future transactions are
not affected.
Then whoever is deleted first, the creds are already captured and the cascade
completes. The only case we cannot really solve is cross-transaction (drop the
credentials in one txn, drop the table in a later one) - but that is
recoverable by recreating the mapping, so a clear error at DROP TABLE time is
probably enough there.
I did not add a hard dependency on purpose: a table -> user mapping dependency
is the wrong model, because credentials are per-user (or PUBLIC) and dynamic,
so it would wrongly cascade-drop a shared table when one user drops their own
mapping.
Until now an iceberg_catalog server carried client_id / client_secret
on the SERVER row, so every role on the cluster shared the same
credentials and operators had to recreate the server on rotation.
Move credentials to per-user state and close the security holes that
move uncovered.
Credential model
----------------
The new resolution order is, lowest to highest priority:
1. pg_lake_iceberg.rest_catalog_* GUC defaults
2. iceberg_catalog SERVER options (non-secret only)
3. pg_user_mapping options (user-created servers only)
Servers split into two disjoint kinds:
- The built-in pg_lake_rest_catalog (catalog='rest') stops after
step 2 on purpose: its credentials live exclusively in the GUCs so
the built-in stays a single, global, instance-wide configuration
with no hidden per-user view. CREATE/ALTER USER MAPPING is
rejected for all three built-in long names.
- User-created REST servers stop GUC propagation between step 1 and
step 2: BuildRestCatalogOptionsFromServer clears
opts->clientId/clientSecret for any non-built-in server before
applying overrides, so USER MAPPING is the only writer of
credentials on user-created servers. Without this, a non-superuser
holding USAGE on the iceberg_catalog FDW (lake_write in 3.4) could
CREATE SERVER pointing at an attacker URL and have
ApplyGUCDefaults ship production credentials to it on the first
CREATE TABLE.
Implementation
--------------
* iceberg_catalog_option_descs[] gains a CATALOG_OPT_CTX_* bitmask
per option. The same table now drives the validator, the
per-context "Valid options are: ..." hint, and the option->struct
applier. client_id / client_secret are USER MAPPING-only; scope
is accepted on both, with the USER MAPPING value winning because
it is applied last during resolution.
* RestCatalogOptions gains a userMappingOid field; the token cache
key is now (serverOid, userMappingOid) so different SET ROLEs in
the same backend each get their own user mapping's credentials.
A USERMAPPINGOID syscache callback invalidates cached tokens on
CREATE/ALTER/DROP USER MAPPING.
* ValidateRestCatalogOptions performs an early auth-type-aware
credentials check at resolution time: client_secret is always
required, client_id is required unless rest_auth_type is
'horizon'. Missing credentials surface as "no credentials found
for REST catalog ..." with ERRCODE_FDW_OPTION_NAME_NOT_FOUND.
The error hint differs by server kind: built-in 'rest' points at
the GUCs, user-created servers point at USER MAPPING and
explicitly disavow GUC fallback.
Statement-text redaction
------------------------
A new ProcessUtility handler, RedactRestCatalogUserMappingSecrets,
scrubs CREATE/ALTER USER MAPPING for any iceberg_catalog server before
the statement reaches pg_stat_statements, log_min_duration_statement,
or any ereport context. The handler is registered after the DDL
validator so it runs first (the handler list is prepend-LIFO),
ensuring failing DDL never leaks secrets.
Redaction is whole-statement: the slice bounded by
PlannedStmt->stmt_location / stmt_len is overwritten with a fixed
marker and padded with spaces. An earlier draft tried to lex SQL
string literals and overwrite only the credential body, but that has
real, exploitable gaps -- a SQL comment between the option name and
its value, or a string continuation across whitespace, would stop the
parser mid-value and leak the suffix. Coarse erasure has no parser
to outsmart. A gating helper, HasRedactableUserMappingSecret, keeps
non-secret ALTER USER MAPPING (e.g. SET scope) fully visible: only
statements that actually move credential bytes through queryString
are scrubbed. DDL itself reads option values from DefElem->arg, so
pg_user_mapping still stores the plaintext credential -- only the
query string surfaces see the redacted form.
DDL guards on user-created servers
----------------------------------
Two guards, gated by uniform predicates
(ServerHasForeignUserMappings / ServerHasDependentRestIcebergTable):
1. Endpoint redirection. ALTER SERVER OPTIONS rest_endpoint and
oauth_endpoint are rejected while the server has any user mapping
or dependent iceberg table. Flipping the endpoint under existing
mappings would redirect every mapping owner's credentials to the
new URL on the next OAuth grant.
2. Credential removal wedge. Dropping the credentials behind a
server that still has dependent iceberg tables would leave the
tables undroppable: writable REST iceberg DROP records a remote
REST DELETE in the current transaction and resolves credentials
at drop time, so the subsequent DROP TABLE fails with "no
credentials found". Three paths are blocked:
- DROP USER MAPPING, and cascade-driven UM removal during
DROP SERVER ... CASCADE, via an OAT_DROP hook on
UserMappingRelationId (deleteOneObject invokes it for both
direct drops and cascade deletions).
- ALTER USER MAPPING OPTIONS DROP client_id and DROP
client_secret, via the ProcessUtility handler. SET and ADD
are rotation, not removal, and remain allowed.
Tests
-----
* test_iceberg_catalog_server.py covers the new resolution and DDL
surface that does not need a live REST catalog: per-context
option lists and hints, valid/invalid options on each side, FOR
CURRENT_USER with all three options, per-role mappings on the
same server, built-in long-name blocks for CREATE/ALTER USER
MAPPING, the GUC-non-fallback property for user-created servers
(sentinel-leak test), the USAGE-on-server permission check at
CREATE TABLE, the endpoint-redirection guard (parametrized
rest_endpoint / oauth_endpoint blocks, two-role redirect repro,
happy paths confirming the guard does not over-block), and
redaction (whole-statement marker, scope-only skip, ordering vs
the DDL validator, plaintext storage preservation, escape
handling, non-iceberg server passthrough).
* test_modify_iceberg_rest_table.py exercises the runtime path
against a live Polaris. The server-option-overrides-GUC
parametrization drops client_id / client_secret (they no longer
belong on SERVER); test_user_mapping_credential_overrides_guc
covers the same resolution-order direction with USER MAPPING in
step 3; the credential-removal DDL guards are exercised against
real iceberg tables (DROP USER MAPPING repro, DROP SERVER CASCADE
wedge-prevention, ALTER USER MAPPING DROP for both credential
options, plus happy paths for SET rotation and for DROP USER
MAPPING without dependents); and
test_alter_user_mapping_credentials_invalidates_token_cache
covers the USERMAPPINGOID syscache callback.
* test_writable_iceberg_common.py: the shared writable-rest-server
fixture now puts credentials on a PUBLIC user mapping and drops
the server with CASCADE to sweep up the mapping.
Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
730ff8f to
f301489
Compare
…action Follow-up to f301489 ("pg_lake_iceberg: move REST catalog credentials to USER MAPPING") with four review-driven changes: * Move the OAT_DROP guard for DROP USER MAPPING / DROP SERVER ... CASCADE out of pg_lake_table and into pg_lake_iceberg's own chained object_access_hook (InitializeIcebergCatalogObjectAccessHook), matching the pg_extension_base precedent. * Make redaction the default for unknown server names. Renamed and inverted IsIcebergCatalogServerByName to IsKnownNonIcebergCatalogServer: skip redaction only when we can prove the target belongs to a different FDW. A fat-fingered CREATE USER MAPPING no longer leaks the secret via core's failing lookup. * Push the "credentials only feed the built-in catalog" rule into ApplyGUCDefaults via an isBuiltin flag, replacing the fragile set-then-null pattern in BuildRestCatalogOptionsFromServer. * Move local declarations next to their first use in helpers added by f301489 (no behavioural change). Adds test_redact_runs_for_unknown_server_name; existing coverage unaffected. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
…ng drop
The OAT_DROP guard erred on DROP USER MAPPING when the
server had dependent iceberg tables, which made
DROP SERVER ... CASCADE and DROP EXTENSION pg_lake_iceberg CASCADE
non-deterministic: cascade siblings are visited in
descending-OID order, so depending on which object Postgres saw first
the same command either succeeded or aborted with "no credentials
found" / "cannot drop the credentials".
* Replace the ereport in the OAT_DROP user-mapping path with a
transaction-local credential capture. pg_lake_iceberg dispatches
via PgLake_RestCatalogXactCaptureCallback (set by pg_lake_table at
_PG_init time); the callback resolves the about-to-vanish mapping's
RestCatalogOptions via the new BuildRestCatalogOptionsFromUserMapping
and deep-copies it into TopTransactionContext. Whichever cascade
sibling fires first wins, so the cascade always succeeds.
* After the first bind, EnsureXactBoundToRestCatalog re-resolves only
the iceberg_catalog server OID -- no pg_user_mapping lookup, no
credential validation -- via the new ResolveRestCatalogServerId /
GetRestCatalogServerIdForRelation helpers. Same-server identity
check stays correct after the UM has been dropped earlier in the
same txn.
* The hook-pointer pattern keeps layering clean: pg_lake_iceberg owns
the predicate (ServerHasDependentRestIcebergTable + the
iceberg_catalog FDW filter) and the dispatch site; pg_lake_table
owns the txn-local state and the capture function. The callback
stays NULL when pg_lake_table is not loaded and the dispatch site
no-ops.
Cross-transaction (DROP USER MAPPING in one txn; DROP TABLE in
another) still fails clearly with the standard "no credentials found"
error -- captures are txn-local by design. Recovery is to recreate
the mapping.
Tests in test_modify_iceberg_rest_table.py rewritten to cover the
four scenarios that matter:
test_drop_user_mapping_in_isolation_succeeds
test_drop_user_mapping_then_drop_table_in_same_txn_succeeds
test_drop_user_mapping_then_drop_table_in_separate_txn_fails_clearly
test_drop_server_cascade_does_not_wedge[um_before_table |
table_before_um]
The cascade test is parametrized over both creation orderings, using
a DROP+RECREATE on the user mapping to flip OID order so cascade
visits UM-first vs table-first.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
sfc-gh-npuka
left a comment
There was a problem hiding this comment.
still left to address from review: #255 (comment)
A server owner can run catalog operations under another user's credentials in the same transactionThis one is a security follow-up on the capture-on-drop change (again a consequence of what we added — see below). A non-superuser who owns a shared ReproSetup: -- as adam (server owner)
BEGIN;
DROP USER MAPPING FOR sarah SERVER shared_srv; -- the server owner is allowed to do this
CREATE TABLE adam_t (a int) USING iceberg WITH (catalog = 'shared_srv'); -- or INSERT into his own table
COMMIT;
-- the post-commit REST create/commit runs with SARAH's client_id/secret, not adam'sadam's own table operations now go to Polaris authenticated as sarah. If adam has no credentials of his own, he even gains write access he didn't have. It's a confused-deputy: the trust model says each user's credentials are their own, but here the owner acts as another user against the remote catalog. Scope: needs a shared, user-created server with per-user mappings and a non-superuser server owner as the attacker. Where the server owner is the admin anyway, it's much less interesting; it matters for multi-tenant shared servers. WhyTwo pieces, both from the capture-on-drop design:
if (PgLakeXactRestCatalog->catalogOpts == NULL)
{
// first bind: resolve + copy
return;
}
Oid relationServerOid = GetRestCatalogServerIdForRelation(relationId);
if (PgLakeXactRestCatalog->catalogOpts->serverOid != relationServerOid)
// only a same-server check; credentials are reused as-isSo a UM-drop capture binds the victim's credentials, and the attacker's own DML in the same transaction silently runs on them. Is this from the earlier suggestion?Yes, fairly directly. We proposed (a) capturing the about-to-vanish mapping's credentials on drop, and (b) "use one credential snapshot for the whole transaction, only re-check the server OID afterwards" as a consistency/perf win. (b) is exactly what lets the captured credentials drive unrelated DML. At the time we framed "a mapping change mid-tx does not switch credentials" as a feature; it turns out to be the hole. FixDon't capture another role's credentials in the first place — only ever snapshot the current user's own mapping or a PUBLIC one. Form_pg_user_mapping umForm = (Form_pg_user_mapping) GETSTRUCT(tup);
Oid serverOid = umForm->umserver;
Oid umUser = umForm->umuser; /* InvalidOid == PUBLIC */
ReleaseSysCache(tup);
...
/* Only snapshot our own mapping or a PUBLIC one; never another role's. */
if (OidIsValid(umUser) && umUser != GetUserId())
return; /* skip capture -> cannot be reused to impersonate */With this, dropping a foreign user's mapping no longer binds their credentials, so any later (The bigger alternative, if we ever want to clean up a foreign user's tables with their exact creds while still blocking impersonation, is to tag capture-sourced |
Confusing
|
sfc-gh-okalaci
left a comment
There was a problem hiding this comment.
Approving this — really nice work, and thank you for the patience through all the rounds here. The whole credential flow via USER MAPPING reads much cleaner now, and honestly the UX for REST catalogs is shaping up very nicely.
My approval is conditional on the two follow-ups I posted just above:
- the credential-capture impersonation (the small
umusergate) — this is the one I'd really like handled before/with merge, since it's a real cross-user boundary; - the misleading "no credentials" WARNING on drop — minor/cosmetic, totally fine as a tiny follow-up or even just documented.
For the record, here are the manual scripts I ran against a live Polaris while reviewing (credentials redacted). Sharing in case they are useful to fold into the suite or to re-run — the cascade loop and the warning repro are what surfaced the two follow-ups above.
pr-255-manual-tests.sql
-- ============================================================================
-- PR #255 - REST catalog credentials via CREATE USER MAPPING
-- Manual test script (run with psql, e.g.: psql -d postgres -f pr-255-manual-tests.sql)
--
-- Goes from simple -> complex:
-- 0. Setup / sanity
-- 1. Happy path (your working example)
-- 2. Per-user USER MAPPING + SET ROLE
-- 3. Credential resolution order (server vs user mapping)
-- 4. Option validation / error cases
-- 5. GUC restriction (the security fix)
-- 6. Redaction of secrets in query text
-- 7. Built-in catalog server protections
-- 8. ALTER USER MAPPING flows (rotate / add / drop)
-- 9. ALTER SERVER endpoint guard with dependent tables
-- 10. Token cache behavior (multiple roles in one backend)
-- 99. Cleanup
--
-- Real credentials/endpoint live in the \set lines right below so you can
-- swap them in one place. They are substituted with :'NAME' (auto-quoted).
-- ============================================================================
\set ON_ERROR_STOP off
\set ENDPOINT '<your-polaris-endpoint>'
\set CID '<your-client-id>'
\set CSECRET '<your-client-secret>'
-- ============================================================================
-- 0. SETUP / SANITY
-- ============================================================================
\echo '=== 0. setup / sanity ==='
CREATE EXTENSION IF NOT EXISTS pg_lake CASCADE;
-- Confirm pg_lake_iceberg is at >= 3.4 (the version that adds iceberg_catalog).
SELECT extname, extversion FROM pg_extension WHERE extname = 'pg_lake_iceberg';
-- The iceberg_catalog FDW and the three built-in structural servers.
SELECT fdwname FROM pg_foreign_data_wrapper WHERE fdwname = 'iceberg_catalog';
SELECT srvname, srvtype
FROM pg_foreign_server
WHERE srvname LIKE 'pg_lake_%catalog'
ORDER BY srvname;
-- ----------------------------------------------------------------------------
-- Robust from-scratch reset.
--
-- IMPORTANT FOOTGUN: dropping a *writable* REST iceberg table sends a DELETE
-- to the remote catalog, which needs credentials. If the USER MAPPING that
-- held those credentials is already gone, the table drop fails with
-- "no credentials found" and the object is wedged. DROP SERVER ... CASCADE is
-- unsafe for the same reason -- it may drop the mapping before the tables.
--
-- So we always: (1) (re)create a PUBLIC mapping so drops can authenticate,
-- (2) drop every table on these servers, (3) drop mappings, (4) drop servers.
-- RESET ROLE first in case a previous partial run left us inside SET ROLE.
-- ----------------------------------------------------------------------------
RESET ROLE;
-- (1) ensure credentials exist before dropping anything that needs them.
-- (errors harmlessly with ON_ERROR_STOP off if the server is absent)
CREATE USER MAPPING IF NOT EXISTS FOR PUBLIC SERVER my_polaris
OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE USER MAPPING IF NOT EXISTS FOR PUBLIC SERVER my_polaris2
OPTIONS (client_id :'CID', client_secret :'CSECRET');
-- wrong_creds_srv may be left with a BAD secret if an earlier run aborted
-- mid-rotation; force-set good creds so its dependent tables can be dropped.
CREATE USER MAPPING IF NOT EXISTS FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (client_id :'CID', client_secret :'CSECRET');
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (SET client_id :'CID', SET client_secret :'CSECRET');
-- (2) drop every iceberg table that depends on any of our test servers,
-- including ad-hoc leftovers from earlier manual runs.
--
-- NOTE: pg_lake iceberg tables do NOT set ft.ftserver to the catalog
-- server; the catalog link is recorded in pg_depend (which is what
-- DROP SERVER ... CASCADE follows). So we discover dependents via
-- pg_depend, not pg_foreign_table.ftserver. We must drop them here,
-- while the PUBLIC mapping above still supplies credentials for the
-- per-table remote DELETE.
DO $reset$
DECLARE r record;
BEGIN
FOR r IN
SELECT DISTINCT n.nspname, c.relname
FROM pg_depend d
JOIN pg_foreign_server s ON s.oid = d.refobjid AND d.refclassid = 'pg_foreign_server'::regclass
JOIN pg_class c ON c.oid = d.objid AND d.classid = 'pg_class'::regclass
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE s.srvname IN ('my_polaris', 'my_polaris2', 'no_creds_srv', 'wrong_creds_srv')
AND c.relkind = 'f'
LOOP
EXECUTE format('DROP TABLE IF EXISTS %I.%I', r.nspname, r.relname);
END LOOP;
END
$reset$;
DROP TABLE IF EXISTS t_simple;
DROP TABLE IF EXISTS t_dep;
-- (3) drop mappings, then (4) drop servers.
DROP USER MAPPING IF EXISTS FOR analyst SERVER my_polaris;
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER my_polaris;
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER my_polaris2;
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER wrong_creds_srv;
DROP SERVER IF EXISTS my_polaris CASCADE;
DROP SERVER IF EXISTS my_polaris2 CASCADE;
DROP SERVER IF EXISTS no_creds_srv CASCADE;
DROP SERVER IF EXISTS wrong_creds_srv CASCADE;
-- ============================================================================
-- 1. HAPPY PATH (your working example)
-- ============================================================================
\echo '=== 1. happy path: server + PUBLIC mapping + table ==='
-- 1a. Server carries only non-secret config (no credentials here).
CREATE SERVER my_polaris TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
-- 1b. Credentials attached via a PUBLIC user mapping.
CREATE USER MAPPING FOR PUBLIC SERVER my_polaris
OPTIONS (client_id :'CID', client_secret :'CSECRET');
-- Inspect what got stored (note: client_secret is visible to superuser here;
-- redaction is about the *query text*, not pg_user_mappings).
SELECT srvname, umoptions
FROM pg_user_mappings
WHERE srvname = 'my_polaris';
-- 1c. Create a table against the catalog (this triggers a real REST call).
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (1), (2), (3);
SELECT count(*) AS rows_in_t_simple FROM t_simple;
SELECT * FROM t_simple ORDER BY a;
DROP TABLE t_simple;
-- ============================================================================
-- 2. PER-USER USER MAPPING + SET ROLE
-- ============================================================================
\echo '=== 2. per-user mapping + SET ROLE ==='
-- A non-superuser role that can use the FDW (lake_write is granted USAGE in 3.4).
DROP ROLE IF EXISTS analyst;
CREATE ROLE analyst LOGIN;
-- Role grants needed to create/write iceberg tables:
-- lake_write -> USAGE on the iceberg_catalog FDW + built-in catalogs
-- lake_read_write -> USAGE on the pg_lake_iceberg data server (the server
-- the iceberg foreign tables actually attach to)
GRANT lake_write, lake_read_write TO analyst;
-- A user-created catalog server needs an explicit USAGE grant for non-owners.
GRANT USAGE ON FOREIGN SERVER my_polaris TO analyst;
-- Since PG15 the public schema does not allow non-owners to CREATE; grant it
-- so analyst can create the test table here.
GRANT CREATE ON SCHEMA public TO analyst;
-- Per-user mapping for analyst (wins over PUBLIC for that role).
DROP USER MAPPING IF EXISTS FOR analyst SERVER my_polaris;
CREATE USER MAPPING FOR analyst SERVER my_polaris
OPTIONS (client_id :'CID', client_secret :'CSECRET');
-- There should now be a PUBLIC mapping and an analyst mapping on my_polaris.
SELECT usename, srvname
FROM pg_user_mappings
WHERE srvname = 'my_polaris'
ORDER BY usename NULLS FIRST;
-- analyst resolves to its own mapping; this should succeed.
SET ROLE analyst;
SELECT current_user;
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (10), (20);
SELECT count(*) AS analyst_rows FROM t_simple;
DROP TABLE t_simple;
RESET ROLE;
-- ============================================================================
-- 3. CREDENTIAL RESOLUTION ORDER (server vs user mapping)
-- Order for user-created servers: server options < user mapping (UM wins).
-- `scope` is the only option allowed on BOTH; UM scope overrides server scope.
-- ============================================================================
\echo '=== 3. resolution order: scope on server vs user mapping ==='
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER my_polaris2;
DROP SERVER IF EXISTS my_polaris2 CASCADE;
-- Server sets a scope; user mapping overrides it.
CREATE SERVER my_polaris2 TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT', scope 'PRINCIPAL_ROLE:ALL');
CREATE USER MAPPING FOR PUBLIC SERVER my_polaris2
OPTIONS (client_id :'CID', client_secret :'CSECRET', scope 'PRINCIPAL_ROLE:ALL');
-- Both layers are accepted; the user-mapping scope is the effective one.
SELECT srvname, srvoptions FROM pg_foreign_server WHERE srvname = 'my_polaris2';
SELECT srvname, umoptions FROM pg_user_mappings WHERE srvname = 'my_polaris2';
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris2');
SELECT count(*) FROM t_simple;
DROP TABLE t_simple;
DROP USER MAPPING FOR PUBLIC SERVER my_polaris2;
DROP SERVER my_polaris2 CASCADE;
-- ============================================================================
-- 4. OPTION VALIDATION / ERROR CASES (each statement below should ERROR)
-- ============================================================================
\echo '=== 4. validation errors (each should fail) ==='
-- 4a. Credentials are NOT valid on a SERVER (only on a user mapping).
\echo '--- 4a. client_id/client_secret on SERVER -> error ---'
CREATE SERVER bad_srv TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT', client_id :'CID');
-- 4b. Server-only options are NOT valid on a USER MAPPING.
-- Target CURRENT_USER (which has no mapping on my_polaris yet) so we hit the
-- option-context validation, not a "user mapping already exists" conflict.
\echo '--- 4b. rest_endpoint on USER MAPPING -> error ---'
CREATE USER MAPPING FOR CURRENT_USER SERVER my_polaris
OPTIONS (rest_endpoint :'ENDPOINT');
-- 4c. Invalid auth type.
\echo '--- 4c. invalid rest_auth_type -> error ---'
CREATE SERVER bad_auth TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT', rest_auth_type 'banana');
-- 4d. Empty value rejected.
\echo '--- 4d. empty rest_endpoint -> error ---'
CREATE SERVER bad_empty TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint '');
-- 4e. Endpoint without a URI scheme rejected.
\echo '--- 4e. rest_endpoint without scheme -> error ---'
CREATE SERVER bad_scheme TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint 'polaris.example.com/no/scheme');
-- 4f. Unknown option name rejected (hint lists valid options).
\echo '--- 4f. unknown option -> error ---'
CREATE SERVER bad_opt TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT', bogus_option 'x');
-- 4g. Server with endpoint but NO user mapping -> "no credentials found"
-- with the user-created-server hint (NOT the GUC hint).
\echo '--- 4g. no user mapping -> no credentials found (UM hint) ---'
CREATE SERVER no_creds_srv TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'no_creds_srv');
DROP SERVER no_creds_srv CASCADE;
-- ============================================================================
-- 4b. WRONG CREDENTIALS (well-formed, but rejected by the catalog)
-- Unlike group 4g ("no credentials found", which fails locally at
-- resolution time), these credentials are present and non-empty, so they
-- pass local validation. They fail later, at the OAuth *token request*
-- against Polaris, with an error like:
-- ERROR: Rest Catalog OAuth token request failed (HTTP 401)
-- ============================================================================
\echo '=== 4b. wrong credentials: well-formed but rejected by catalog ==='
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER wrong_creds_srv;
DROP SERVER IF EXISTS wrong_creds_srv CASCADE;
CREATE SERVER wrong_creds_srv TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
-- 4b-i. valid client_id, bogus client_secret -> token request rejected.
\echo '--- 4b-i. wrong client_secret -> OAuth token request fails ---'
CREATE USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (client_id :'CID', client_secret 'totally-wrong-secret');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'wrong_creds_srv');
-- 4b-ii. bogus client_id, valid client_secret -> token request rejected.
\echo '--- 4b-ii. wrong client_id -> OAuth token request fails ---'
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (SET client_id 'totally-wrong-id', SET client_secret :'CSECRET');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'wrong_creds_srv');
-- 4b-iii. start valid, then rotate to a wrong secret: the ALTER invalidates the
-- cached token, so the very next catalog call re-authenticates with the
-- bad secret and the query fails on a previously-working table.
\echo '--- 4b-iii. rotate good -> bad: existing-table queries start failing ---'
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (SET client_id :'CID', SET client_secret :'CSECRET');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'wrong_creds_srv');
INSERT INTO t_simple VALUES (1); -- works (good creds)
SELECT count(*) AS rows_before_rotation FROM t_simple;
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (SET client_secret 'now-its-wrong'); -- invalidates token cache
-- The WRITE path reliably forces re-authentication, so this INSERT must FAIL
-- (token cache was invalidated by the ALTER; re-auth uses the bad secret).
\echo '--- this INSERT should FAIL (re-auth with bad secret) ---'
INSERT INTO t_simple VALUES (2);
-- NOTE: a SELECT here may still SUCCEED -- once the table''s Iceberg metadata
-- has been resolved in this session, reads can be served from cached metadata
-- plus object-store credentials without re-hitting the catalog token endpoint.
-- So this is informational, not a pass/fail assertion.
\echo '--- this SELECT may still succeed (served from cached metadata) ---'
SELECT count(*) FROM t_simple;
-- restore good creds so the table can be dropped cleanly (remote DELETE needs auth)
ALTER USER MAPPING FOR PUBLIC SERVER wrong_creds_srv
OPTIONS (SET client_secret :'CSECRET');
DROP TABLE t_simple;
DROP USER MAPPING FOR PUBLIC SERVER wrong_creds_srv;
DROP SERVER wrong_creds_srv CASCADE;
-- ============================================================================
-- 5. GUC RESTRICTION (the security fix in this PR)
-- GUC credentials feed ONLY the built-in 'rest' catalog. A user-created
-- server must NOT inherit them, even with the GUCs set in the session.
-- ============================================================================
\echo '=== 5. GUC credentials do NOT leak to user-created servers ==='
-- Set credential GUCs in this session.
SET pg_lake_iceberg.rest_catalog_client_id = :'CID';
SET pg_lake_iceberg.rest_catalog_client_secret = :'CSECRET';
SET pg_lake_iceberg.rest_catalog_host = :'ENDPOINT';
SHOW pg_lake_iceberg.rest_catalog_client_id;
-- 5a. A user-created server with NO user mapping must STILL fail: GUC creds
-- are not inherited. Expect "no credentials found" with the UM hint.
\echo '--- 5a. user server ignores GUC creds -> error ---'
DROP SERVER IF EXISTS guc_leak_srv CASCADE;
CREATE SERVER guc_leak_srv TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'guc_leak_srv');
DROP SERVER guc_leak_srv CASCADE;
-- 5b. The built-in 'rest' catalog DOES use the GUC credentials (this should
-- succeed end-to-end against Polaris, using host/id/secret from GUCs).
\echo '--- 5b. built-in rest catalog uses GUC creds -> ok ---'
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'rest');
INSERT INTO t_simple VALUES (42);
SELECT count(*) AS builtin_rest_rows FROM t_simple;
DROP TABLE t_simple;
RESET pg_lake_iceberg.rest_catalog_client_id;
RESET pg_lake_iceberg.rest_catalog_client_secret;
RESET pg_lake_iceberg.rest_catalog_host;
-- ============================================================================
-- 6. REDACTION of secrets in query text
-- Requires pg_stat_statements (must be in shared_preload_libraries).
-- The whole CREATE/ALTER USER MAPPING statement text is blanked when it
-- carries a credential option; a scope-only ALTER stays visible.
-- ============================================================================
\echo '=== 6. redaction in pg_stat_statements ==='
-- This will fail if pg_stat_statements is not preloaded; that is fine.
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
SELECT pg_stat_statements_reset();
DROP SERVER IF EXISTS redact_srv CASCADE;
CREATE SERVER redact_srv TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
-- 6a. Statement WITH credentials -> should appear redacted.
CREATE USER MAPPING FOR PUBLIC SERVER redact_srv
OPTIONS (client_id 'REDACT_ME_ID', client_secret 'REDACT_ME_SECRET');
-- 6b. scope-only ALTER -> should stay visible (no credential bytes moved).
ALTER USER MAPPING FOR PUBLIC SERVER redact_srv
OPTIONS (ADD scope 'PRINCIPAL_ROLE:VISIBLE_SCOPE');
-- 6c. ALTER that touches a credential -> should be redacted again.
ALTER USER MAPPING FOR PUBLIC SERVER redact_srv
OPTIONS (SET client_secret 'ROTATED_REDACT_ME');
-- Inspect: the secret strings must NOT appear; redaction marker should.
-- The scope-only ALTER (with VISIBLE_SCOPE) SHOULD still be visible.
SELECT query
FROM pg_stat_statements
WHERE query ILIKE '%user mapping%' OR query ILIKE '%redacted%'
ORDER BY query;
\echo '--- sanity: these counts should all be 0 ---'
SELECT
count(*) FILTER (WHERE query ILIKE '%REDACT_ME_ID%') AS leaked_client_id,
count(*) FILTER (WHERE query ILIKE '%REDACT_ME_SECRET%') AS leaked_secret,
count(*) FILTER (WHERE query ILIKE '%ROTATED_REDACT_ME%') AS leaked_rotated
FROM pg_stat_statements;
\echo '--- sanity: scope-only ALTER should still be visible (count >= 1) ---'
SELECT count(*) AS visible_scope_only
FROM pg_stat_statements
WHERE query ILIKE '%VISIBLE_SCOPE%';
DROP USER MAPPING FOR PUBLIC SERVER redact_srv;
DROP SERVER redact_srv CASCADE;
-- ============================================================================
-- 7. BUILT-IN CATALOG SERVER PROTECTIONS (each statement below should ERROR)
-- ============================================================================
\echo '=== 7. built-in server protections (each should fail) ==='
\echo '--- 7a. user mapping on built-in rest catalog -> error ---'
CREATE USER MAPPING FOR PUBLIC SERVER pg_lake_rest_catalog
OPTIONS (client_id :'CID', client_secret :'CSECRET');
\echo '--- 7b. ALTER built-in server -> error ---'
ALTER SERVER pg_lake_rest_catalog OPTIONS (ADD rest_endpoint :'ENDPOINT');
\echo '--- 7c. rename built-in server -> error ---'
ALTER SERVER pg_lake_rest_catalog RENAME TO my_rest;
\echo '--- 7d. create server with reserved long name -> error ---'
CREATE SERVER pg_lake_rest_catalog TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
\echo '--- 7e. create iceberg_catalog server with TYPE postgres -> error ---'
CREATE SERVER wrong_type TYPE 'postgres'
FOREIGN DATA WRAPPER iceberg_catalog;
\echo '--- 7f. create iceberg_catalog server with no TYPE -> error ---'
CREATE SERVER no_type
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
\echo '--- 7g. rename a user-created iceberg_catalog server -> error ---'
ALTER SERVER my_polaris RENAME TO my_polaris_renamed;
\echo '--- 7h. ALTER FOREIGN DATA WRAPPER iceberg_catalog -> error ---'
ALTER FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (ADD x 'y');
-- ============================================================================
-- 8. ALTER USER MAPPING flows (rotate / add / drop)
-- ============================================================================
\echo '=== 8. ALTER USER MAPPING flows ==='
-- Rotate the secret (token cache is invalidated via syscache callback).
ALTER USER MAPPING FOR PUBLIC SERVER my_polaris
OPTIONS (SET client_secret :'CSECRET');
-- Add a scope override on the mapping.
ALTER USER MAPPING FOR PUBLIC SERVER my_polaris
OPTIONS (ADD scope 'PRINCIPAL_ROLE:ALL');
SELECT srvname, umoptions FROM pg_user_mappings WHERE srvname = 'my_polaris' ORDER BY usename NULLS FIRST;
-- Drop the scope override again.
ALTER USER MAPPING FOR PUBLIC SERVER my_polaris
OPTIONS (DROP scope);
-- Table still works after the credential rotation.
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
SELECT count(*) FROM t_simple;
DROP TABLE t_simple;
-- ============================================================================
-- 9. ALTER SERVER endpoint guard with dependent tables
-- Changing rest_endpoint while a dependent iceberg table exists is blocked.
-- ============================================================================
\echo '=== 9. ALTER SERVER rest_endpoint with dependent table -> error ==='
CREATE TABLE t_dep (a int) USING iceberg WITH (catalog = 'my_polaris');
\echo '--- 9a. change rest_endpoint while t_dep exists -> error ---'
ALTER SERVER my_polaris OPTIONS (SET rest_endpoint 'https://other.example.com/polaris');
-- After dropping the dependent table, the same ALTER is allowed (then revert).
DROP TABLE t_dep;
\echo '--- 9b. same ALTER after dropping dependents -> ok, then revert ---'
ALTER SERVER my_polaris OPTIONS (SET rest_endpoint 'https://other.example.com/polaris');
ALTER SERVER my_polaris OPTIONS (SET rest_endpoint :'ENDPOINT');
-- ============================================================================
-- 10. TOKEN CACHE: multiple roles in one backend
-- PUBLIC + per-user mappings; each role resolves its own credentials.
-- ============================================================================
\echo '=== 10. token cache across SET ROLE in one session ==='
-- analyst (per-user mapping) and a second role using PUBLIC.
DROP ROLE IF EXISTS reader;
CREATE ROLE reader LOGIN;
GRANT lake_write, lake_read_write TO reader;
GRANT USAGE ON FOREIGN SERVER my_polaris TO reader;
GRANT CREATE ON SCHEMA public TO reader;
-- analyst uses its own mapping; reader falls back to PUBLIC. Both should work
-- within the same backend, exercising the (serverOid, userMappingOid) cache key.
SET ROLE analyst;
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (1);
SELECT current_user, count(*) FROM t_simple;
DROP TABLE t_simple;
RESET ROLE;
SET ROLE reader;
CREATE TABLE t_simple (a int) USING iceberg WITH (catalog = 'my_polaris');
INSERT INTO t_simple VALUES (2);
SELECT current_user, count(*) FROM t_simple;
DROP TABLE t_simple;
RESET ROLE;
-- ============================================================================
-- 99. CLEANUP
-- ============================================================================
\echo '=== 99. cleanup ==='
DROP TABLE IF EXISTS t_simple;
DROP TABLE IF EXISTS t_dep;
DROP USER MAPPING IF EXISTS FOR analyst SERVER my_polaris;
DROP USER MAPPING IF EXISTS FOR PUBLIC SERVER my_polaris;
DROP SERVER IF EXISTS my_polaris CASCADE;
-- Clear privileges/ownership these roles hold (e.g. CREATE ON SCHEMA public,
-- USAGE on servers) so DROP ROLE doesn't fail with a dependency error.
DROP OWNED BY analyst, reader;
DROP ROLE IF EXISTS analyst;
DROP ROLE IF EXISTS reader;
\echo '=== done ==='
pr-255-cascade-loop.sql
-- ============================================================================
-- PR #255 - CASCADE capture-on-drop stress test (single iteration).
--
-- Exercises the be7f7b17 fix ("capture REST catalog credentials on
-- user-mapping drop"). Dropping a *writable* REST iceberg table sends a
-- remote DELETE that needs credentials. In a DROP SERVER ... CASCADE the
-- user mapping (which holds the credentials) and the dependent tables are
-- both direct dependents of the server, and Postgres visits cascade siblings
-- in descending-OID order. Before the fix, if the mapping was visited first
-- the credentials vanished before the table DELETEs could authenticate and
-- the whole cascade wedged with "no credentials found" -- non-deterministic
-- depending on OID order. The fix captures the credentials into the
-- transaction the moment the mapping is dropped, so the cascade always works.
--
-- This script must complete with ZERO errors/warnings. Run it in a loop:
-- for i in $(seq 1 N); do
-- psql -d postgres -v ON_ERROR_STOP=1 -f pr-255-cascade-loop.sql || break
-- done
-- and watch for any ERROR / WARNING / "no credentials" in the output.
-- ============================================================================
\set ENDPOINT '<your-polaris-endpoint>'
\set CID '<your-client-id>'
\set CSECRET '<your-client-secret>'
\echo '########## CASCADE capture-on-drop iteration start ##########'
-- ----------------------------------------------------------------------------
-- Forgiving reset (a previous *broken* run could leave wedged objects).
-- In the normal case these are all no-ops because every section below tears
-- itself down. Errors here are tolerated but will still print, so a clean
-- run shows nothing.
-- ----------------------------------------------------------------------------
\set ON_ERROR_STOP off
RESET ROLE;
DROP SERVER IF EXISTS cascade_a CASCADE;
DROP SERVER IF EXISTS cascade_b CASCADE;
DROP SERVER IF EXISTS cascade_c CASCADE;
DO $reset$
BEGIN
IF EXISTS (SELECT FROM pg_roles WHERE rolname = 'cascade_user') THEN
EXECUTE 'DROP OWNED BY cascade_user';
END IF;
END
$reset$;
DROP ROLE IF EXISTS cascade_user;
-- From here on, ANY error must fail the iteration (and the loop).
\set ON_ERROR_STOP on
-- ============================================================================
-- Case A: table-first ordering.
-- Mapping is created BEFORE the tables, so the tables get higher OIDs and
-- the cascade visits them first. This is the "easy" ordering, but with many
-- tables it still proves every per-table remote DELETE authenticates.
-- ============================================================================
\echo '--- A: DROP SERVER CASCADE, table-first ordering (5 tables) ---'
CREATE SERVER cascade_a TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER cascade_a
OPTIONS (client_id :'CID', client_secret :'CSECRET');
DO $mktabs$
DECLARE i int;
BEGIN
FOR i IN 1..5 LOOP
EXECUTE format('CREATE TABLE cascade_a_t%s (a int) USING iceberg WITH (catalog = %L)', i, 'cascade_a');
EXECUTE format('INSERT INTO cascade_a_t%s VALUES (%s), (%s)', i, i, i * 10);
END LOOP;
END
$mktabs$;
-- All five writable tables + the mapping go away together; the per-table
-- remote DELETEs must authenticate. Must NOT wedge.
DROP SERVER cascade_a CASCADE;
-- ============================================================================
-- Case B: user-mapping-first ordering + multiple mappings.
-- Tables are created first, THEN the PUBLIC mapping is dropped+recreated so
-- it gets the highest OID and the cascade visits the mapping *before* the
-- tables. This is exactly the ordering that wedged before the fix.
-- A second (per-user) mapping is present too, so the cascade drops several
-- mappings.
-- ============================================================================
\echo '--- B: DROP SERVER CASCADE, user-mapping-first ordering (5 tables, 2 mappings) ---'
CREATE ROLE cascade_user;
CREATE SERVER cascade_b TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER cascade_b
OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE USER MAPPING FOR cascade_user SERVER cascade_b
OPTIONS (client_id :'CID', client_secret :'CSECRET');
DO $mktabs$
DECLARE i int;
BEGIN
FOR i IN 1..5 LOOP
EXECUTE format('CREATE TABLE cascade_b_t%s (a int) USING iceberg WITH (catalog = %L)', i, 'cascade_b');
EXECUTE format('INSERT INTO cascade_b_t%s VALUES (%s)', i, i);
END LOOP;
END
$mktabs$;
-- Flip OID order: recreate the PUBLIC mapping LAST so it is the highest-OID
-- dependent and the cascade visits it first. (Dropping the mapping while
-- tables exist is itself the capture path -- it must not error.)
DROP USER MAPPING FOR PUBLIC SERVER cascade_b;
CREATE USER MAPPING FOR PUBLIC SERVER cascade_b
OPTIONS (client_id :'CID', client_secret :'CSECRET');
-- Cascade now visits the mapping first; capture-on-drop must let it succeed.
DROP SERVER cascade_b CASCADE;
DROP ROLE IF EXISTS cascade_user;
-- ============================================================================
-- Case C: same-transaction DROP USER MAPPING then DROP TABLE.
-- The mapping is dropped first inside the txn; the capture binds its
-- credentials so the table drop fired afterwards can still authenticate its
-- post-commit DELETE.
-- ============================================================================
\echo '--- C: same-txn DROP USER MAPPING then DROP TABLE ---'
CREATE SERVER cascade_c TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER cascade_c
OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE cascade_c_t1 (a int) USING iceberg WITH (catalog = 'cascade_c');
INSERT INTO cascade_c_t1 VALUES (1), (2), (3);
BEGIN;
DROP USER MAPPING FOR PUBLIC SERVER cascade_c;
DROP TABLE cascade_c_t1;
COMMIT;
DROP SERVER cascade_c CASCADE;
-- ============================================================================
-- Verify nothing leaked through.
-- ============================================================================
DO $verify$
BEGIN
IF EXISTS (SELECT FROM pg_foreign_server
WHERE srvname IN ('cascade_a', 'cascade_b', 'cascade_c')) THEN
RAISE EXCEPTION 'CASCADE test FAILED: leftover server(s) remain';
END IF;
IF EXISTS (SELECT FROM pg_class WHERE relname LIKE 'cascade_%_t%') THEN
RAISE EXCEPTION 'CASCADE test FAILED: leftover table(s) remain';
END IF;
END
$verify$;
\echo '########## ITERATION OK ##########'
pr-255-warning-repro.sql
\set ENDPOINT '<your-polaris-endpoint>'
\set CID '<your-client-id>'
\set CSECRET '<your-client-secret>'
\set ON_ERROR_STOP off
DROP SERVER IF EXISTS w_a CASCADE;
DROP SERVER IF EXISTS w_b CASCADE;
DROP SERVER IF EXISTS w_c CASCADE;
\set ON_ERROR_STOP on
\echo ''
\echo '===== CASE 1: plain DROP SERVER CASCADE, natural order (1 table) ====='
CREATE SERVER w_a TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER w_a OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE w_a_t1 (a int) USING iceberg WITH (catalog = 'w_a');
INSERT INTO w_a_t1 VALUES (1);
\echo '--- DROP SERVER w_a CASCADE (expect: clean) ---'
DROP SERVER w_a CASCADE;
\echo ''
\echo '===== CASE 2: same-txn DROP USER MAPPING then DROP TABLE (1 table) ====='
CREATE SERVER w_b TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER w_b OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE w_b_t1 (a int) USING iceberg WITH (catalog = 'w_b');
INSERT INTO w_b_t1 VALUES (1);
\echo '--- BEGIN; DROP USER MAPPING; DROP TABLE; COMMIT (expect: WARNING) ---'
BEGIN;
DROP USER MAPPING FOR PUBLIC SERVER w_b;
DROP TABLE w_b_t1;
COMMIT;
DROP SERVER w_b CASCADE;
\echo ''
\echo '===== CASE 3: plain DROP SERVER CASCADE, mapping rotated after table (1 table) ====='
CREATE SERVER w_c TYPE 'rest' FOREIGN DATA WRAPPER iceberg_catalog OPTIONS (rest_endpoint :'ENDPOINT');
CREATE USER MAPPING FOR PUBLIC SERVER w_c OPTIONS (client_id :'CID', client_secret :'CSECRET');
CREATE TABLE w_c_t1 (a int) USING iceberg WITH (catalog = 'w_c');
INSERT INTO w_c_t1 VALUES (1);
-- recreate the mapping AFTER the table (a routine credential rotation),
-- so the mapping now has a higher OID than the table.
DROP USER MAPPING FOR PUBLIC SERVER w_c;
CREATE USER MAPPING FOR PUBLIC SERVER w_c OPTIONS (client_id :'CID', client_secret :'CSECRET');
\echo '--- DROP SERVER w_c CASCADE (expect: WARNING) ---'
DROP SERVER w_c CASCADE;
\echo ''
\echo '===== done ====='
…P TABLE warning Two follow-ups from review. 1) OAT_DROP capture hook: a server owner is allowed by Postgres to DROP any role's USER MAPPING on the server. The previous capture path snapshotted the dropped mapping's credentials unconditionally, which let the dropping session authenticate same-transaction REST requests as the mapping's owner. Restrict capture to the current session's own mapping or PUBLIC; for other mappings the hook is a no-op, so the live resolver picks up the dropping session's own credentials (or fails clearly). A new pytest sets up a server owner role and a victim role with deliberately-invalid credentials and asserts that the post-commit REST POST does not run under the foreign mapping. 2) MarkAllReferencedFilesForDeletion warning wording: when a USER MAPPING is dropped in the same transaction as a DROP TABLE on a REST iceberg table, the synchronous metadata read goes through the live resolver and surfaces the generic resolver error whose HINT asks the user to create a USER MAPPING -- misleading mid-drop, and the post-commit REST DELETE still succeeds via the captured snapshot. Replace this one specific case with a concise WARNING; the legacy WARNING-with-original-text path is preserved for every other FDW_OPTION_NAME_NOT_FOUND site (e.g. rest_endpoint not configured) by gating on both the errcode and the message prefix. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: sfc-gh-npuka <naisila.puka@snowflake.com>
Description
This change moves credential storage (
client_id,client_secret) out ofCREATE SERVERoptions where they are publicly readable, intoCREATE USER MAPPING— per-user credentials stored inpg_user_mapping, providing user-level isolation.The credential GUCs (
pg_lake_iceberg.rest_catalog_client_id/rest_catalog_client_secret) feed only the built-inpg_lake_rest_catalogserver. User-creatediceberg_catalogservers must supply their own credentials viapg_user_mapping.User-defined catalog with shared credentials
Per-user credential isolation
Different users can have their own credentials on the same server. The resolver picks the mapping for the current user first, falling back to PUBLIC if none exists.
BuildRestCatalogOptionsFromServerresolves options differently depending on which kind of server it is building for, because the credential trust boundary differs.Built-in
pg_lake_rest_catalog(catalog'rest')rest_catalog_client_id/rest_catalog_client_secret.ALTER SERVERon the built-in is blocked.The built-in skips the user-mapping phase:
CREATE USER MAPPINGon it is rejected upfront, so its credentials live exclusively in GUCs and the built-in stays a single, instance-wide configuration.User-created server (
CREATE SERVER ... FOREIGN DATA WRAPPER iceberg_catalog)rest_catalog_client_id/rest_catalog_client_secretare intentionally not inherited.CATALOG_OPT_CTX_SERVER.pg_user_mappingoptions — per-current-user lookup, fallback to PUBLIC.Why credentials are gated to the built-in: any role with
USAGEon theiceberg_catalogFDW (lake_write, via the 3.4 grant) canCREATE SERVERand chooserest_endpoint/oauth_endpoint. If user-created servers inherited the system-wide credential GUCs, the nextCREATE TABLE ... USING iceberg WITH (catalog='evil')would POST the production credentials to whatever endpoint that server's owner picked — a non-superuser credential-exfiltration path. So GUC credentials are restricted to the single extension-owned built-in server.scopeis accepted in both server options and user-mapping options. The effective priority is: user mapping > server option > GUC.ValidateRestCatalogOptionsruns after all layers have been folded in and raises"no credentials found for REST catalog ..."up front on first DML ifclient_id/client_secretare missing for the configured auth flow.The
iceberg_catalog_validatordistinguishes between server and user-mapping contexts via aCATALOG_OPT_CTX_*bitmask on each option descriptor:CATALOG_OPT_CTX_SERVER):rest_endpoint,scope,rest_auth_type,oauth_endpoint,enable_vended_credentials,location_prefix,catalog_name.CATALOG_OPT_CTX_USER_MAPPING):client_id,client_secret,scope.scopeappears in both contexts. The validator checksdesc->contexts & contextBitand produces context-specific hint strings viaGetValidCatalogOptionsHint(contextBit)— server hints only list server options, user-mapping hints only list user-mapping options.ValidateIcebergCatalogServerDDL(aProcessUtilityhandler) enforces several invariants oniceberg_catalogservers and the user mappings that target them:CREATE SERVERandRENAME TOcannot use any of the reserved short names (rest,postgres,object_store) or the long names (pg_lake_rest_catalog,pg_lake_postgres_catalog,pg_lake_object_store_catalog).ALTER SERVERoptions, owner changes, andCREATE/ALTER/DROP USER MAPPINGagainst built-ins are rejected.CREATE SERVERmust specifyTYPE 'rest'.TYPE 'postgres'/'object_store'are reserved for the built-in servers.iceberg_catalogserver is blocked. Dependent iceberg tables record the server name as a string option (catalog='<name>') inftoptions; a rename would silently break those references.rest_endpoint/oauth_endpointcannot be changed on a user-created server while it has user mappings or dependent iceberg tables. Both options pin the URL the OAuth grant is POSTed to; flipping them under existing mappings would redirect those mappings' credentials.ALTER USER MAPPING ... OPTIONS (DROP client_id|client_secret)is blocked while the server has dependent iceberg tables.SET/ADD(rotation) remains allowed.ALTER EXTENSION pg_lake_iceberg DROP SERVER|FOREIGN DATA WRAPPERis rejected foriceberg_catalogobjects, preventing detachment of theDEPENDENCY_EXTENSIONedge that shields them from standaloneDROP.ALTER FOREIGN DATA WRAPPER iceberg_catalogis rejected, preventing replacement of the validator or addition of a handler.The last two are defense-in-depth against superuser misuse: both already require superuser privilege, but a superuser detaching the extension edge or swapping the validator would silently weaken the protection model.
DROP USER MAPPINGand cascade safetyA chained
object_access_hook(InitializeIcebergCatalogObjectAccessHookinpg_lake_iceberg) fires onOAT_DROPfor any user mapping on a user-creatediceberg_catalogserver with dependent iceberg tables. The hook does not error — that would have madeDROP SERVER ... CASCADEandDROP EXTENSION ... CASCADEnon-deterministic, since Postgres visits cascade siblings in descending-OID order and the same command would either succeed or abort depending on which OID was higher. Instead, the hook captures the about-to-vanish credentials into the transaction:pg_lake_icebergowns the predicate (ServerHasDependentRestIcebergTableplus theiceberg_catalog-FDW filter) and the dispatch site.pg_lake_tableregisters a hook-pointer (PgLake_RestCatalogXactCaptureCallback) at_PG_inittime. The callback deep-copies the resolvedRestCatalogOptionsintoTopTransactionContextvia the newBuildRestCatalogOptionsFromUserMapping.DROP TABLE— direct or cascade-driven underDROP SERVER/DROP EXTENSION ... CASCADE— authenticates its post-commit RESTDELETEagainst the captured snapshot. Whichever sibling Postgres visits first wins; the cascade always succeeds.DROP USER MAPPINGcommits in one txn,DROP TABLEruns in a later one) still fails clearly with the standard"no credentials found"error — captures are txn-local by design. Recovery is to recreate the mapping.Credentials in
CREATE USER MAPPINGandALTER USER MAPPINGDDL appear in plaintext inpg_stat_statements,log_min_duration_statement, andereporterror contexts. AProcessUtilityhandler (RedactRestCatalogUserMappingSecrets) scrubs the query-string slice for any such statement that carries a credential option (client_idorclient_secret).Whole-statement scrub
The handler overwrites the entire utility-statement slice (bounded by
pstmt->stmt_locationandpstmt->stmt_len) with a fixed<redacted: USER MAPPING with credentials>marker, padding with spaces so the buffer stays the same length. Surrounding statements in a multi-statement query string are left untouched. DDL execution is unaffected becauseCREATE/ALTER USER MAPPINGreads option values from the parse tree (DefElemnodes), not fromqueryString.This is deliberately coarser than a per-option string-literal scrub: it trades a little ops observability (only statements that actually carry a credential are scrubbed — a non-secret
ALTER USER MAPPING SET scope '...'is left visible) for a much smaller attack surface than hand-rolling a parser for''doubled-quote escapes,E''escape strings, andU&''Unicode strings.Conservative server-name gate
IsKnownNonIcebergCatalogServerreturns true only when we can affirmatively prove the target is a real foreign server backed by a different FDW. Every other input — a typo, a not-yet-existent server, a built-iniceberg_cataloglong name, a NULL name — falls into the "could be iceberg" bucket and is scrubbed defensively. This closes the typo-leak: a fat-fingered server name on aCREATE USER MAPPING ... OPTIONS (client_id, client_secret)would otherwise survive intopg_stat_statementsand the failing core lookup's error context. A real, non-iceberg FDW mapping is left alone — we don't volunteer ourselves as redactor for someone else's option vocabulary.Handler ordering
RedactRestCatalogUserMappingSecretsis registered in_PG_initafterValidateIcebergCatalogServerDDL. BecauseRegisterUtilityStatementHandlerprepends to a LIFO list, redaction runs before validation — so a failing built-in-server path never surfaces an unredacted query string in its error context.The per-backend token cache is keyed by the pair (
serverOid,userMappingOid) instead ofserverOidalone. This ensures that differentSET ROLEs in the same backend each get the credentials of their own user mapping (or PUBLIC), whileInvalidOidis used when no user mapping is involved (built-in server, or a user-created server falling back to GUCs). A syscache invalidation callback onUSERMAPPINGOIDis registered alongside the existingFOREIGNSERVEROIDcallback, soALTER USER MAPPINGandDROP USER MAPPINGimmediately flush stale tokens.Three new helpers in
rest_catalog.cunderpin the cascade-safe credential capture and the same-txn same-server identity check:BuildRestCatalogOptionsFromUserMapping(Oid umOid)— resolves a fully-validatedRestCatalogOptionsfrom a specific user-mapping OID, bypassing the per-current-user resolution path. ReturnsNULLwhen the mapping is no longer in the syscache; ereports the standard"no credentials found for REST catalog ..."on a malformed mapping (the cascade transaction aborts cleanly).ResolveRestCatalogServerId(const char *catalog)— returns just the iceberg_catalog server OID for a user-facing catalog identifier, skipping the user-mapping lookup and credential validation thatResolveRestCatalogOptionsdoes.GetRestCatalogServerIdForRelation(Oid relationId)— relation-keyed companion to the above.EnsureXactBoundToRestCataloginpg_lake_tableuses these so that after the first mutation in a transaction (which captures full credentials via the per-current-user resolver), every subsequent mutation only re-resolves the server OID. Nopg_user_mappinglookup, no credential validation — the same-server identity check stays correct even after the user mapping has been dropped earlier in the same transaction (same-txnDROP USER MAPPINGthenDROP TABLE, or cascade-driven UM removal).LookupUserMappingOptions/LookupUserMappingOptionsByOidTwo
pg_user_mappinglookup helpers:LookupUserMappingOptions(Oid serverOid, Oid *umidOut)— resolves via the current user (GetUserId()) with a PUBLIC fallback. Used by the per-server resolver.LookupUserMappingOptionsByOid(Oid umOid, Oid *serverOidOut)— by-OID counterpart used by the OAT_DROP capture path.Both return the untransformed option list, or
NIL(with*outOid = InvalidOid) when no mapping is found, avoiding theERRORthat PostgreSQL'sGetUserMappingraises.ApplyUserMappingOptionsListshares the option-application loop between both call sites.ValidateRestCatalogOptionsChecks the resolved options carry the minimum fields needed for the configured auth flow at resolution time — after GUCs and the user mapping have been folded in. Credential requirements are auth-type specific:
client_idANDclient_secret(Basic auth header).client_secretonly (client_idis intentionally ignored).This means an incompletely-configured catalog fails up front on first DML rather than silently issuing an unauthenticated request to the OAuth endpoint.
test_iceberg_catalog_server.pygains coverage for:CREATE/ALTER/DROP USER MAPPINGblocked on the built-in server.CREATE USER MAPPING,ALTER USER MAPPING,''doubled-quote escapes,E''strings,scopeleft untouched, non-iceberg servers skipped, redaction runs before built-in rejection, stored credentials preserved, plustest_redact_runs_for_unknown_server_namefor the conservative typo case.ALTER SERVERguards:rest_endpoint/oauth_endpointrejected while the server has user mappings or dependent iceberg tables.ALTER USER MAPPINGguards:DROP client_id|client_secretrejected while the server has dependent iceberg tables.ALTER EXTENSION ... DROP SERVER|FDWandALTER FOREIGN DATA WRAPPER iceberg_catalogrejected.test_modify_iceberg_rest_table.pyadds end-to-end coverage for:ALTER USER MAPPINGinvalidates cached tokens mid-session.test_drop_user_mapping_in_isolation_succeeds— isolatedDROP USER MAPPINGsucceeds (capture happens but goes unused).test_drop_user_mapping_then_drop_table_in_same_txn_succeeds— same-txn drops authenticate against the captured snapshot.test_drop_user_mapping_then_drop_table_in_separate_txn_fails_clearly— cross-txn failure mode with the standard error message.test_drop_server_cascade_does_not_wedge[um_before_table | table_before_um]— parametrized over both OID orderings, asserting unconditional cascade success.Existing tests that used
client_id/client_secretinCREATE SERVER OPTIONSwere migrated toCREATE USER MAPPING FOR PUBLIC.Fixes remaining part of #230
Checklist