Skip to content

feat: add multi-database support via df.start() database parameter - #41

Merged
Pino de Candia (pinodeca) merged 3 commits into
mainfrom
pinodeca/multi-database
Mar 6, 2026
Merged

Pino de Candia (pinodeca) merged 3 commits into
mainfrom
pinodeca/multi-database

Conversation

@pinodeca

Copy link
Copy Markdown
Contributor

Summary

Allow durable functions to target any database on the cluster by adding an optional database parameter to df.start(). A single function invocation targets exactly one database; cross-database workflows can use dblink/postgres_fdw or separate durable functions.

Design spec: docs/multi-database.md

API

-- Existing (unchanged):
SELECT df.start(df.sql('SELECT 1'));
SELECT df.start(df.sql('SELECT 1'), 'my-label');

-- New: target a specific database
SELECT df.start(df.sql('SELECT 1'), database => 'analytics');
SELECT df.start(df.sql('SELECT 1'), 'my-label', 'analytics');

Invalid database names are rejected immediately at df.start() time.

Changes

File Change
src/lib.rs Add database TEXT column to df.nodes and df.instances DDL
src/dsl.rs Add database param to df.start() with pg_database validation
src/types.rs Add database to FunctionNode; add database param to connect_as_user()
src/activities/execute_sql.rs Add database to ExecuteSqlInput, pass to connect_as_user()
src/activities/load_function_graph.rs Include database in node SELECT
src/orchestrations/execute_function_graph.rs Include node.database in SQL activity input
tests/e2e/sql/34_multi_database.sql E2E test: cross-db execution, invalid db, regression
scripts/test-e2e-local.sh Add test 34 to superuser list

Testing

  • All 75 unit tests pass
  • All E2E tests pass including new test 34 (cross-database execution, invalid database rejection, backward compatibility)
  • Clippy clean, fmt clean

Allow durable functions to target any database on the cluster by adding
an optional 'database' parameter to df.start(). A single function
invocation targets exactly one database; cross-database workflows can
use dblink/postgres_fdw or separate durable functions.

Changes:
- Add 'database TEXT' column to df.nodes and df.instances
- Add 'database' parameter to df.start() with pg_database validation
- Thread database through FunctionNode, load_function_graph, execute_sql
  activity, orchestration, and connect_as_user()
- Add E2E test (34_multi_database.sql) covering cross-database execution,
  invalid database rejection, and backward compatibility

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds multi-database targeting to pg_durable by allowing df.start() to specify a target database per durable-function invocation, while keeping pg_durable metadata and orchestration in the extension’s home database.

Changes:

  • Extend df.start() with an optional database parameter and validate it against pg_database.
  • Plumb the database value through nodes/instances metadata, graph loading, orchestration input, and SQL execution connections.
  • Add an E2E test covering cross-database execution, invalid database rejection, and backward compatibility.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/dsl.rs Adds database parameter to df.start(), validates existence, persists it to df.nodes/df.instances.
src/lib.rs Adds database column to df.nodes and df.instances DDL; updates unit tests for new start() signature.
src/types.rs Adds database to FunctionNode; extends connect_as_user() to accept an optional database.
src/activities/load_function_graph.rs Loads database from df.nodes into FunctionNode.
src/orchestrations/execute_function_graph.rs Passes node.database into execute_sql activity input JSON.
src/activities/execute_sql.rs Includes database in activity input and uses it to connect to the right DB; improves trace logging.
tests/e2e/sql/34_multi_database.sql New E2E test for cross-database execution + invalid DB rejection + regression behavior.
scripts/test-e2e-local.sh Ensures test 34 runs under superuser (creates/drops a database).
docs/multi-database.md Design spec documenting API, behavior, schema, and testing strategy.

Comment thread src/dsl.rs
Comment thread tests/e2e/sql/34_multi_database.sql Outdated
Comment thread src/lib.rs
Comment thread src/lib.rs
- Handle SPI errors explicitly in database existence check (dsl.rs)
  instead of swallowing via .ok().flatten()
- Move SET/RESET SESSION AUTHORIZATION outside DO block in test 34
  where PL/pgSQL does not allow these statements
- Include database name in connect_as_user error message for
  easier debugging of multi-database connection failures
- Add test 4: multi-node sequence graph targeting another database
- Add test 5: deferred connection failure when database is dropped
  after df.start() (uses df.loop to verify clean failure)
@pinodeca
Pino de Candia (pinodeca) merged commit 7e8a035 into main Mar 6, 2026
5 checks passed
@pinodeca
Pino de Candia (pinodeca) deleted the pinodeca/multi-database branch March 6, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants