Proposal: add instant database provisioning for Memori BYODB

[bb7133]

Summary

Memori already supports BYODB through user-provided databases such as PostgreSQL, MySQL, CockroachDB, OceanBase, and TiDB.

This proposal is to add a small database provisioning layer for BYODB, starting with providers such as TiDB Cloud Zero and Neon Launchpad.

The goal is to make the BYODB setup flow easier for developers and AI agents: Memori can provision a database, receive a connection string, initialize the existing schema, and return a ready-to-use Memori instance.

This should not introduce new storage drivers. Provisioning should only create or retrieve a database connection; storage should continue using the existing MySQL/TiDB and PostgreSQL driver paths.

Motivation

BYODB is flexible, but today users still need to manually create a database, copy the connection string, configure TLS if needed, and then initialize Memori.

For agent and developer-tool workflows, it would be useful if Memori could handle this setup step directly while still keeping the BYODB architecture intact.

Examples of provider candidates:

• TiDB Cloud Zero for the MySQL/TiDB family
• Neon Launchpad for the PostgreSQL family

Both can return normal SQL connection strings, which means Memori can reuse its existing storage drivers after provisioning.

Proposed API

For Python:

mem = Memori.provision(provider="tidb-zero", build=True)

For CLI:

memori provision --provider tidb-zero
memori provision --provider neon-launchpad

A provider could return:

ProvisionResult(
    provider="tidb-zero",
    family="mysql",
    dsn="...",
    connect_args={...},
    claim_url="...",
    expires_at="...",
    metadata={...},
)

Memori would then pass the DSN to the existing storage layer and run the normal schema build path.

Important details

• Keep provisioning separate from storage drivers.
• Cache provisioned results locally to avoid creating duplicate databases on reruns.
• Redact credentials in logs and CLI output.
• Surface claim_url and expires_at clearly when the provider returns them.
• Keep provider implementations isolated because provider APIs may evolve.

Non-goals

• This is not a replacement for Memori's hosted storage service.
• This is not a new TiDB Zero or Neon-specific storage driver.
• TiDB native vector/HNSW support should be handled separately as a future storage optimization.
• Authenticated cloud providers such as CockroachDB, Supabase, or Atlas can remain normal BYODB targets unless there is demand for provider-specific provisioning later.

[devwdave]

Hi @bb7133,

This is a novel idea with merit. The Memori Labs team is going to add this to our priority list and will provide updates here as we make progress.

Thank you for the suggestion!

[om-prakash16]

Hi @bb7133 and @devwdave,

I’d love to take a crack at building this provisioning layer! I work with provisioning APIs like Neon frequently, and I really like the architectural decision to keep this strictly separated from the underlying storage drivers.

Here is my proposed plan of attack:

1. Data Model: Create the ProvisionResult dataclass/Pydantic model to strictly define the expected return payload.
2. Provider Implementation: Start with the neon-launchpad integration, hooking into their REST API to generate the database and grab the ephemeral connection string.
3. Caching Layer: Implement local caching (e.g., writing to a .memori_cache.json file) so that subsequent script runs reuse the same database until the expires_at threshold is hit.
4. Integration & Security: Wire up the Memori.provision() class method and the CLI commands, specifically ensuring that the password in the DSN is redacted before any console logging.

[devwdave]

Hi @om-prakash16, thanks for the thoughtful plan. This is aligned with the direction I had in mind.

I’m going to take the first pass on the shared provisioning foundation, including the provider interface, ProvisionResult, Memori.provision(), CLI wiring, caching, and credential redaction.

Once that foundation is in place, Neon Launchpad support would be a great follow up contribution. It should plug into the shared provider interface and return a normal PostgreSQL DSN for the existing Postgres storage path.

I’ll continue with the core layer and TiDB Zero support first, then Neon can follow using the same pattern.

[om-prakash16]

Thanks for the clarification @devwdave - sounds great.

I’ll follow the shared provisioning work in #536 closely and would be happy to contribute the Neon Launchpad provider once the foundation layer is in place.

Really like the direction of keeping provisioning decoupled from storage drivers.

[devwdave]

I'm going to close this issue as we now have support for db provisioning in the python sdk. I will open a new issue for adding support for Neon Launchpad.