3.0 postgres - Next.js Standalone mode not possible

[yobottehg]

Link to reproduction

No response

Payload Version

3.0

Node Version

20 LTS

Next.js Version

15 canary 58

Describe the Bug

As per discussion in Discord opening a separate issue for this:

Problem:

• The docker image in the beta docs will not work with postgres because the Payload cli will not be bundled in Next.js standalone output.
• The .bin directory is missing so it's not possible to call payload mgirate
• Image size without standalone output is nearly 10x

Some more details in this comment:
#7050 (comment)

And this Discord discussion:
https://discord.com/channels/967097582721572934/1262348369293213798

Reproduction Steps

• Use payload v3 with postgres
• Deploy it via docker image from the beta docs
• Try to run migrations

Adapters and Plugins

db-postgres

[jmikrut]

Hey @yobottehg — thanks for bringing this up! Here are some thoughts on this one.

You're right in that Next.js standalone mode does not bundle the dependencies necessary for Payload's CLI to work. This is because Payload's CLI is meant to be able to parse your raw TS Payload config. We do this with @swc/core which is a fairly large module. In the Postgres case, specifically, drizzle-kit is also a large package that is only meant to be run from the CLI.

So the options here are as follows:

Run migrations after build

You can adjust the Dockerfile to run migrations after the build step, where you have full access to all node_modules dependencies. Generally, we see people using a script like pnpm ci which does something like next build && payload migrate.

Don't use standalone mode

You can also just disable standalone mode if you 1000% want to run migrations at runtime, in your production Docker image. You will definitely need to have a larger production image though, in that case, due to the fact that you'll need to run Payload's CLI and Drizzle's CLI.

HOWEVER, all of this being said, the team and I have been brainstorming in regards to a way to run migrations programmatically, from payload.onInit or similar.

I think we could achieve that as a feature that we would build in the future. Here's what would need to happen in order to allow running migrations from your production code:

1. Make it so that Postgres migration files do not require any node_modules access - meaning no imports. Right now, in Postgres, we import sql from @payloadcms/db-postgres which would not work at runtime in production if you are in Next.js standalone mode. If we removed those imports from our migration templates, and instead exposed sql as a property on the Drizzle DB adapter itself (so we could call it as payload.db.sql(), we could leverage the sql function from the bundled standalone JS without having to import it again in a migration file.
2. Create a TypeScript build step that builds your migrations folder into raw JS, which can be loaded in production without the use of TS loaders like swc
3. Make the Dockerfile call our new ts build step to build migrations for production
4. Make sure the built JS migrations get included in the standalone output folder
5. Then just execute payload.db.migrate() in your Payload config's onInit method, which looks in the standalone/migrations folder or similar

That would be the most ideal way to allow running Payload migrations at runtime. In the meantime, if you absolutely need to run migrations at runtime / server start in production, you should just disable standalone mode for now.

But if you can get away with running migrations at build time, then simply update the command your Dockerfile uses to build, like this: build: next build && payload migrate.

I will convert this to a feature request as we want to track it and make sure that we are in a position to support this in the future. Thanks for bringing it up!

[yobottehg]

Thanks for your reply, all the work and options provided!

My case is that i need to build the docker containers once, push them to a registry and another provider will run them together with the database. So running migrations on build will not work in my case. I also have no access to the DB at all and all environment variables are runtime environment variables where I don't have access to on build time.

I went with a normal build as this suits our case the best.

[jmikrut]

Hey @yobottehg — we've updated Payload 3.x to support production migrations! So now you can use Next.js standalone mode!

To run migrations in production, follow the docs here:
https://github.com/payloadcms/payload/blob/beta/docs/database/migrations.mdx#running-migrations-in-production

Give that a shot!

[yobottehg]

@jmikrut Thank you. Already saw that and will test that with the next release.

[chrisvanmook]

@jmikrut The migrations only work when I include the complete node_modules directory in the image, which is not ideal. When I only include the standalone directory, which should be sufficient, I get Error: Cannot find module 'drizzle-kit/api'

So I have this in my Dockerfile:

COPY --from=builder /app ./
COPY --from=builder /app/apps/payload/.next/standalone/ ./

But I think I should have only this

COPY --from=builder /app/apps/payload/.next/standalone/ ./