Skip to content

Commit f845211

Browse files
atinuxatinux
authored
feat(database): add support for multiple database migrations directories (#423)
1 parent 859a5f8 commit f845211

File tree

17 files changed

+350
-91
lines changed

17 files changed

+350
-91
lines changed

‎docs/content/1.docs/2.features/database.md

+83-16
Original file line numberDiff line numberDiff line change
@@ -241,11 +241,63 @@ This method can have poorer performance (prepared statements can be reused in so
241241

242242
## Database Migrations
243243

244-
Database migrations provide version control for your database schema. They track changes and ensure consistent schema evolution across all environments through incremental updates.
244+
Database migrations provide version control for your database schema. They track changes and ensure consistent schema evolution across all environments through incremental updates. NuxtHub supports SQL migration files (`.sql`).
245+
246+
### Migrations Directories
247+
248+
NuxtHub scans the `server/database/migrations` directory for migrations **for each [Nuxt layer](https://nuxt.com/docs/getting-started/layers)**.
249+
250+
If you need to scan additional migrations directories, you can specify them in your `nuxt.config.ts` file.
251+
252+
```ts [nuxt.config.ts]
253+
export default defineNuxtConfig({
254+
hub: {
255+
// Array of additional migration directories to scan
256+
databaseMigrationsDirs: [
257+
'my-module/db-migrations/'
258+
]
259+
}
260+
})
261+
```
262+
::note
263+
NuxtHub will scan both `server/database/migrations` and `my-module/db-migrations` directories for `.sql` files.
264+
::
265+
266+
If you want more control to the migrations directories or you are working on a [Nuxt module](https://nuxt.com/docs/guide/going-further/modules), you can use the `hub:database:migrations:dirs` hook:
267+
268+
::code-group
269+
```ts [modules/auth/index.ts]
270+
import { createResolver, defineNuxtModule } from 'nuxt/kit'
271+
272+
export default defineNuxtModule({
273+
meta: {
274+
name: 'my-auth-module'
275+
},
276+
setup(options, nuxt) {
277+
const { resolve } = createResolver(import.meta.url)
278+
279+
nuxt.hook('hub:database:migrations:dirs', (dirs) => {
280+
dirs.push(resolve('db-migrations'))
281+
})
282+
}
283+
})
284+
```
285+
```sql [modules/auth/db-migrations/0001_create-users.sql]
286+
CREATE TABLE IF NOT EXISTS users (
287+
id INTEGER PRIMARY KEY,
288+
name TEXT NOT NULL,
289+
email TEXT NOT NULL
290+
);
291+
```
292+
::
293+
294+
::tip
295+
All migrations files are copied to the `.data/hub/database/migrations` directory when you run Nuxt. This consolidated view helps you track all migrations and enables you to use `npx nuxthub database migrations <command>` commands.
296+
::
245297

246298
### Automatic Application
247299

248-
SQL migrations in `server/database/migrations/*.sql` are automatically applied when you:
300+
All `.sql` files in the database migrations directories are automatically applied when you:
249301
- Start the development server (`npx nuxt dev` or [`npx nuxt dev --remote`](/docs/getting-started/remote-storage))
250302
- Preview builds locally ([`npx nuxthub preview`](/changelog/nuxthub-preview))
251303
- Deploy via [`npx nuxthub deploy`](/docs/getting-started/deploy#nuxthub-cli) or [Cloudflare Pages CI](/docs/getting-started/deploy#cloudflare-pages-ci)
@@ -275,7 +327,6 @@ Migration files are created in `server/database/migrations/`.
275327

276328
After creation, add your SQL queries to modify the database schema.
277329

278-
279330
::note{to="/docs/recipes/drizzle#npm-run-dbgenerate"}
280331
With [Drizzle ORM](/docs/recipes/drizzle), migrations are automatically created when you run `npx drizzle-kit generate`.
281332
::
@@ -327,23 +378,39 @@ NUXT_HUB_PROJECT_URL=<url> NUXT_HUB_PROJECT_SECRET_KEY=<secret> nuxthub database
327378
```
328379
::
329380

330-
### Migrating from Drizzle ORM
381+
### Post-Migration Queries
331382

332-
Since NuxtHub doesn't recognize previously applied Drizzle ORM migrations (stored in `__drizzle_migrations`), it will attempt to rerun all migrations in `server/database/migrations/*.sql`. To prevent this:
383+
::important
384+
This feature is for advanced use cases. As the queries are run after the migrations process (see [Automatic Application](#automatic-application)), you want to make sure your queries are idempotent.
385+
::
333386

334-
1. Mark existing migrations as applied in each environment:
387+
Sometimes you need to run additional queries after migrations are applied without tracking them in the migrations table.
335388

336-
```bash [Terminal]
337-
# Local environment
338-
npx nuxthub database migrations mark-all-applied
389+
NuxtHub provides the `hub:database:queries:paths` hook for this purpose:
339390

340-
# Preview environment
341-
npx nuxthub database migrations mark-all-applied --preview
391+
::code-group
392+
```ts [modules/admin/index.ts]
393+
import { createResolver, defineNuxtModule } from 'nuxt/kit'
342394

343-
# Production environment
344-
npx nuxthub database migrations mark-all-applied --production
345-
```
395+
export default defineNuxtModule({
396+
meta: {
397+
name: 'my-auth-module'
398+
},
399+
setup(options, nuxt) {
400+
const { resolve } = createResolver(import.meta.url)
346401

347-
2. Remove `server/plugins/database.ts` as it's no longer needed.
402+
nuxt.hook('hub:database:queries:paths', (queries) => {
403+
// Add SQL files to run after migrations
404+
queries.push(resolve('./db-queries/seed-admin.sql'))
405+
})
406+
}
407+
})
408+
```
409+
```sql [modules/admin/db-queries/seed-admin.sql]
410+
INSERT OR IGNORE INTO admin_users (id, email, password) VALUES (1, '[email protected]', 'admin');
411+
```
412+
::
348413

349-
That's it! You can keep using `npx drizzle-kit generate` to generate migrations when updating your Drizzle ORM schema.
414+
::note
415+
These queries run after all migrations are applied but are not tracked in the `_hub_migrations` table. Use this for operations that should run when deploying your project.
416+
::

‎docs/nuxt.config.ts

+5
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,11 @@ export default defineNuxtConfig({
1717
devtools: {
1818
enabled: true
1919
},
20+
content: {
21+
highlight: {
22+
langs: ['sql']
23+
}
24+
},
2025
routeRules: {
2126
'/': { prerender: true },
2227
'/api/search.json': { prerender: true },

‎docs/pages/index.vue

+1-1
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ onMounted(() => {
9898
/>
9999
</UAvatarGroup>
100100
<span class="text-sm text-gray-500 dark:text-gray-400">
101-
Used and loved by <span class="font-medium dark:text-white text-gray-900">7K+ developers and teams</span>.
101+
Used and loved by <span class="font-medium dark:text-white text-gray-900">8K+ developers and teams</span>.
102102
</span>
103103
</div>
104104
<UDivider type="dashed" class="w-24" />

‎playground/layers/auth/nuxt.config.ts

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export default defineNuxtConfig({})

‎playground/layers/auth/server/database/migrations/0001_create-users.sql

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
CREATE TABLE IF NOT EXISTS users (
2+
id INTEGER PRIMARY KEY AUTOINCREMENT,
3+
email TEXT NOT NULL,
4+
password TEXT NOT NULL
5+
);

‎playground/modules/cms/db-migrations/0001_create-pages.sql

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
CREATE TABLE IF NOT EXISTS pages (
2+
id INTEGER PRIMARY KEY,
3+
title TEXT NOT NULL,
4+
content TEXT NOT NULL
5+
);

‎playground/modules/cms/index.ts

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
import { createResolver, defineNuxtModule } from 'nuxt/kit'
2+
3+
export default defineNuxtModule({
4+
meta: {
5+
name: 'my-auth-module'
6+
},
7+
setup(options, nuxt) {
8+
const { resolve } = createResolver(import.meta.url)
9+
10+
nuxt.hook('hub:database:migrations:dirs', (dirs) => {
11+
dirs.push(resolve('db-migrations'))
12+
})
13+
}
14+
})

‎playground/nuxt.config.ts

+11
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
// import { encodeHost } from 'ufo'
2+
import { createResolver } from 'nuxt/kit'
23
import module from '../src/module'
34

5+
const resolver = createResolver(import.meta.url)
6+
47
export default defineNuxtConfig({
58
modules: [
69
'@nuxt/ui',
@@ -55,6 +58,14 @@ export default defineNuxtConfig({
5558
}
5659
// projectUrl: ({ branch }) => branch === 'main' ? 'https://playground.nuxt.dev' : `https://${encodeHost(branch).replace(/\//g, '-')}.playground-to39.pages.dev`
5760
},
61+
hooks: {
62+
'hub:database:migrations:dirs': (dirs) => {
63+
dirs.push('my-module/database/migrations')
64+
},
65+
'hub:database:queries:paths': (queries) => {
66+
queries.push(resolver.resolve('server/database/queries/admin.sql'))
67+
}
68+
},
5869

5970
basicAuth: {
6071
enabled: process.env.NODE_ENV === 'production',

‎playground/server/database/queries/admin.sql

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
CREATE TABLE IF NOT EXISTS admin_users (
2+
id INTEGER PRIMARY KEY AUTOINCREMENT,
3+
email TEXT NOT NULL UNIQUE,
4+
password TEXT NOT NULL
5+
);
6+
INSERT OR IGNORE INTO admin_users (id, email, password) VALUES (1, '[email protected]', 'admin');

‎src/features.ts

+28-5
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import { execSync } from 'node:child_process'
22
import { pathToFileURL } from 'node:url'
3+
import { mkdir } from 'node:fs/promises'
34
import { isWindows } from 'std-env'
45
import type { Nuxt } from '@nuxt/schema'
56
import { join } from 'pathe'
@@ -9,6 +10,7 @@ import { defu } from 'defu'
910
import { $fetch } from 'ofetch'
1011
import { addDevToolsCustomTabs } from './utils/devtools'
1112
import { getCloudflareAccessHeaders } from './runtime/utils/cloudflareAccess'
13+
import { copyDatabaseMigrationsToHubDir, copyDatabaseQueriesToHubDir } from './runtime/database/server/utils/migrations/helpers'
1214

1315
const log = logger.withTag('nuxt:hub')
1416
const { resolve, resolvePath } = createResolver(import.meta.url)
@@ -57,11 +59,25 @@ export interface HubConfig {
5759
} & Record<string, boolean>
5860
}
5961

60-
migrationsPath?: string
62+
dir?: string
63+
databaseMigrationsDirs?: string[]
64+
databaseQueriesPaths?: string[]
6165
openAPIRoute?: string
6266
}
6367

64-
export function setupBase(nuxt: Nuxt, hub: HubConfig) {
68+
export async function setupBase(nuxt: Nuxt, hub: HubConfig) {
69+
// Create the hub.dir directory
70+
hub.dir = join(nuxt.options.rootDir, hub.dir!)
71+
try {
72+
await mkdir(hub.dir, { recursive: true })
73+
} catch (e: any) {
74+
if (e.errno === -17) {
75+
// File already exists
76+
} else {
77+
throw e
78+
}
79+
}
80+
6581
// Add Server scanning
6682
addServerScanDir(resolve('./runtime/base/server'))
6783
addServerImportsDir([resolve('./runtime/base/server/utils'), resolve('./runtime/base/server/utils/migrations')])
@@ -196,12 +212,19 @@ export async function setupCache(nuxt: Nuxt) {
196212
addServerScanDir(resolve('./runtime/cache/server'))
197213
}
198214

199-
export function setupDatabase(nuxt: Nuxt, hub: HubConfig) {
200-
// Keep track of the path to migrations
201-
hub.migrationsPath = join(nuxt.options.rootDir, 'server/database/migrations')
215+
export async function setupDatabase(nuxt: Nuxt, hub: HubConfig) {
202216
// Add Server scanning
203217
addServerScanDir(resolve('./runtime/database/server'))
204218
addServerImportsDir(resolve('./runtime/database/server/utils'))
219+
nuxt.hook('modules:done', async () => {
220+
// Call hub:database:migrations:dirs hook
221+
await nuxt.callHook('hub:database:migrations:dirs', hub.databaseMigrationsDirs!)
222+
// Copy all migrations files to the hub.dir directory
223+
await copyDatabaseMigrationsToHubDir(hub)
224+
// Call hub:database:migrations:queries hook
225+
await nuxt.callHook('hub:database:queries:paths', hub.databaseQueriesPaths!)
226+
await copyDatabaseQueriesToHubDir(hub)
227+
})
205228
}
206229

207230
export function setupKV(_nuxt: Nuxt) {

‎src/module.ts

+9-16
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { mkdir, writeFile, readFile } from 'node:fs/promises'
1+
import { writeFile, readFile } from 'node:fs/promises'
22
import { argv } from 'node:process'
33
import { defineNuxtModule, createResolver, logger, installModule, addServerHandler, addServerPlugin } from '@nuxt/kit'
44
import { join } from 'pathe'
@@ -38,6 +38,7 @@ export default defineNuxtModule<ModuleOptions>({
3838
let remoteArg = parseArgs(argv, { remote: { type: 'string' } }).remote as string
3939
remoteArg = (remoteArg === '' ? 'true' : remoteArg)
4040
const runtimeConfig = nuxt.options.runtimeConfig
41+
const databaseMigrationsDirs = nuxt.options._layers?.map(layer => join(layer.config.serverDir!, 'database/migrations')).filter(Boolean)
4142
const hub = defu(runtimeConfig.hub || {}, options, {
4243
// Self-hosted project
4344
projectUrl: process.env.NUXT_HUB_PROJECT_URL || '',
@@ -60,6 +61,9 @@ export default defineNuxtModule<ModuleOptions>({
6061
database: false,
6162
kv: false,
6263
vectorize: {},
64+
// Database Migrations
65+
databaseMigrationsDirs,
66+
databaseQueriesPaths: [],
6367
// Other options
6468
version,
6569
env: process.env.NUXT_HUB_ENV || 'production',
@@ -102,14 +106,14 @@ export default defineNuxtModule<ModuleOptions>({
102106
})
103107
}
104108

105-
setupBase(nuxt, hub as HubConfig)
109+
await setupBase(nuxt, hub as HubConfig)
106110
hub.openapi && setupOpenAPI(nuxt, hub as HubConfig)
107111
hub.ai && await setupAI(nuxt, hub as HubConfig)
108112
hub.analytics && setupAnalytics(nuxt)
109113
hub.blob && setupBlob(nuxt)
110114
hub.browser && await setupBrowser(nuxt)
111115
hub.cache && await setupCache(nuxt)
112-
hub.database && setupDatabase(nuxt, hub as HubConfig)
116+
hub.database && await setupDatabase(nuxt, hub as HubConfig)
113117
hub.kv && setupKV(nuxt)
114118
Object.keys(hub.vectorize!).length && setupVectorize(nuxt, hub as HubConfig)
115119

@@ -196,17 +200,6 @@ export default defineNuxtModule<ModuleOptions>({
196200
log.info(`Using local storage from \`${hub.dir}\``)
197201
}
198202

199-
// Create the hub.dir directory
200-
const hubDir = join(rootDir, hub.dir)
201-
try {
202-
await mkdir(hubDir, { recursive: true })
203-
} catch (e: any) {
204-
if (e.errno === -17) {
205-
// File already exists
206-
} else {
207-
throw e
208-
}
209-
}
210203
const workspaceDir = await findWorkspaceDir(rootDir)
211204
// Add it to .gitignore
212205
const gitignorePath = join(workspaceDir, '.gitignore')
@@ -219,11 +212,11 @@ export default defineNuxtModule<ModuleOptions>({
219212
// const needWrangler = Boolean(hub.analytics || hub.blob || hub.database || hub.kv || Object.keys(hub.bindings.hyperdrive).length > 0)
220213
if (needWrangler) {
221214
// Generate the wrangler.toml file
222-
const wranglerPath = join(hubDir, './wrangler.toml')
215+
const wranglerPath = join(hub.dir, './wrangler.toml')
223216
await writeFile(wranglerPath, generateWrangler(nuxt, hub as HubConfig), 'utf-8')
224217
// @ts-expect-error cloudflareDev is not typed here
225218
nuxt.options.nitro.cloudflareDev = {
226-
persistDir: hubDir,
219+
persistDir: hub.dir,
227220
configPath: wranglerPath,
228221
silent: true
229222
}

‎src/runtime/database/server/plugins/migrations.dev.ts

+6-4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
import { applyRemoteMigrations } from '../utils/migrations/remote'
1+
import { applyRemoteDatabaseMigrations, applyRemoteDatabaseQueries } from '../utils/migrations/remote'
22
import { hubHooks } from '../../../base/server/utils/hooks'
3-
import { applyMigrations } from '../utils/migrations/migrations'
3+
import { applyDatabaseMigrations, applyDatabaseQueries } from '../utils/migrations/migrations'
44
import { useRuntimeConfig, defineNitroPlugin } from '#imports'
55

66
export default defineNitroPlugin(async () => {
@@ -11,9 +11,11 @@ export default defineNitroPlugin(async () => {
1111

1212
hubHooks.hookOnce('bindings:ready', async () => {
1313
if (hub.remote && hub.projectKey) { // linked to a NuxtHub project
14-
await applyRemoteMigrations(hub)
14+
await applyRemoteDatabaseMigrations(hub)
15+
await applyRemoteDatabaseQueries(hub)
1516
} else { // local dev & self hosted
16-
await applyMigrations(hub)
17+
await applyDatabaseMigrations(hub)
18+
await applyDatabaseQueries(hub)
1719
}
1820

1921
await hubHooks.callHookParallel('database:migrations:done')

0 commit comments

Comments
 (0)