Skip to content

fix: optional key configuration push for npm run migrate #1471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,19 @@ npm run watch

instead, which will monitor for changes with `nodemon`.

## Initialisation
## Initialisation and migration

Once the services are up and running we need to initialise the CouchDB
database. This is done with the migrate script:
database.

If you want to include pushing the keys (only recommended for local development):

```bash
npm run migrate --keys
```

If you just want to migrate databases without applying any public key
configuration, then exclude this flag

```bash
npm run migrate
Expand All @@ -82,7 +91,7 @@ run in the container rather than from your local environment:
docker compose exec conductor npm run migrate
```

This ensures that the correct CouchDB URL is used to access the database. The same
This ensures that the correct CouchDB URL is used to access the database. The same
applies for the commands below.

For development, there is also a script that will populate the database with projects (notebooks
Expand Down
42 changes: 29 additions & 13 deletions api/src/couchdb/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -449,22 +449,31 @@ export const initialiseDataDb = async ({
};

/**
* Critical method which initialises all databases, including remotely on the configured couch instance.
* Critical method which initialises all databases, including remotely on the
* configured couch instance.
*
* This systematically generates a set of initialisation content from the data model, then applies this initialisation using a helper method in the data model.
* This systematically generates a set of initialisation content from the data
* model, then applies this initialisation using a helper method in the data
* model.
*
* Some local information is injected as part of the config generation step - e.g. conductor name/description.
* Some local information is injected as part of the config generation step -
* e.g. conductor name/description.
*
* Also initialises keys based on the configured key service.
*
* If force = true, documents will always be written, even if it already exists.
*
* If pushKeys = true, will update the public keys
*
* @param force Write on clash
*/
export const initialiseDbAndKeys = async ({
force = false,
pushKeys = true,
}: {
force?: boolean;
// Should we push the key configuration?
pushKeys?: boolean;
}) => {
// Are we in a testing environment?
const isTesting = process.env.NODE_ENV === 'test';
Expand Down Expand Up @@ -622,15 +631,19 @@ export const initialiseDbAndKeys = async ({
await initialiseDataDb({projectId, force});
}

// Setup keys
try {
await initialiseJWTKey();
} catch (error) {
console.log(
'something wrong PUTing jwt_keys into the db configuration...',
error
);
throw error;
if (pushKeys) {
// Setup keys
try {
await initialiseJWTKey();
} catch (error) {
console.log(
'something wrong PUTing jwt_keys into the db configuration...',
error
);
throw error;
}
} else {
console.log('Not pushing key configuration.');
}
};

Expand All @@ -639,10 +652,13 @@ export const initialiseDbAndKeys = async ({
*/
export const initialiseAndMigrateDBs = async ({
force = false,
pushKeys = true,
}: {
force?: boolean;
// Should we push the key configuration?
pushKeys?: boolean;
}) => {
await initialiseDbAndKeys({force});
await initialiseDbAndKeys({force, pushKeys});

let dbs: {dbType: DATABASE_TYPE; dbName: string; db: PouchDB.Database}[] = [
{db: getAuthDB(), dbType: DatabaseType.AUTH, dbName: AUTH_DB_NAME},
Expand Down
20 changes: 19 additions & 1 deletion api/src/scripts/migrate.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,26 @@
/* eslint-disable n/no-process-exit */
import {initialiseAndMigrateDBs} from '../couchdb';

/**
* Main function to run database initialization and migration
* Accepts optional --keys flag to control whether public keys should be pushed
*/
const main = async () => {
try {
await initialiseAndMigrateDBs({force: true});
// Check if --keys flag is present in command line arguments
const pushKeys = process.argv.includes('--keys');

// Log whether keys will be configured
console.log(
`Public keys will ${pushKeys ? '' : 'not '}be configured during migration`
);

// Run database initialization and migration with force and pushKeys parameters
await initialiseAndMigrateDBs({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only called in one place and force is set to true, is there any need to retain this any more? Do we ever want to not overwrite old design documents now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's defer this as out of scope for this fix - but good point

https://jira.csiro.au/browse/BSS-946

force: true,
pushKeys: pushKeys,
});

console.log('Migration completed successfully');
process.exit(0);
} catch (error) {
Expand All @@ -12,4 +29,5 @@ const main = async () => {
}
};

// Execute the main function
main();
4 changes: 2 additions & 2 deletions localdev.sh
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,8 @@ else
fi

echo "Initialising database using API container"
echo ">docker compose exec api sh -c \"cd api && npm run migrate\""
docker compose exec api sh -c "cd api && npm run migrate"
echo ">docker compose exec api sh -c \"cd api && npm run migrate --keys\""
docker compose exec api sh -c "cd api && npm run migrate --keys"


echo "Service is setup, to load notebooks and templates follow the below steps"
Expand Down
Loading