-
-
Notifications
You must be signed in to change notification settings - Fork 13
feat: add keyring state migration framework #505
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 56 commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
94a742a
feat: add keyring state migration framework to keyring-sdk
danroc 3361c30
docs: update keyring-sdk changelog for migration framework
danroc 091e618
refactor: use superstruct for VersionedState validation
danroc c12fe95
refactor: simplify migrations with entries() and getVersionAndData he…
danroc ad0c009
docs: simplify migration documentation
danroc 3d24a0b
feat: add migrated flag, inputSchema validation, and idempotent const…
danroc 1f544b7
feat: add defineMigrations for typed applyMigrations result
danroc 53fcb9f
docs: remove redundant API reference from migrations doc
danroc bfd9445
docs: remove duplicate defineMigration JSDoc
danroc 08461ac
fix: use integer() for versioned state, remove unnecessary cast in de…
danroc 1ff2a37
docs: simplify JSDoc for defineMigration parameters
danroc cd11da6
chore: make `migrations` a `const` to allow type inference
danroc 6920d01
chore: rename some tests
danroc be2b5cb
refactor: allow `validate` to be `undefined`
danroc 16c0d01
docs: simplify documentation
danroc 3aade3f
docs: improve keyring state migrations documentation
danroc 8cd7239
docs: clarify state persistence responsibility in migrations
danroc e217cc8
docs: clean up unused code in migrations example
danroc 872f1dd
chore: remove prettier config change
danroc f1f4f53
docs: remove redundant migrated flag from migrations example
danroc 9ccbf0a
docs: remove redundant migrated flag reference from best practices
danroc 03fb855
chore: remove changes to tsconfig
danroc e4eabba
chore: rename migrations.ts to migration.ts (migration engine)
danroc a63e0a8
chore: unwrap lines
danroc 0739302
refactor(keyring-sdk): remove redundant type params inferred from sch…
danroc 57e11c3
refactor(keyring-sdk): remove defineMigrations and use as const patte…
danroc 1d6d984
feat: use JsonStruct to validate inner state when inputSchema is not …
danroc d04146a
chore: simplify comment
danroc 2901740
chore: refine types of isVersionedState and getVersionAndData
danroc 2f754ad
refactor(keyring-sdk): make validate always defined with no-op fallback
danroc b849a10
refactor(keyring-sdk): simplify state assertion in defineMigration fu…
danroc e84fb6f
refactor(keyring-sdk): streamline migration validation call
danroc 2aa615e
chore: remove redundant comment
danroc 776ca1f
chore: clarify `data` vs `state`
danroc 200a55c
chore: simplify comment
danroc b23dc4e
chore: use reduce instead of array spread
danroc 3ab5aeb
chore(keyring-sdk): fix review issues in migration module
danroc de9280f
chore: update changelog
danroc 28a63ce
chore(keyring-sdk): fix import style lint error after rebase
danroc 26efec9
docs: update `getVersionAndData` JSDoc
danroc ae43235
refactor(keyring-sdk): replace array-based migrations with a chaining…
danroc f28b500
feat(keyring-sdk): validate migration step input/output against schemas
danroc 49ff874
fix(keyring-sdk): bind outputSchema/inputSchema to their step's gener…
danroc 70f9954
docs(keyring-sdk): rewrite migrations guide for the chaining builder API
danroc bc4d96b
docs(keyring-sdk): drop unused createMigrations import in Step 3 example
danroc 7cf2aaf
chore(keyring-sdk): update changelog for the migration builder API
danroc 5995cd5
chore(keyring-sdk): address final-review minor findings
danroc fd14097
docs(keyring-sdk): avoid em dashes and semicolons in comments and docs
danroc d33acea
chore: refactor buildChain function
danroc 89c3eac
chore: wrap JSDoc
danroc 6fcd3d4
chore(keyring-sdk): put Input before Output in migration type params
danroc 08f5251
Merge branch 'main' into dr/kering-migrations
danroc 87eec6e
chore: reorder inputSchema/outputSchema
danroc aade27a
chore: remove redundant text
danroc afe6d3c
chore: simplify example
danroc 6152ce5
test(keyring-sdk): move migration type checks to migration.test-d.ts
danroc 08ccf0d
fix(keyring-sdk): reject negative migration state versions
danroc 1c2113b
chore: update type test
danroc 688b35e
test(keyring-sdk): restore rejection check dropped by a merged sugges…
danroc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| # Keyring State Migrations | ||
|
|
||
| A framework for evolving keyring serialized state across versions. Migrations run during `deserialize()` and transform old state into the current format. | ||
|
|
||
| Versioned state is stored as an envelope: | ||
|
|
||
| ```json5 | ||
| { | ||
| "version": 1, | ||
| "data": { /* Keyring state */ } | ||
| } | ||
| ``` | ||
|
|
||
| Unversioned state (vaults created before migration support was added) has no envelope. The framework treats it as version 0 and applies all steps. | ||
|
|
||
| ## Key Concepts | ||
|
|
||
| - **`createMigrations()`**: Starts an empty migration chain. | ||
| - **`.add(step)`**: Appends a step to the chain and returns a new chain typed to that step's output. | ||
| - **`outputSchema`**: (Optional) Validates the **output** of a step at runtime. | ||
| - **`inputSchema`**: (Optional) Validates the **input** before the `migrate` function is called. | ||
| - **Positional versions**: The first `.add()` call produces version 1, the second version 2, and so on. | ||
|
|
||
| ## Example | ||
|
|
||
| ### 1. Define State Schemas | ||
|
|
||
| Define a schema for each version of your state. | ||
|
|
||
| ```typescript | ||
| import { object, array, number, string } from '@metamask/superstruct'; | ||
| import type { Infer } from '@metamask/superstruct'; | ||
|
|
||
| const HdStateV0Schema = object({ | ||
| numberOfAccounts: number(), // legacy field name | ||
| mnemonic: array(number()), | ||
| hdPath: string(), | ||
| }); | ||
|
|
||
| const HdStateV1Schema = object({ | ||
| accountCount: number(), // renamed from numberOfAccounts | ||
| mnemonic: array(number()), | ||
| hdPath: string(), | ||
| }); | ||
|
|
||
| const HdStateV2Schema = object({ | ||
| accountCount: number(), | ||
| mnemonic: array(number()), | ||
| hdPath: string(), | ||
| createdAt: number(), // new field | ||
| }); | ||
|
|
||
| type HdStateV0 = Infer<typeof HdStateV0Schema>; | ||
| type HdStateV1 = Infer<typeof HdStateV1Schema>; | ||
| type HdStateV2 = Infer<typeof HdStateV2Schema>; | ||
| ``` | ||
|
|
||
| ### 2. Define the Migration Chain | ||
|
|
||
| ```typescript | ||
| import { createMigrations } from '@metamask/keyring-sdk'; | ||
|
|
||
| const migrations = createMigrations() | ||
| .add({ | ||
| inputSchema: HdStateV0Schema, | ||
| outputSchema: HdStateV1Schema, | ||
| migrate: (data) => ({ | ||
| accountCount: data.numberOfAccounts, | ||
| mnemonic: data.mnemonic, | ||
| hdPath: data.hdPath, | ||
| }), | ||
| }) | ||
| .add({ | ||
| outputSchema: HdStateV2Schema, | ||
| migrate: (data) => ({ ...data, createdAt: Date.now() }), // data is typed as HdStateV1, no cast needed | ||
| }); | ||
| ``` | ||
|
|
||
| ### 3. Implement in your Keyring | ||
|
|
||
| ```typescript | ||
| import type { VersionedState } from '@metamask/keyring-sdk'; | ||
| import type { Json } from '@metamask/utils'; | ||
|
|
||
| class MyKeyring { | ||
| async deserialize(state: Json): Promise<void> { | ||
| const { data } = await migrations.apply(state); | ||
|
|
||
| // data is typed as HdStateV2 | ||
| this.#mnemonic = data.mnemonic; | ||
| this.#accountCount = data.accountCount; | ||
| this.#hdPath = data.hdPath; | ||
| } | ||
|
|
||
| async serialize(): Promise<VersionedState<HdStateV2>> { | ||
| return { | ||
| version: migrations.version, | ||
| data: { | ||
| mnemonic: this.#mnemonic, | ||
| accountCount: this.#accountCount, | ||
| hdPath: this.#hdPath, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Best Practices | ||
|
|
||
| - **Idempotent migrations**: Design steps so re-running them on already-migrated data is harmless. | ||
| - **Immutability**: Treat the input `data` as immutable within the `migrate` function. | ||
| - **Schema coverage**: Ensure `outputSchema` covers all fields expected in the new version to prevent runtime errors. | ||
| - **Non-mutating chains**: `.add()` returns a new chain rather than mutating the one it's called on, so it's safe to branch multiple chains off a shared base. | ||
|
|
||
| ## Constraints | ||
|
|
||
| - **Forward-only**: there is no downgrade path. Code that does not understand the versioned envelope will fail on migrated state. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| export * from './keyring-account-registry'; | ||
| export * from './migration'; | ||
| export * from './mnemonic'; | ||
| export * from './eth'; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| import { object, number, string } from '@metamask/superstruct'; | ||
| import type { Json } from '@metamask/utils'; | ||
| import { expectType } from 'tsd'; | ||
|
|
||
| import { createMigrations } from './migration'; | ||
| import type { MigrationChain, MigrationResult } from './migration'; | ||
|
|
||
| // `createMigrations()` starts an empty chain typed to accept `Json`. | ||
| expectType<MigrationChain<Json>>(createMigrations()); | ||
|
|
||
| // A step's `migrate` receives the previous step's output shape, with no cast needed. | ||
| createMigrations() | ||
| .add({ migrate: (): { count: number } => ({ count: 1 }) }) | ||
| .add({ | ||
| migrate: (data) => { | ||
| expectType<number>(data.count); | ||
| return data; | ||
| }, | ||
| }); | ||
|
|
||
| // A step's `migrate` must accept the previous step's output shape. | ||
| createMigrations() | ||
| .add({ migrate: (): { count: number } => ({ count: 1 }) }) | ||
| // @ts-expect-error [test] `data` is `{ count: number }`, not `{ label: string }`. | ||
| .add({ migrate: (data: { label: string }) => data.label }); | ||
|
danroc marked this conversation as resolved.
|
||
|
|
||
| // `inputSchema` narrows `migrate`'s input to the schema's inferred type, with no cast | ||
| // needed. | ||
| createMigrations().add({ | ||
| inputSchema: object({ oldCount: number() }), | ||
| migrate: (data) => { | ||
| expectType<number>(data.oldCount); | ||
| return { count: data.oldCount }; | ||
| }, | ||
| }); | ||
|
|
||
| // `inputSchema` must be compatible with the chain's current data type, just like | ||
| // `migrate`. | ||
| createMigrations() | ||
| .add({ migrate: (): { count: number } => ({ count: 1 }) }) | ||
| .add({ | ||
| // @ts-expect-error [test] `inputSchema`'s inferred type doesn't extend `{ count: number }`. | ||
| inputSchema: object({ label: string() }), | ||
| migrate: () => 'x', | ||
| }); | ||
|
|
||
| // `apply()` resolves to the final step's output type. | ||
| const migrationsForApply = createMigrations().add({ | ||
| migrate: (): { count: number } => ({ count: 1 }), | ||
| }); | ||
| expectType<Promise<MigrationResult<{ count: number }>>>( | ||
| migrationsForApply.apply({}), | ||
| ); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.