Skip to content

Commit e215162

Browse files
edenoedeno
committed
Merge phase-7-persistence-migration: versioned forward-migration framework (Post-v3 #6)
The persistence layer now upgrades old localStorage blobs FORWARD through an ordered registry of pure migrators (src/state/workspaceMigrations.js) instead of discarding them on a schemaVersion it doesn't recognize. Mechanism only — no persisted-shape change, no version bump: today's v1→v2 is the first (identity) migrator, MIGRATABLE_SCHEMA_VERSIONS is derived from the registry, and migrateWorkspace runs before normalize/shape-ensure so downstream only sees the current shape. Checked-in v1/v2 blob fixtures prove a v1 blob hydrates identically to the equivalent v2 (no discard, no data loss) — the regression guard every future migrator inherits. Release-gating for Phases 8 and 10. Existing persistence tests pass unchanged; full vitest (4583), persistence/recovery e2e, typecheck, and lint green; baselines unaffected.
2 parents ee334b3 + 656e46f commit e215162

9 files changed

Lines changed: 467 additions & 36 deletions

File tree

‎.claude/docs/plans/design-feedback-remediation/PLAN.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Design-Feedback Remediation Implementation Plan
22

3-
**Status:** Phases 1–6 complete (merged into `modern`); Phases 7–10 not started.
3+
**Status:** Phases 1–7 complete (merged into `modern`); Phases 8–10 not started.
44

55
This plan turns six pieces of user design feedback plus a re-evaluated backlog into shippable work:
66
it guarantees recording days stay date-ordered, restores guided DIO entry (a Type + Index control),
@@ -36,7 +36,7 @@ For agent invocation, **load only the slice you need**:
3636
- [phase-4-remove-channel-maps-editor.md](phase-4-remove-channel-maps-editor.md) — delete the workspace channel-maps editor (F1), full reference sweep. ✅ **Complete (merged)**.
3737
- [phase-5-dayeditor-structural-prep.md](phase-5-dayeditor-structural-prep.md) — `DayEditorContext` (kill prop-drill) + shared `deviceOverrideMerge` module; behavior-preserving. ✅ **Complete (merged)**.
3838
- [phase-6-tasks-epochs-redesign.md](phase-6-tasks-epochs-redesign.md) — Tasks & Epochs in-place clarity redesign (F4 quick wins) + a11y. ✅ **Complete (merged)**.
39-
- [phase-7-persistence-migration.md](phase-7-persistence-migration.md) — versioned persisted-blob forward migration (Post-v3 #6).
39+
- [phase-7-persistence-migration.md](phase-7-persistence-migration.md) — versioned persisted-blob forward migration (Post-v3 #6). ✅ **Complete (merged)**.
4040
- [phase-8-task-type-catalog.md](phase-8-task-type-catalog.md) — animal-level task-type catalog; days pick/order epochs (F4 full) + trodes_to_nwb integration check.
4141
- [phase-9-architecture-cleanup.md](phase-9-architecture-cleanup.md) — split `validation.js`; decompose `ValidationSummary`/`RecordingDaysTab`; continue TS; **re-arm the build gate**.
4242
- [phase-10-post-v3-ux.md](phase-10-post-v3-ux.md) — device_type summaries, reconfig wizard, validated indicator, calendar a11y, appliedToDays→derived.

‎CHANGELOG.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
116116

117117
### Changed
118118

119+
- **Versioned forward-migration for the persisted workspace (mechanism only; no behavior change
120+
yet).** The persistence layer previously *discarded* a localStorage blob whose `schemaVersion` it
121+
didn't recognize. It now upgrades old blobs **forward** through an ordered registry of pure
122+
migrators (`src/state/workspaceMigrations.js`, `migrators[n]: vN → vN+1`), applied by
123+
`migrateWorkspace` in `loadWorkspace` before device-normalization and shape-ensuring, so a future
124+
persisted-shape change won't throw away saved work. Today's v1→v2 handling is encoded as the first
125+
registered migrator (current blobs hydrate byte-for-byte as before), and `MIGRATABLE_SCHEMA_VERSIONS`
126+
is now derived from the registry instead of hand-maintained. No persisted-shape change or version
127+
bump ships in this entry — the mechanism lands ahead of the first real shape change, which will
128+
register its own `vN→vN+1` migrator plus a checked-in `vN` blob fixture. (Resolves the release-gated
129+
persistence-migration follow-up.)
119130
- **Tasks & Epochs screen — clarity redesign (no data-model or export change).** The day editor's
120131
most-complex screen now opens with a plain-language framing that defines a *task* (one activity
121132
in one environment, with its cameras) and an *epoch* (a numbered time block of that task, each

‎CLAUDE.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -543,6 +543,21 @@ The application includes comprehensive optogenetics configuration:
543543
- All `<details>` elements are opened before form submission for validation visibility
544544
- Production detection via `isProduction()` adjusts navigation links for GitHub Pages deployment
545545
546+
### Persistence & schema migration
547+
548+
The workspace slice is persisted to localStorage as `{ schemaVersion, workspace }`
549+
([src/state/persistence.js](src/state/persistence.js)). Old blobs are upgraded **forward** by an
550+
ordered registry of pure migrators in [src/state/workspaceMigrations.js](src/state/workspaceMigrations.js)
551+
(`migrators[n]: vN → vN+1`, applied by `migrateWorkspace` before device-normalize / shape-ensure),
552+
not discarded. `MIGRATABLE_SCHEMA_VERSIONS` is **derived** from the registry — never hand-edited.
553+
554+
**Rule — bump `WORKSPACE_SCHEMA_VERSION` ONLY together with:** (1) a registered migrator from the
555+
previous version, and (2) a checked-in `vN` blob fixture
556+
([src/state/__tests__/fixtures/persistence/](src/state/__tests__/fixtures/persistence/)) whose test
557+
proves `load(vN-blob)` hydrates to the current shape with no discard and no data loss. Migrators are
558+
total and non-destructive (a field one can't map forward is preserved or surfaced via the
559+
`recovered`/`discarded` notice path, never silently dropped).
560+
546561
## Key Patterns
547562
548563
- **Immutable Updates:** All state updates use `structuredClone()` to avoid mutation

‎docs/POST_V3_FOLLOWUPS.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -70,12 +70,14 @@ out of scope for the cleanup pass and are tracked here.
7070

7171
## Release-gated
7272

73-
6. **Persistence-blob forward migration.** The persistence layer versions the localStorage
74-
blob and *discards with a notice* on `schemaVersion` mismatch — no migration. Acceptable for v3.0.0
75-
(no real v1 blobs exist yet), but once users have v1 blobs a future shape change would silently
76-
discard their saved work. Schedule a forward-migration path (transform old blobs forward instead of
77-
discarding) **before the first post-v3.0.0 change that touches the persisted shape** — including
78-
tech-debt item #1 above, which changes that shape.
73+
6. **Persistence-blob forward migration.** ✅ **RESOLVED.** A versioned forward-migration
74+
framework now lives in `src/state/workspaceMigrations.js`: an ordered registry of pure migrators
75+
(`n: vN → vN+1`) and `migrateWorkspace`, which `loadWorkspace` runs (before device-normalize /
76+
shape-ensure) to upgrade an old blob forward instead of discarding it. Today's v1→v2 behavior is
77+
encoded as the first registered migrator; `MIGRATABLE_SCHEMA_VERSIONS` is derived from the
78+
registry. The mechanism is in place with no shape change yet — the first real shape change (the
79+
task catalog; the `appliedToDays` derivation) registers its `vN→vN+1` migrator + a `vN` fixture.
80+
Rule: bump `WORKSPACE_SCHEMA_VERSION` only together with a registered migrator and a fixture test.
7981

8082
## From the pre-cutover UX audits (audit docs since removed)
8183

‎src/state/__tests__/fixtures/persistence/v1-workspace.json‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
{
2+
"schemaVersion": 1,
3+
"workspace": {
4+
"version": "1.0.0",
5+
"lastModified": "2023-06-22T12:00:00.000Z",
6+
"animals": {
7+
"remy": {
8+
"id": "remy",
9+
"subject": {
10+
"description": "Long Evans Rat from Charles River",
11+
"genotype": "Wild Type",
12+
"species": "Rattus norvegicus",
13+
"sex": "M",
14+
"subject_id": "remy",
15+
"weight": 485,
16+
"date_of_birth": "2023-01-10T00:00:00"
17+
},
18+
"devices": {
19+
"data_acq_device": [
20+
{ "name": "SpikeGadgets", "system": "SpikeGadgets", "amplifier": "Intan", "adc_circuit": "Intan" }
21+
],
22+
"device": { "name": ["Trodes"] },
23+
"electrode_groups": [
24+
{ "id": 0, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 1", "targeted_location": "CA1", "targeted_x": 3, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" },
25+
{ "id": 1, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 2", "targeted_location": "CA1", "targeted_x": 3.1, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" }
26+
],
27+
"ntrode_electrode_group_channel_map": [
28+
{ "ntrode_id": 1, "electrode_group_id": 0, "bad_channels": [], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } },
29+
{ "ntrode_id": 2, "electrode_group_id": 1, "bad_channels": [2], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } }
30+
]
31+
},
32+
"cameras": [
33+
{ "id": 0, "meters_per_pixel": 0.00085, "manufacturer": "Allied Vision", "model": "Mako G-158", "lens": "Fujinon HF16HA-1B", "camera_name": "overhead_camera" }
34+
],
35+
"experimenters": {
36+
"experimenter_name": ["Guidera, Jennifer"],
37+
"lab": "Frank",
38+
"institution": "University of California, San Francisco"
39+
},
40+
"optogenetics": null,
41+
"behavioral_events": [],
42+
"days": ["remy-2023-06-22"],
43+
"created": "2023-06-22T12:00:00.000Z",
44+
"lastModified": "2023-06-22T12:00:00.000Z",
45+
"configurationHistory": [
46+
{
47+
"version": 1,
48+
"date": "2023-06-22",
49+
"description": "Initial configuration",
50+
"devices": {
51+
"electrode_groups": [
52+
{ "id": 0, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 1", "targeted_location": "CA1", "targeted_x": 3, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" },
53+
{ "id": 1, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 2", "targeted_location": "CA1", "targeted_x": 3.1, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" }
54+
],
55+
"ntrode_electrode_group_channel_map": [
56+
{ "ntrode_id": 1, "electrode_group_id": 0, "bad_channels": [], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } },
57+
{ "ntrode_id": 2, "electrode_group_id": 1, "bad_channels": [2], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } }
58+
]
59+
},
60+
"appliedToDays": []
61+
}
62+
]
63+
}
64+
},
65+
"days": {
66+
"remy-2023-06-22": {
67+
"id": "remy-2023-06-22",
68+
"animalId": "remy",
69+
"date": "2023-06-22",
70+
"experimentDate": "06222023",
71+
"session": {
72+
"session_id": "remy_20230622",
73+
"session_description": "Day 45 of chronic recording, W-track alternation",
74+
"experiment_description": "Chronic tetrode recording during spatial navigation"
75+
},
76+
"tasks": [
77+
{ "task_name": "sleep", "task_description": "Rest in home cage", "task_environment": "home cage", "camera_id": [0], "task_epochs": [1] }
78+
],
79+
"behavioral_events": [
80+
{ "description": "Reward delivery at left arm", "name": "reward_left" }
81+
],
82+
"associated_files": [],
83+
"associated_video_files": [],
84+
"technical": {
85+
"times_period_multiplier": 1.5,
86+
"raw_data_to_volts": 0.195,
87+
"default_header_file_path": ""
88+
},
89+
"state": { "draft": true, "validated": false, "exported": false },
90+
"created": "2023-06-22T12:00:00.000Z",
91+
"lastModified": "2023-06-22T12:00:00.000Z",
92+
"configurationVersion": 1
93+
}
94+
},
95+
"settings": {
96+
"defaultLab": "Frank",
97+
"defaultInstitution": "University of California, San Francisco",
98+
"defaultExperimenters": ["Guidera, Jennifer"],
99+
"autoSaveInterval": 30000,
100+
"shadowExportEnabled": true
101+
}
102+
}
103+
}

‎src/state/__tests__/fixtures/persistence/v2-workspace.json‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
{
2+
"schemaVersion": 2,
3+
"workspace": {
4+
"version": "1.0.0",
5+
"lastModified": "2023-06-22T12:00:00.000Z",
6+
"animals": {
7+
"remy": {
8+
"id": "remy",
9+
"subject": {
10+
"description": "Long Evans Rat from Charles River",
11+
"genotype": "Wild Type",
12+
"species": "Rattus norvegicus",
13+
"sex": "M",
14+
"subject_id": "remy",
15+
"weight": 485,
16+
"date_of_birth": "2023-01-10T00:00:00"
17+
},
18+
"devices": {
19+
"data_acq_device": [
20+
{ "name": "SpikeGadgets", "system": "SpikeGadgets", "amplifier": "Intan", "adc_circuit": "Intan" }
21+
],
22+
"device": { "name": ["Trodes"] },
23+
"electrode_groups": [
24+
{ "id": 0, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 1", "targeted_location": "CA1", "targeted_x": 3, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" },
25+
{ "id": 1, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 2", "targeted_location": "CA1", "targeted_x": 3.1, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" }
26+
],
27+
"ntrode_electrode_group_channel_map": [
28+
{ "ntrode_id": 1, "electrode_group_id": 0, "bad_channels": [], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } },
29+
{ "ntrode_id": 2, "electrode_group_id": 1, "bad_channels": [2], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } }
30+
]
31+
},
32+
"cameras": [
33+
{ "id": 0, "meters_per_pixel": 0.00085, "manufacturer": "Allied Vision", "model": "Mako G-158", "lens": "Fujinon HF16HA-1B", "camera_name": "overhead_camera" }
34+
],
35+
"experimenters": {
36+
"experimenter_name": ["Guidera, Jennifer"],
37+
"lab": "Frank",
38+
"institution": "University of California, San Francisco"
39+
},
40+
"optogenetics": null,
41+
"behavioral_events": [],
42+
"days": ["remy-2023-06-22"],
43+
"created": "2023-06-22T12:00:00.000Z",
44+
"lastModified": "2023-06-22T12:00:00.000Z",
45+
"configurationHistory": [
46+
{
47+
"version": 1,
48+
"date": "2023-06-22",
49+
"description": "Initial configuration",
50+
"devices": {
51+
"electrode_groups": [
52+
{ "id": 0, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 1", "targeted_location": "CA1", "targeted_x": 3, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" },
53+
{ "id": 1, "location": "CA1", "device_type": "tetrode_12.5", "description": "CA1 tetrode 2", "targeted_location": "CA1", "targeted_x": 3.1, "targeted_y": 2.5, "targeted_z": 2, "units": "mm" }
54+
],
55+
"ntrode_electrode_group_channel_map": [
56+
{ "ntrode_id": 1, "electrode_group_id": 0, "bad_channels": [], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } },
57+
{ "ntrode_id": 2, "electrode_group_id": 1, "bad_channels": [2], "map": { "0": 0, "1": 1, "2": 2, "3": 3 } }
58+
]
59+
},
60+
"appliedToDays": []
61+
}
62+
]
63+
}
64+
},
65+
"days": {
66+
"remy-2023-06-22": {
67+
"id": "remy-2023-06-22",
68+
"animalId": "remy",
69+
"date": "2023-06-22",
70+
"experimentDate": "06222023",
71+
"session": {
72+
"session_id": "remy_20230622",
73+
"session_description": "Day 45 of chronic recording, W-track alternation",
74+
"experiment_description": "Chronic tetrode recording during spatial navigation"
75+
},
76+
"tasks": [
77+
{ "task_name": "sleep", "task_description": "Rest in home cage", "task_environment": "home cage", "camera_id": [0], "task_epochs": [1] }
78+
],
79+
"behavioral_events": [
80+
{ "description": "Reward delivery at left arm", "name": "reward_left" }
81+
],
82+
"associated_files": [],
83+
"associated_video_files": [],
84+
"technical": {
85+
"times_period_multiplier": 1.5,
86+
"raw_data_to_volts": 0.195,
87+
"default_header_file_path": ""
88+
},
89+
"state": { "draft": true, "validated": false, "exported": false },
90+
"created": "2023-06-22T12:00:00.000Z",
91+
"lastModified": "2023-06-22T12:00:00.000Z",
92+
"configurationVersion": 1
93+
}
94+
},
95+
"settings": {
96+
"defaultLab": "Frank",
97+
"defaultInstitution": "University of California, San Francisco",
98+
"defaultExperimenters": ["Guidera, Jennifer"],
99+
"autoSaveInterval": 30000,
100+
"shadowExportEnabled": true
101+
}
102+
}
103+
}

0 commit comments

Comments
 (0)