Skip to content

Commit d360fcb

Browse files
claudeclaude
committed
docs: add session recording schema reference
The exported TypeScript types in `recording.ts` were the only reference for anyone writing a replayer against `getSessionRecording()`. This adds a markdown reference under docs/reference/session-recording-schema.md covering: - The replayer model (DOM-canonical, no code re-execution) - Top-level shape with field-by-field documentation - Per-trial structure and the procedure for replaying a single trial - The DOM representation (node ids, element extras, what is and isn't captured) - Each event type in the discriminated union (DOM mutations, input, clipboard, media, focus, scroll), with field-level meaning - Viewport state and how to resolve "what was the viewport at time t?" - The session-level rng_calls log and how to consume it on replay - End reasons - Privacy notes (verbatim text capture, clipboard payloads, Math.random patching) - Backward compatibility expectations (additive minor changes ignore policy for replayers) Wires the new page into mkdocs.yml under Reference and cross-links it from the `getSessionRecording` reference.
1 parent 45be441 commit d360fcb

3 files changed

Lines changed: 290 additions & 1 deletion

File tree

‎docs/reference/jspsych.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -479,7 +479,7 @@ Returns a `SessionRecording` object when the experiment was initialized with `re
479479

480480
### Description
481481

482-
Returns the high-fidelity session recording produced by the `record_session` option. The recording includes a snapshot of the trial parameters and rendered DOM at the start of every trial, all DOM mutations within `#jspsych-content`, mouse, touch, keyboard, and clipboard events, video and audio playback events, viewport changes, and every `Math.random()` output. The returned object contains the schema version (`schema_version: 1`); the on-disk format is the contract between recorder and any replayer.
482+
Returns the high-fidelity session recording produced by the `record_session` option. The recording includes the rendered DOM at the start of every trial, all DOM mutations within `#jspsych-content`, mouse, touch, keyboard, and clipboard events, scroll position (window and per-element), video and audio playback events, viewport changes, and every `Math.random()` output. The returned object is versioned (`schema_version: 1`); see [Session Recording Schema](./session-recording-schema.md) for the full format reference.
483483

484484
### Example
485485

‎docs/reference/session-recording-schema.md‎

Lines changed: 288 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,288 @@
1+
# Session Recording Schema
2+
3+
When `initJsPsych` is called with `record_session: true`, jsPsych captures a session recording sufficient to reconstruct a visual replay of the participant's experience. This page documents the JSON-serializable shape returned by `jsPsych.getSessionRecording()`.
4+
5+
The schema is versioned. Schemas with the same major version are read-compatible; consumers should branch on `schema_version` and reject anything they don't understand.
6+
7+
**Current version: `1`.**
8+
9+
## Replayer model
10+
11+
A replayer treats this recording as observational. It does not re-execute trial code. The replay is reconstructed from three things:
12+
13+
1. The DOM snapshot captured at each trial's `on_load` (`trial.initial_dom`).
14+
2. The chronological mutation and input event log scoped to that trial (`trial.events`).
15+
3. Session-level metadata for context (viewport, scroll, RNG outputs, focus/blur, fullscreen).
16+
17+
Each trial is a self-contained replay unit because jsPsych wipes the display element between trials.
18+
19+
## Top-level shape
20+
21+
```ts
22+
interface SessionRecording {
23+
schema_version: 1;
24+
jspsych_version: string;
25+
recording_started_at: string; // ISO 8601 wall-clock anchor
26+
recording_started_at_perf: number; // performance.now() at start
27+
user_agent: string;
28+
viewport: ViewportState;
29+
rng: { seed: string | null; math_random_patched: boolean };
30+
display_element_id: string;
31+
trials: TrialRecording[];
32+
viewport_changes: ViewportChange[];
33+
rng_calls: RngCall[];
34+
ended_at_perf: number | null;
35+
end_reason: "finished" | "aborted" | "unload" | null;
36+
}
37+
```
38+
39+
| Field | Description |
40+
| ----- | ----------- |
41+
| `schema_version` | Always `1` for recordings produced by this codebase. |
42+
| `jspsych_version` | The jsPsych package version that produced the recording. |
43+
| `recording_started_at` | ISO 8601 timestamp of when `start()` was called. |
44+
| `recording_started_at_perf` | The `performance.now()` value at recording start. All event `t` values are relative to this. |
45+
| `user_agent` | `navigator.userAgent` at recording start. |
46+
| `viewport` | The initial viewport state. See [`ViewportState`](#viewportstate). |
47+
| `rng.seed` | The seed installed via `jsPsych.randomization.setSeed` for the session, or `null` if `Math.random` was already non-native at recording start. |
48+
| `rng.math_random_patched` | `true` while recording is active; `Math.random` is wrapped to log every call into `rng_calls`. |
49+
| `display_element_id` | The `id` attribute of the display element (`#jspsych-content` by default). |
50+
| `trials` | Per-trial recordings, in chronological order. See [`TrialRecording`](#trialrecording). |
51+
| `viewport_changes` | Session-level log of viewport changes (window resize, page zoom, pinch zoom, pinch pan). |
52+
| `rng_calls` | Chronological log of every `Math.random` output. Includes calls outside trial boundaries (parameter eval, ITI, `on_finish`). |
53+
| `ended_at_perf` | The `performance.now()` at `stop()`. |
54+
| `end_reason` | How the session ended. `"aborted"` when `abortExperiment()` was called. |
55+
56+
All `t` fields elsewhere in the document are floats in milliseconds, relative to `recording_started_at_perf` (i.e. `performance.now() - recording_started_at_perf`).
57+
58+
## TrialRecording
59+
60+
```ts
61+
interface TrialRecording {
62+
trial_index: number;
63+
t_start: number;
64+
t_dom_ready: number | null;
65+
t_end: number | null;
66+
plugin: string;
67+
initial_dom: DomNode | null;
68+
events: RecordedEvent[];
69+
trial_data: JsonValue;
70+
}
71+
```
72+
73+
| Field | Description |
74+
| ----- | ----------- |
75+
| `trial_index` | The jsPsych trial index. Matches `trial_data.trial_index` and the row in `data.csv`. |
76+
| `t_start` | When the trial entered `onTrialStart` (before parameter evaluation completes). |
77+
| `t_dom_ready` | When the plugin's initial render completed (`on_load` fired). `initial_dom` was sampled at this moment. `null` if a recording was stopped before this point. |
78+
| `t_end` | When the trial ended. `null` if the recording was stopped before the trial finished. |
79+
| `plugin` | The plugin name (e.g. `"html-keyboard-response"`). For labeling and filtering only; replay does not depend on it. |
80+
| `initial_dom` | The DOM subtree of the display element at `t_dom_ready`. See [DOM representation](#dom-representation). |
81+
| `events` | Chronological log of mutations and input events from `t_dom_ready` to `t_end`. Empty arrays are valid. |
82+
| `trial_data` | The data row written to `jsPsych.data` for this trial (the same object exported via `data.csv` / `data.json`). Includes `rt`, `response`, `trial_type`, `trial_index`, etc. |
83+
84+
### Replay procedure for a single trial
85+
86+
1. Clear the display element.
87+
2. Instantiate `initial_dom` into the display element.
88+
3. Apply each entry in `events` at its `t`, in order.
89+
90+
## DOM representation
91+
92+
```ts
93+
type DomNode = ElementNode | TextNode | CommentNode;
94+
95+
interface ElementNode {
96+
id: number;
97+
kind: "element";
98+
tag: string; // lowercased tag name
99+
attrs: Record<string, string>;
100+
children: DomNode[];
101+
canvas_size?: { w: number; h: number };
102+
media_src?: string;
103+
}
104+
105+
interface TextNode { id: number; kind: "text"; text: string; }
106+
interface CommentNode { id: number; kind: "comment"; text: string; }
107+
```
108+
109+
Every node in `initial_dom` is assigned a monotonically-increasing integer `id`. Mutation events reference these ids. New nodes added later (via `dom.add`) carry their own id and may have child nodes that recursively carry ids of their own.
110+
111+
**Per-trial scope.** Node ids are reset at every `t_dom_ready`. Ids in `trials[i]` have no relationship to ids in `trials[j]`.
112+
113+
**Element-specific extras.**
114+
115+
- `canvas_size`: present on `<canvas>` elements; carries `width`/`height` attributes at snapshot time. Note that the *contents* of a canvas (the rendered pixels) are not captured.
116+
- `media_src`: present on `<video>` and `<audio>` elements; carries `currentSrc` if available, falling back to the `src` attribute.
117+
118+
**What is not captured.**
119+
120+
- Canvas/WebGL pixel content. Only the element and its dimensions are recorded.
121+
- Audio/video media data. Only playback events (`media.play`, `media.pause`, etc.) and the source URL are recorded.
122+
- Shadow DOM. jsPsych does not use it in core; recordings will not capture mutations inside shadow roots.
123+
- Styles applied via `<link rel="stylesheet">` referencing external sheets. The link element itself is recorded; the resolved CSS is not. Replayers generally need to load the same stylesheets out-of-band.
124+
125+
## Event types
126+
127+
`events` and `viewport_changes` are arrays of one of the following discriminated unions.
128+
129+
```ts
130+
type RecordedEvent =
131+
| DomMutation
132+
| InputRecord
133+
| ClipboardRecord
134+
| MediaRecord
135+
| FocusRecord
136+
| ScrollRecord;
137+
```
138+
139+
### DOM mutations
140+
141+
```ts
142+
type DomMutation =
143+
| { type: "dom.add"; t: number; parent: number; before: number | null; node: DomNode }
144+
| { type: "dom.remove"; t: number; node: number }
145+
| { type: "dom.attr"; t: number; node: number; name: string; value: string | null }
146+
| { type: "dom.text"; t: number; node: number; text: string };
147+
```
148+
149+
| Type | Field | Meaning |
150+
| ---- | ----- | ------- |
151+
| `dom.add` | `parent` | Id of the parent element. |
152+
| | `before` | Id of the next sibling, or `null` to append at end. |
153+
| | `node` | Full subtree of the added node, with ids assigned. |
154+
| `dom.remove` | `node` | Id of the removed node. The node's children are also released. |
155+
| `dom.attr` | `node` | Id of the element. |
156+
| | `name` | Attribute name. |
157+
| | `value` | New value, or `null` if the attribute was removed. |
158+
| `dom.text` | `node` | Id of the text/comment node. |
159+
| | `text` | New text content. |
160+
161+
### Input events
162+
163+
```ts
164+
type InputRecord =
165+
| { type: "mouse.move"; t: number; x: number; y: number }
166+
| {
167+
type: "mouse.down" | "mouse.up" | "mouse.click";
168+
t: number; x: number; y: number; button: number; target: number | null;
169+
}
170+
| {
171+
type: "touch.start" | "touch.move" | "touch.end";
172+
t: number; touches: { id: number; x: number; y: number }[];
173+
}
174+
| {
175+
type: "key.down" | "key.up";
176+
t: number; key: string; code: string;
177+
mods: { ctrl: boolean; shift: boolean; alt: boolean; meta: boolean };
178+
repeat: boolean; target: number | null;
179+
};
180+
```
181+
182+
`mouse.move` is throttled to one record per animation frame, carrying the latest position. `target` is the id of the element under the event target if it is a tracked node within the trial's DOM, otherwise `null`.
183+
184+
### Clipboard events
185+
186+
```ts
187+
interface ClipboardRecord {
188+
type: "clipboard.copy" | "clipboard.cut" | "clipboard.paste";
189+
t: number;
190+
text: string | null;
191+
html: string | null;
192+
target: number | null;
193+
}
194+
```
195+
196+
`text` and `html` carry the `text/plain` and `text/html` representations from the `ClipboardEvent.clipboardData` payload. Either may be `null` if the corresponding type isn't present or access is denied.
197+
198+
### Media events
199+
200+
```ts
201+
type MediaRecord = {
202+
type: "media.play" | "media.pause" | "media.ended" | "media.seeked" | "media.time";
203+
t: number;
204+
node: number; // id of the <video>/<audio> element
205+
current_time: number; // element.currentTime, in seconds
206+
};
207+
```
208+
209+
`media.time` records throttled `timeupdate` events at roughly 4 Hz and is intended for replay scrubbing rather than exact synchronization.
210+
211+
### Focus events
212+
213+
```ts
214+
interface FocusRecord {
215+
type: "focus" | "blur" | "fullscreen.enter" | "fullscreen.exit";
216+
t: number;
217+
}
218+
```
219+
220+
Window-level focus state. `focus`/`blur` track participant attention (e.g. switching tabs); `fullscreen.enter`/`fullscreen.exit` track whether the browser is in fullscreen mode.
221+
222+
### Scroll events
223+
224+
```ts
225+
type ScrollRecord =
226+
| { type: "scroll.window"; t: number; x: number; y: number }
227+
| { type: "scroll.element"; t: number; node: number; x: number; y: number };
228+
```
229+
230+
`scroll.window` carries `window.scrollX`/`scrollY`. `scroll.element` carries the element's `scrollLeft`/`scrollTop`, keyed by tracked node id. Both are throttled to one record per animation frame per target.
231+
232+
## Viewport state
233+
234+
```ts
235+
interface ViewportState {
236+
w: number; // window.innerWidth
237+
h: number; // window.innerHeight
238+
dpr: number; // window.devicePixelRatio
239+
scale: number; // visualViewport.scale (pinch zoom; 1.0 = none)
240+
offset_x: number; // visualViewport.offsetLeft (pinch pan)
241+
offset_y: number; // visualViewport.offsetTop
242+
}
243+
244+
interface ViewportChange extends ViewportState { t: number }
245+
```
246+
247+
The top-level `viewport` carries the initial state. `viewport_changes` is an append-only log of changes (debounced at ~100ms trailing edge) covering window resize, page zoom (`Ctrl+/Ctrl-`), pinch zoom, and pinch pan.
248+
249+
To find the active viewport at any time `t`, take the last entry in `viewport_changes` with `entry.t <= t`, or fall back to the top-level `viewport` if there is none.
250+
251+
## RNG calls
252+
253+
```ts
254+
interface RngCall {
255+
t: number;
256+
fn: string; // currently always "Math.random"
257+
args: JsonValue; // currently always []
258+
result: JsonValue; // the number that was returned
259+
}
260+
```
261+
262+
While recording, `Math.random` is wrapped so every call is logged into the session-level `rng_calls` array, in the order it was consumed. This includes calls made:
263+
264+
- During trial parameter evaluation (before `t_start`).
265+
- During trial execution (between `t_start` and `t_end`).
266+
- During the post-trial gap.
267+
- During the experimenter's session-level `on_finish` callback.
268+
269+
For deterministic re-execution, a replayer can patch `Math.random` to return `rng_calls[cursor++].result` and let trial code re-consume the tape in order.
270+
271+
## End reasons
272+
273+
| Value | Meaning |
274+
| ----- | ------- |
275+
| `"finished"` | The experiment ran to completion. |
276+
| `"aborted"` | `jsPsych.abortExperiment()` was called. |
277+
| `"unload"` | Reserved for future use; not currently emitted by core. |
278+
| `null` | The recording is still active or was retrieved before `stop()` ran. |
279+
280+
## Privacy considerations
281+
282+
- **Text input is captured verbatim.** Any text typed into form fields, including in survey plugins, appears in the DOM mutation log. Inform participants accordingly when enabling `record_session`.
283+
- **Clipboard payloads are recorded.** Content the participant copies, cuts, or pastes within the experiment is preserved.
284+
- **Math.random is wrapped while recording is active.** The wrapper is removed when the experiment ends. User code that calls `Math.random` will have its outputs in `rng_calls`.
285+
286+
## Backward compatibility
287+
288+
`schema_version: 1` is the contract for v1 recorders. Additive fields (new event types, new optional fields on existing types) may be introduced in future minor revisions without bumping the major version. Replayers should ignore unrecognized event `type` values rather than failing.

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ nav:
7373
- 'jsPsych.turk': 'reference/jspsych-turk.md'
7474
- 'jsPsych.utils': 'reference/jspsych-utils.md'
7575
- 'jsPsych.pluginAPI': 'reference/jspsych-pluginAPI.md'
76+
- 'Session Recording Schema': 'reference/session-recording-schema.md'
7677
- Plugins:
7778
- 'List of Plugins': 'plugins/list-of-plugins.md'
7879
- 'animation': 'plugins/animation.md'

0 commit comments

Comments
 (0)