Rl analytics

plugins/rl-analytics/.gitignore

@@ -0,0 +1 @@
+dist/

plugins/rl-analytics/CodeReview.md

@@ -0,0 +1,135 @@
+# Code Review Reference: jbrowse-plugin-rl-analytics
+
+A snapshot of the plugin's current architecture, intended as a
+reference for a PR reviewer. Describes what the code does today, not
+the history of how it got here.
+
+## Purpose
+
+Captures user interactions with a JBrowse 2 session as RL-compatible
+`(state, action, reward, next_state)` tuples and exports them as
+JSONL or streams them to a webhook. Designed to enable training and
+evaluation of agents that operate JBrowse on behalf of users.
+
+## Architecture
+
+```
+session ──onAction──▶ ActionListener ──▶ ActionBuffer (debounce)
+                                               │
+                                               ▼
+                                       EpisodeManager
+                                       (StateEncoder)
+                                               │
+                       ┌───────────────────────┴───────────────┐
+                       ▼                                       ▼
+                 JSONLExporter                          WebhookExporter
+                 (delta-encoded)                        (batch + interval)
+```
+
+### Source layout (`src/`)
+
+| File | Role |
+|------|------|
+| `index.ts` | Plugin entry. Installs middleware on the session, wires components, exposes `getExportManager()` / `getEpisodeManager()` / `getActionListener()` for tests via a module-level `WeakMap<Plugin, refs>`. |
+| `config.ts` | Six config slots, all consumed via `readConfObject`: `enabled`, `webhookUrl`, `webhookBatchSize`, `webhookFlushIntervalMs`, `inactivityTimeoutMs`, `maxEpisodes`. |
+| `ActionLogger/ActionListener.ts` | MST `addMiddleware` listener. Filters sub-actions via `parentActionEvent`. Maps MST action names → semantic `ActionType` via `ACTION_MAP`. Extracts per-action metadata in `extractMetadata()`. |
+| `ActionLogger/ActionBuffer.ts` | Debounces same-type actions in a sliding window. Calls `onDebouncedAction` with the merged event. |
+| `ActionLogger/ActionTypes.ts` | `ActionType` enum + `ClassifiedAction` shape. |
+| `RLPipeline/StateEncoder.ts` | Reads MST view getters → `BrowserState`. Memoizes within a frame keyed on `(bpPerPx, offsetPx, numTracks, firstRegion, lastActionTimestamp)`. Tracks action counts and unique refNames visited. `encode()` returns a fixed 21-dim numeric vector. |
+| `RLPipeline/EpisodeManager.ts` | Builds `Step` records from prev/next `BrowserState`, manages episode lifecycle, inactivity timeout (dynamic check interval = clamp(1s, 30s, timeout/2)), bounded ring buffer of completed episodes via `maxEpisodes`. |
+| `RLPipeline/types.ts` | `BrowserState`, `Step`, `Episode` types. |
+| `Export/ExportManager.ts` | Holds the JSONLExporter and (optional) WebhookExporter; exposes `getJSONL()` for tests/UI. |
+| `Export/JSONLExporter.ts` | Serializes episodes to JSONL. Delta-encodes `next_observation` against `observation` to reduce file size. |
+| `Export/WebhookExporter.ts` | Buffered POSTer with batch-size and interval triggers. Restores buffer on fetch failure. Uses `navigator.sendBeacon` on dispose if available. |
+| `ObserverView/viewModel.ts` | The "Action Monitor" view model. Reactive log via `observable.array<LogEntry>` for O(1) push and stable React keys. |
+| `ObserverView/ObserverView.tsx` | The view UI. Uses MUI `useTheme()` for dark/light. `requestAnimationFrame`-coalesced auto-scroll. |
+
+### Tests (`src/__tests__/`)
+
+| File | Tests | Coverage |
+|------|-------|----------|
+| `ActionBuffer.test.ts` | Debounce window, merge semantics, flush on type change. |
+| `EpisodeManager.test.ts` | Episode lifecycle, inactivity timeout, `maxEpisodes` eviction, prevState caching. |
+| `StateEncoder.test.ts` | Shape, zoomLevel buckets, label visibility, track type flags, `encode()` vector dims, action count tracking, unique refName tracking, throwing-getter resilience, frame memoization (hit + invalidation). |
+| `WebhookExporter.test.ts` | Batch flush, interval flush, empty buffer, fetch-failure restore, `sendBeacon` on dispose, empty-URL disabled — all via mocked `global.fetch`. |
+
+Plus `products/jbrowse-web/src/tests/RLAnalytics.test.tsx`:
+end-to-end integration through the real `LinearGenomeView`. Covers
+MST `onAction` propagation, full pipeline → JSONL with enriched
+state, prevState caching across steps, `SHOW_TRACK` capture, and
+delta-encoded JSONL output.
+
+**Total: 35 tests across 5 files.**
+
+## Key design decisions
+
+1. **Middleware on session, not rootModel.** Action capture is scoped
+   to a single session; this avoids capturing assembly-loading and
+   bootstrap actions that occur before the user takes control.
+
+2. **Plugin state in a `WeakMap`.** No mutable state on the plugin
+   instance itself; refs (`actionListener`, `episodeManager`,
+   `exportManager`) live in a module-level WeakMap keyed by the
+   plugin object. Accessor methods (`getExportManager()` etc.) exist
+   for the integration test.
+
+3. **`parentActionEvent` filter.** Suppresses sub-actions like
+   `scrollTo` inside `zoomTo`, so only top-level user-initiated
+   actions become Steps.
+
+4. **Frame-level state memoization.** `StateEncoder.extractState`
+   returns the same object on repeat calls within a frame to avoid
+   re-walking MST getters. Invalidated on the next animation frame
+   or when any view input changes.
+
+5. **Delta-encoded JSONL.** `next_observation` is stored as the diff
+   from `observation`, computed by shallow-equality on each
+   `BrowserState` field. The Python converter
+   (`scripts/convert_to_gymnasium.py`) reconstructs full
+   observations.
+
+6. **`isAlive()` checks** on the observer view in
+   `getObserverView()` to handle MST destruction races.
+
+7. **Webhook export is fully optional and gated by config.** If
+   `webhookUrl` is empty the exporter short-circuits and never
+   touches `fetch`.
+
+## Generated docs
+
+`docs/ACTIONS.md` is auto-generated from `ActionListener.ts` by
+`scripts/generate_action_doc.mjs`. It lists all 13 semantic action
+types, the MST action names mapped to each, and the metadata fields
+extracted per action. Regenerate with:
+
+    node scripts/generate_action_doc.mjs
+
+## Type safety
+
+- Zero `as any` in `index.ts` — uses `AbstractSessionModel` and
+  `AbstractViewModel` from `@jbrowse/core/util`, plus a local
+  `RootModelWithSession` interface.
+- Zero `eslint-disable` in `index.ts`.
+- The few `as any` casts in tests are for constructing minimal mock
+  views/steps and are isolated to fixtures.
+
+## Metrics
+
+- **Source**: ~1100 lines of TypeScript across 12 source files
+- **Tests**: 5 files, 35 tests
+- **UMD bundle**: ~36 KB
+- **Published tarball**: `"files": ["esm"]` — excludes `scripts/`,
+  `docs/`, `dist/`, tests
+- **Dependencies**: aligned with the monorepo (`@jbrowse/core`
+  workspace, MUI `^7.3.8`, mobx 6.15, mobx-react 9.2)
+
+## Known follow-ups (non-blocking)
+
+1. `extractMetadata` falls back to storing raw `args` for unknown
+   action names (`logOtherActions` opt-in). If an unknown action
+   passes MST node references, JSON serialization could fail. Safer
+   to drop unknown args entirely.
+2. A generated Python dataclass for the JSONL `BrowserState` schema
+   would let downstream RL training stay in sync with the plugin.
+3. `crypto.randomUUID()` requires Node 18+ / jsdom 20+. Fine on
+   current CI; flag if the matrix widens.

plugins/rl-analytics/README.md

@@ -0,0 +1,168 @@
+# @jbrowse/plugin-rl-analytics
+
+A JBrowse 2 plugin that instruments user interactions for reinforcement
+learning research. Captures semantic user actions (pan, zoom, track toggle,
+feature click, etc.) along with rich browser state observations, and exports
+them as JSONL for offline RL training.
+
+## What it does
+
+The plugin intercepts MST actions on the JBrowse session tree via
+`addMiddleware`, classifies them into a canonical action vocabulary, and
+records (state, action, next_state) tuples with ~21 dimensions of browser
+state per step. Data is exported as newline-delimited JSON compatible with
+Gymnasium/d3rlpy offline RL pipelines.
+
+A built-in view (the **Action Monitor**) provides a live terminal-style
+log of what the instrumentation is capturing, useful for development and
+demonstration.
+
+## Installed as a core plugin
+
+The plugin is registered in `products/jbrowse-web/src/corePlugins.ts`, so
+it's available in any JBrowse 2 build that includes this monorepo. To use
+it in a session, either:
+
+- Add `&rlObserver` to the URL to auto-open the Action Monitor view, or
+- Open it via **Add → Action Monitor** in the menu.
+
+Data is exported via **Tools → Export RL Data (JSONL)**.
+
+## Using the plugin parasitically on a remote JBrowse instance
+
+The plugin can also be deployed as a UMD bundle that loads into any
+existing JBrowse 2 instance without rebuilding it.
+
+### 1. Build the UMD bundle
+
+```bash
+cd plugins/rl-analytics
+pnpm build:umd
+```
+
+This produces `dist/jbrowse-plugin-rl-analytics.umd.js` (~36 KB).
+
+### 2. Host the UMD bundle
+
+Upload `dist/jbrowse-plugin-rl-analytics.umd.js` to any CORS-accessible
+static host (S3 public bucket, GitHub Pages, Vercel, Cloudflare Pages, etc.).
+
+### 3. Generate a parasitic config
+
+Point the generator at the target JBrowse instance. It fetches that
+instance's `config.json`, resolves all relative data URIs to absolute, and
+injects the plugin entry.
+
+```bash
+node plugins/rl-analytics/scripts/generate_parasitic_config.mjs \
+  https://jbrowse.org/code/jb2/main \
+  https://your-cdn.com/jbrowse-plugin-rl-analytics.umd.js \
+  https://your-webhook.com/ingest \
+  > parasitic_config.json
+```
+
+Arguments:
+- **Source URL** — the base URL of the target JBrowse instance
+- **Plugin URL** — where you hosted the UMD bundle in step 2
+- **Webhook URL** (optional) — endpoint for real-time data collection
+
+### 4. Serve the parasitic config
+
+Host `parasitic_config.json` on a CORS-accessible server. Then navigate
+to the target JBrowse instance with `?config=` pointing at your config:
+
+```
+https://jbrowse.org/code/jb2/main/?config=https://your-server.com/parasitic_config.json
+```
+
+The plugin loads at runtime, instruments the session, and (if a webhook URL
+was configured) streams action data to your endpoint.
+
+### 5. Run the webhook receiver (optional)
+
+A lightweight Node HTTP server is included for collecting webhook data.
+It has no dependencies beyond `node:http` and `node:fs`.
+
+```bash
+node plugins/rl-analytics/scripts/webhook_receiver.mjs 8081 ./collected.jsonl
+```
+
+Endpoints:
+- `POST /ingest` — receives `{steps: [...]}` from WebhookExporter
+- `GET /status` — collection stats
+- `GET /episodes` — episode summary
+
+### 6. Convert JSONL to Gymnasium format
+
+Exported JSONL can be converted to NumPy `.npz` files compatible with
+d3rlpy and Decision Transformer:
+
+```bash
+python3 plugins/rl-analytics/scripts/convert_to_gymnasium.py \
+  exported.jsonl \
+  exported.npz
+```
+
+Output arrays: `observations` (N × 21 float32), `actions` (N int64),
+`rewards` (N float32 — zero, for post-hoc reward shaping), `terminals`
+(N bool), `timeouts` (N bool).
+
+## Observation space (21 dimensions)
+
+Per step, the state vector includes:
+
+| Fields | What it encodes |
+|--------|----------------|
+| `bpPerPx`, `offsetPx`, `viewWidthPx`, `viewportBp`, `viewportCenterBp` | Viewport geometry and position |
+| `zoomLevel` | Semantic zoom: `genome`/`region`/`gene`/`sequence`/`basepair` |
+| `hasReferenceSequence`, `hasGeneTrack`, `hasAlignmentTrack`, `hasVariantTrack`, `hasQuantitativeTrack` | Track-type presence flags |
+| `activeTracks[]` | Full track list with trackId, trackType, displayType, height, colorScheme, sortedBy |
+| `labelsVisible`, `visibleContentBlocks`, `openWidgets[]`, `displayedRegions[]` | What the user can currently see |
+| `timeSinceLastAction`, `actionsInLast5Seconds`, `sessionDurationMs` | Temporal features |
+| `actionCountsByType`, `uniqueRefNamesVisited`, `totalActionsThisSession` | Session-level behavior |
+
+## Action vocabulary
+
+Captured MST actions are classified into these semantic types:
+
+| Type | MST actions | Metadata |
+|------|------------|----------|
+| `ZOOM` | zoomTo, moveTo, setNewView | target bpPerPx, refName |
+| `PAN` | horizontalScroll, scrollTo | distance (px), offset |
+| `NAV_TO` | navToLocString, navToSearchString | raw search text, parsed target |
+| `SHOW_TRACK` / `HIDE_TRACK` | showTrack, toggleTrack, hideTrack | trackId |
+| `REORDER_TRACK` | moveTrack, moveTrackUp/Down/ToTop/ToBottom | resolved source/target trackIds |
+| `CONFIG_CHANGE` | setColorScheme, setSortedBy, setShowCenterLine, exportSvg, ... | config-specific (colorBy, sortBy, etc.) |
+| `OPEN_WIDGET` | addWidget | widget type (e.g. BaseFeatureWidget for feature clicks) |
+| `BOOKMARK` | addBookmark, addToHighlights | highlight region |
+| `UNDO` | undo, redo | operation name |
+| `ADD_VIEW` / `REMOVE_VIEW` | addView, removeView | view type |
+| `FLIP_VIEW` | horizontallyFlip | — |
+
+## Configuration
+
+Available config slots (all optional, with sensible defaults):
+
+| Slot | Type | Default | Description |
+|------|------|---------|-------------|
+| `actionBufferSize` | number | 10000 | Max actions buffered |
+| `debounceMs` | number | 500 | Debounce window for merging rapid actions |
+| `inactivityTimeoutMs` | number | 300000 | Episode timeout on inactivity |
+| `maxEpisodes` | number | 100 | Max completed episodes in memory |
+| `logOtherActions` | boolean | false | Log unclassified MST actions (debugging) |
+| `webhookUrl` | string | '' | Real-time webhook endpoint (empty = disabled) |
+
+## Reward
+
+The exported JSONL sets `reward: 0` for every step. Reward is assigned
+post-hoc by the RL researcher based on the objective they're training
+toward (e.g., "navigated to a gene" = +1, "oscillated without progress"
+= -1). Reward shaping is a post-processing step on the JSONL, not
+instrumentation.
+
+## Testing
+
+```bash
+cd ../..  # repo root
+pnpm jest --passWithNoTests plugins/rl-analytics products/jbrowse-web/src/tests/RLAnalytics
+```