feat(hparams): persist hparams panes on creation and add an update endpoint (Layer 3 - 9)

.agents/skills/env-persistence/SKILL.md

@@ -28,6 +28,8 @@ Use this skill for persistence format changes and env reload/view layout behavio
 ## Documentation
 
 - [Skill reference](references/REFERENCE.md)
+- `py/visdom/data_model/README.md` — the storage abstraction layer (`DataStore`/`JSONStore`); persistence now funnels through it
+- `py/visdom/data_model/base.py`, `py/visdom/data_model/json_store.py`
 - `py/visdom/utils/server_utils.py`
 - `py/visdom/server/handlers/web_handlers.py`
 - `py/visdom/server/app.py`

README.md

@@ -358,6 +358,17 @@ vis._send({'data': [trace], 'layout': layout, 'win': 'mywin'})
 - [`vis.check_connection`](#vischeck_connection): check if the server is connected
 - [`vis.replay_log`](#visreplay_log): replay the actions from the provided log file
 
+### Experiments
+Track experiment metadata (hyper-parameters, metrics, tags) alongside your plots, then search and compare runs across your server:
+- [`vis.experiment`](#visexperiment)  : create or update experiment metadata for an env
+- [`vis.log_metrics`](#vislog_metrics)  : append metric observations to an env's experiment
+- [`vis.finish_experiment`](#visfinish_experiment)  : mark an experiment terminal (finished/failed)
+- [`vis.search_experiments`](#vissearch_experiments)  : search experiments across envs with a query
+- [`vis.compare_experiments`](#viscompare_experiments)  : diff experiments field by field
+- [`vis.suggest_experiment`](#vissuggest_experiment)  : suggest parameters for the next run (reserved)
+- [`vis.hparams`](#vishparams)  : open a hyper-parameter pane over selected runs
+- [`vis.update_hparams`](#visupdate_hparams)  : change or refresh an existing hyper-parameter pane
+
 
 ## Loggers
 
@@ -1106,6 +1117,111 @@ This function takes the contents of a visdom log and replays them to the current
 Arguments:
 - `log_filename`: log file to replay the contents of.
 
+### Experiments
+
+Attach experiment metadata — hyper-parameters, metric observations and tags — to an environment, then search and compare runs across your server. Metadata is stored under the environment's `experiment` key and persisted through the server's data store, so a server started without a persistence path (`env_path=None`) has nothing to store, search or compare. Each function returns the server's reply decoded as a dict.
+
+#### vis.experiment
+
+This function creates or updates the experiment metadata for an environment. Calling it again for the same env merges in new params/tags and overwrites the name/description, so it is safe to call at the start of and again during a run.
+
+Arguments:
+- `name`: Display name for the experiment. Defaults to the env id.
+- `params`: Hyper-parameters as a dict of `{name: value}`.
+- `tags`: Free-form tags as a dict of `{name: value}`.
+- `description`: Free-form description.
+- `env`: Environment to attach the experiment to. Defaults to the client's env.
+
+#### vis.log_metrics
+
+This function appends one or more metric observations to an env's experiment, creating the experiment automatically if it does not exist yet.
+
+Arguments:
+- `metrics`: A non-empty dict of `{name: value}` observations.
+- `step`: Optional training step the observations were recorded at.
+- `env`: Environment whose experiment to log to. Defaults to the client's env.
+
+> **Note**: once an experiment is finished (see `finish_experiment`), further `experiment`/`log_metrics` writes are rejected so a completed run's recorded data cannot change after the fact.
+
+#### vis.finish_experiment
+
+This function marks an env's experiment terminal so its recorded data is frozen.
+
+Arguments:
+- `status`: Terminal status, either `finished` (default) or `failed`.
+- `env`: Environment whose experiment to finish. Defaults to the client's env.
+
+#### vis.search_experiments
+
+This function searches the experiments logged on the server, across all environments. The `query` uses a small readable syntax — comparisons (`<`, `<=`, `>`, `>=`, `=`, `!=`, `contains`) over param, metric and tag names, combined with `AND`/`OR` and parentheses. A name is matched bare (`acc`) or namespaced (`metric.acc`, `param.lr`, `tag.owner`) when ambiguous; metrics compare on their latest value. Queries are evaluated in Python, never `eval`'d, so a hostile query is a parse error rather than code execution.
+
+```python
+vis.search_experiments("lr < 0.01 AND acc > 0.9")
+vis.search_experiments("status = finished AND (dataset contains mnist)")
+```
+
+Arguments:
+- `query`: The filter string. `None` (default) returns everything.
+- `limit`: Maximum results to return (default `100`). `None` returns all matches.
+- `offset`: Number of results to skip, for paging (default `0`).
+- `sort_by`: Field to sort by (any of the same names). Defaults to newest-created first.
+- `descending`: Sort direction (default `True`).
+
+Returns a dict of `experiments` (one page's worth), the unpaged `total` matching the query, and the `limit`/`offset`/`query` used.
+
+#### vis.compare_experiments
+
+This function compares the named experiments field by field to see what differs. Finding the runs is `search_experiments`' job — it answers "which runs match?"; this answers "how do these runs differ?". To compare a query's matches, search first and pass the resulting ids on.
+
+Arguments:
+- `env_ids`: A list (or tuple) of environment ids to compare, in the order given. Each must have an experiment.
+
+Returns a dict with the compared runs (`env_ids`, full `experiments`) and a `params`, `metrics` and `tags` section. Each section lists the union of `fields`, the `shared` ones every run agrees on, the `differing` rest, the per-run `values`, and `groups` clustering the runs that agree per field.
+
+#### vis.suggest_experiment
+
+> **Reserved**: this endpoint is a stub. Choosing the next set of hyper-parameters to try (an Optuna-backed search strategy) is planned for a later layer, so the server currently replies `501 Not Implemented` and this function returns a stub dict `{"status": "not_implemented", "suggestion": None, ...}` rather than a real suggestion. The method, its arguments and the endpoint are in place so callers and docs are ready for when the strategy is wired in.
+
+Arguments:
+- `params`: The search space to suggest over, as a dict of `{name: spec}`. Currently ignored by the stub.
+- `env`: Environment (study) to suggest against. Defaults to the client's env.
+
+#### vis.hparams
+
+This function opens a hyper-parameter pane over the experiments logged on the server: the selected runs are flattened into a table of hyper-parameters against their latest metric values (and tags) and registered as a dedicated `hparams` window. The environment is saved as soon as the pane exists, so it survives a server restart without an explicit save.
+
+```python
+vis.hparams("lr < 0.01 AND acc > 0.9")          # query
+vis.hparams(env_ids=["run-a", "run-b"])         # env_ids
+vis.hparams("acc > 0.9", ["run-a", "run-b"])    # both
+```
+
+Arguments:
+- `query`: Selection query, in the syntax of `search_experiments`.
+- `env_ids`: Explicit list of environment ids, kept in the order given.
+- `mode`: `"query"`, `"env_ids"` or `"both"`; inferred from which of `query`/`env_ids` were given when omitted. There is no "show everything" call — with neither argument the server rejects the request.
+- `win` / `env` / `opts`: As for the other plotting methods.
+
+Returns the created window id. The resolved selection is stored on the window, so `update_hparams` can re-run it later.
+
+#### vis.update_hparams
+
+This function changes or refreshes an existing hyper-parameter pane — the dedicated write path for `hparams` windows, since the generic update route only understands plot windows. Called with a `query`/`env_ids`/`mode` selection the pane's selection is replaced, under the rules of `hparams`; called with only `win` the selection stored on the window is re-run, a manual refresh that picks up runs logged (or newly matching) since the pane was built. The pane keeps its id and position, is rebuilt in place, and the environment is saved so the update reaches disk immediately.
+
+```python
+win = vis.hparams("lr < 0.01")
+vis.update_hparams(win, "lr < 0.1 AND acc > 0.9")  # replace the selection
+vis.update_hparams(win)                            # refresh as-is
+```
+
+Arguments:
+- `win`: Id of the pane to update (required). Unknown windows are a 404; non-hparams windows a 400.
+- `query` / `env_ids` / `mode`: The replacement selection, validated exactly as on `hparams`.
+- `env`: Environment holding the window. Defaults to the client's env.
+- `opts`: Overrides the pane's title/size; when omitted the current ones are kept.
+
+Returns the window id.
+
 ## Customizing Visdom
 The user config directory for visdom is
 - `~/.config/visdom` for Linux

js/api/ApiProvider.js

@@ -4,6 +4,7 @@ import React, { useEffect, useRef, useState } from 'react';
 import { showToast } from '../toasts/toastEvents';
 import ApiContext from './ApiContext';
 import Poller from './Legacy';
+import serverPath from './serverPath';
 
 const ApiProvider = ({ children }) => {
   const [connected, setConnected] = useState(false);
@@ -15,20 +16,7 @@ const ApiProvider = ({ children }) => {
   // helper functions //
   // ---------------- //
 
-  // Normalize window.location by removing specific path segments
-  // and ensuring the pathname ends with a '/'
-  const correctPathname = () => {
-    var pathname = window.location.pathname;
-    if (pathname.indexOf('/env/') > -1) {
-      pathname = pathname.split('/env/')[0];
-    } else if (pathname.indexOf('/compare/') > -1) {
-      pathname = pathname.split('/compare/')[0];
-    }
-    if (pathname.slice(-1) != '/') {
-      pathname = pathname + '/';
-    }
-    return pathname;
-  };
+  const correctPathname = serverPath;
 
   // ------------------- //
   // basic communication //

js/api/experimentsApi.js

@@ -0,0 +1,29 @@
+import serverPath from './serverPath';
+
+export function fetchExperimentComparison(envIds, signal) {
+  const ids = (envIds || []).filter((id) => typeof id === 'string');
+  if (ids.length === 0) {
+    return Promise.reject(new Error('No runs to load.'));
+  }
+  return window
+    .fetch(serverPath() + 'experiments/compare', {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      credentials: 'same-origin',
+      body: JSON.stringify({ env_ids: ids }),
+      signal,
+    })
+    .catch((err) => {
+      if (err && err.name === 'AbortError') throw err;
+      throw new Error('Could not reach the server.');
+    })
+    .then((res) => {
+      if (!res.ok) {
+        const reason = res.statusText || 'request failed';
+        throw new Error('Could not load metric history (' + reason + ').');
+      }
+      return res.json();
+    });
+}
+
+export default fetchExperimentComparison;

js/api/serverPath.js

@@ -0,0 +1,12 @@
+export default function serverPath() {
+  var pathname = window.location.pathname;
+  if (pathname.indexOf('/env/') > -1) {
+    pathname = pathname.split('/env/')[0];
+  } else if (pathname.indexOf('/compare/') > -1) {
+    pathname = pathname.split('/compare/')[0];
+  }
+  if (pathname.slice(-1) != '/') {
+    pathname = pathname + '/';
+  }
+  return pathname;
+}

js/main.js

@@ -12,6 +12,7 @@
 'use strict';
 
 import 'fetch';
+import 'rc-slider/assets/index.css';
 import 'rc-tree-select/assets/index.less';
 
 import React, { useContext, useEffect, useRef, useState } from 'react';