Merge branch 'main' into fix/job-delete-and-cli-commands

Author: aliasaria

.agents/skills/transformerlab-cli/SKILL.md

@@ -37,7 +37,10 @@ lab config set server https://your-server-url
 # Step 2: Login with an API key
 lab login --api-key YOUR_API_KEY --server https://your-server-url
 
-# Step 3: Set the current experiment
+# Step 3: Set the current experiment.
+#   Use `lab experiment list` to see existing experiments.
+#   If yours doesn't exist yet, run `lab experiment create your_experiment_name` first.
+#   `lab experiment set-default <id>` is a convenience equivalent to this command.
 lab config set current_experiment your_experiment_name
 
 # Step 4: Verify connectivity
@@ -242,6 +245,54 @@ lab.finish(message="Hello world complete!")
 lab task add ./hello-world-task --no-interactive
 ```
 
+## Managing Experiments
+
+Use `lab experiment` commands to list, create, delete, and set the default experiment. **Experiments are the container for tasks and jobs** — most `lab task` / `lab job` commands operate against the *current* experiment (the one stored in `~/.lab/config.json` as `current_experiment`).
+
+```bash
+# List all experiments. The current default is marked with `*`.
+lab experiment list
+lab --format json experiment list
+
+# Create a new experiment
+lab experiment create my-experiment
+
+# Create and immediately set as the default
+lab experiment create my-experiment --set-default
+
+# Delete an experiment (`--no-interactive` to skip confirmation)
+lab experiment delete my-experiment --no-interactive
+
+# Switch which experiment is the default. This writes to ~/.lab/config.json.
+lab experiment set-default my-experiment
+```
+
+### `lab experiment set-default` vs `lab config set current_experiment`
+
+Both write the same key (`current_experiment`) to `~/.lab/config.json`. Differences:
+
+- `lab experiment set-default <id>` validates that the experiment exists on the server before writing. Prefer this when scripting — it fails fast on a typo.
+- `lab config set current_experiment <id>` is a raw config write and does not validate. Useful when bootstrapping a config (e.g. before the server is reachable) or when you've already confirmed the experiment exists.
+
+### Finding an experiment ID by name
+
+`lab experiment list` (and the JSON form) is the only sanctioned way to discover experiment IDs. **Do not fall back to `curl /experiment/`** — even when you only have a name and need the ID, this CLI surface covers it:
+
+```bash
+# Get just the ID for a given name
+lab --format json experiment list | jq -r '.experiments[] | select(.name=="my-experiment") | .id'
+```
+
+`lab --format json experiment list` returns:
+```json
+{
+  "current_experiment": "my-experiment",
+  "experiments": [
+    {"id": "my-experiment", "name": "my-experiment", "config": {}}
+  ]
+}
+```
+
 ## Managing Models
 
 Use `lab model` commands to list, inspect, create, edit, and delete model groups on the server. Models are organized as **groups** — each group can contain multiple versions (e.g. v1, v2, …).
@@ -493,6 +544,25 @@ lab task queue abc123 -m "train model"
 
 Don't restate the task name, full hyperparameter dict, or file paths — those are already on the job record. Don't copy the user's last message verbatim — synthesize. If the conversation is truly empty of signal, fall back to `"Rerun of <id>, no changes"`.
 
+### Overriding task parameters per queue: `--param key=value`
+
+`lab task queue` accepts repeatable `--param key=value` (alias `-p`) to override values from the task's `parameters:` block for a single job, without mutating `task.yaml`. Values are parsed as YAML scalars: `score=0.42` is a float, `enabled=true` is a bool, `tag=baseline` is a string. Unknown keys (not declared in the task's `parameters:`) fail hard so typos are caught at queue time.
+
+```bash
+# Sweep the same task with different hyperparameters
+for i in $(seq 1 10); do
+  lab task queue TASK_ID --no-interactive \
+    --param description="iteration $i" \
+    --param score=$(python -c "print(0.4 + 0.04*$i)") \
+    -m "Iteration $i"
+done
+
+# Quoting tip: values may contain '=' (split on first '=' only)
+lab task queue TASK_ID --no-interactive --param notes="key=value pairs OK"
+```
+
+Use this instead of `lab task edit --from-file` between queue calls — editing the task affects already-queued-but-not-yet-dispatched jobs and is racy.
+
 ### Selecting a provider when queuing a task
 
 `lab task queue` has no `--provider` flag. With `--no-interactive` it picks the default (usually Local). To pick a specific provider, drive the interactive prompts via stdin. The flow is:
@@ -571,14 +641,18 @@ This applies to launching jobs, fetching logs, checking cluster status, and ever
 | `lab logout` | Remove stored API key | No |
 | `lab whoami` | Show current user and team | No |
 | `lab version` | Show CLI version | No |
+| `lab experiment list` | List all experiments (current default marked with `*`) | No |
+| `lab experiment create <name>` | Create a new experiment (`--set-default` to also switch to it) | No |
+| `lab experiment delete <id>` | Delete an experiment (`--no-interactive` to skip prompt) | No |
+| `lab experiment set-default <id>` | Set the default experiment (validates server-side, then writes `current_experiment` to `~/.lab/config.json`) | No |
 | `lab task list` | List tasks in current experiment | Yes |
 | `lab task info <id>` | Get task details | Yes |
 | `lab task init` | Scaffold `task.yaml` + `main.py` in the current directory (`--interactive` to prompt) | No |
 | `lab task add [dir]` | Add task from directory or `--from-git` URL (`--no-interactive`, `--dry-run`) | Yes |
 | `lab task edit <id>` | Edit an existing task's `task.yaml` (`--from-file`, `--no-interactive`, `--timeout`) | Yes |
 | `lab task upload <id> <path>` | Upload files/directories into an existing task (`--no-interactive`) | Yes |
 | `lab task delete <id>` | Delete a task (`--no-interactive` to skip confirmation) | Yes |
-| `lab task queue <id>` | Queue task on compute provider (`-m/--description` for a markdown run note; required for agents, see "Always write a run description") | Yes |
+| `lab task queue <id>` | Queue task on compute provider (`-m/--description` for a markdown run note; `-p/--param key=value` to override task parameters per run; required for agents, see "Always write a run description") | Yes |
 | `lab task gallery` | Browse/import from task gallery | Yes |
 | `lab job list` | List jobs (`--running` for active only) | Yes |
 | `lab job info <id>` | Get detailed job information | Yes |
@@ -641,7 +715,7 @@ With non-zero exit code.
 - Commands exit with non-zero status on failure
 - With `--format json`, errors return `{"error": "<message>"}`
 - "config not set" errors → run `lab login` first
-- "current_experiment not set" → run `lab config set current_experiment <id>`
+- "current_experiment not set" → run `lab experiment list` to find an existing experiment, then `lab experiment set-default <id>` (or `lab experiment create <name> --set-default` if none exists)
 - Connection refused → check server URL with `lab config`, verify server is running
 - "No compute providers available" → add a provider in team settings first, or check `provider list`
 

AGENTS.md

@@ -158,6 +158,16 @@ curl -H "Authorization: Bearer $TOKEN" -H "X-Team-Id: <team-id>" http://localhos
 - **X-Team-Id**: Required on all protected endpoints. Get it from `GET /users/me/teams`.
 - **Unprotected endpoints**: `auth`, `api_keys`, `quota`, `compute_provider`, and the OpenAI-compatible API.
 
+## package.json overrides
+
+The `overrides` block in `package.json` forces secure transitive versions to clear known vulnerabilities. Revisit when:
+
+- Bumping `eslint-config-erb` / `@typescript-eslint/*` (parent of `minimatch`)
+- Bumping `webpack` / `terser-webpack-plugin` (parent of `serialize-javascript`)
+- Bumping `webpack` / `css-loader` / `postcss-loader` (parents of `postcss`)
+
+When upgrading those, try removing the override first and re-run a dependency vulnerability scan. If the parent now pulls a safe version on its own, drop the override entry and update this file if necessary.
+
 ## Agentic Performance Optimization
 
 - **Context**: When making changes, look at similar existing files (e.g., "Implement the new `ModelService` following the pattern in `api/transformerlab/services/job_service.py`").

cli/src/transformerlab_cli/commands/task.py

@@ -956,11 +956,34 @@ def _prompt_parameters(parameters: dict) -> dict:
     return values
 
 
+def _parse_param_overrides(raw_params: list[str] | None) -> dict:
+    """Parse `key=value` strings into a dict, with values as YAML scalars.
+
+    Splits on the first `=` only so values may contain `=`. Raises
+    typer.BadParameter on malformed input.
+    """
+    if not raw_params:
+        return {}
+    parsed: dict = {}
+    for raw in raw_params:
+        if "=" not in raw:
+            raise typer.BadParameter(f"Expected key=value, got: {raw!r}")
+        key, _, value = raw.partition("=")
+        if not key:
+            raise typer.BadParameter(f"Empty key in: {raw!r}")
+        try:
+            parsed[key] = yaml.safe_load(value)
+        except yaml.YAMLError as e:
+            raise typer.BadParameter(f"Failed to parse value for {key!r}: {e}") from e
+    return parsed
+
+
 def queue_task(
     task_id: str,
     experiment_id: str,
     interactive: bool = True,
     description: str | None = None,
+    param_overrides: dict | None = None,
 ) -> None:
     """Queue a task on a compute provider."""
     with console.status("[bold success]Fetching task...[/bold success]", spinner="dots"):
@@ -996,10 +1019,21 @@ def queue_task(
         console.print(f"[dim]Using provider: {provider.get('name')}[/dim]")
 
     parameters = task.get("parameters", {})
+    overrides = param_overrides or {}
+    if overrides:
+        if not parameters:
+            raise typer.BadParameter(
+                "Task has no parameters declared, cannot use --param. Add a `parameters:` block to task.yaml first."
+            )
+        unknown = sorted(set(overrides) - set(parameters))
+        if unknown:
+            valid = ", ".join(sorted(parameters)) or "(none)"
+            raise typer.BadParameter(f"Unknown parameter(s): {', '.join(unknown)}. Valid keys: {valid}")
     if interactive and parameters:
         param_values = _prompt_parameters(parameters)
     else:
         param_values = {k: (v.get("default", "") if isinstance(v, dict) else v) for k, v in parameters.items()}
+    param_values.update(overrides)
 
     payload = build_launch_payload(
         task, provider.get("name"), param_values, resource_overrides, description=description
@@ -1029,18 +1063,30 @@ def command_task_queue(
             "Pass '-' to read from stdin."
         ),
     ),
+    params: list[str] = typer.Option(
+        None,
+        "--param",
+        "-p",
+        metavar="KEY=VALUE",
+        help=(
+            "Override a task parameter for this queue (repeatable). Value is parsed as a YAML "
+            "scalar (e.g. score=0.42 -> float, enabled=true -> bool). Unknown keys fail."
+        ),
+    ),
 ):
     """Queue a task on a compute provider."""
     current_experiment = require_current_experiment()
     if description == "-":
         if sys.stdin.isatty():
             raise typer.BadParameter('-m - reads the description from stdin; pipe content in or pass -m "...".')
         description = sys.stdin.read()
+    param_overrides = _parse_param_overrides(params)
     queue_task(
         task_id,
         experiment_id=current_experiment,
         interactive=not no_interactive,
         description=description,
+        param_overrides=param_overrides,
     )
 
 

cli/tests/commands/test_task.py

@@ -100,6 +100,152 @@ def test_task_queue_sends_description(_mock_exp, _mock_get, _mock_providers, moc
     assert body["description"] == "hypothesis: larger batch"
 
 
+SAMPLE_TASK_WITH_PARAMS = {
+    "id": "t1",
+    "name": "finetune",
+    "experiment_id": "exp1",
+    "run": "python main.py",
+    "parameters": {
+        "description": "default description",
+        "score": 0.0,
+        "enabled": False,
+        "tag": "baseline",
+    },
+    "config": {},
+}
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_override_lands_in_config(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """`lab task queue --param k=v` sends the override in the launch payload's config field."""
+    result = runner.invoke(
+        app,
+        ["task", "queue", "t1", "--no-interactive", "--param", "description=iteration 7"],
+    )
+    assert result.exit_code == 0, result.output
+    _path, body = mock_post.call_args.args
+    assert body["config"]["description"] == "iteration 7"
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_yaml_coercion(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """--param values are parsed as YAML scalars: floats, bools, and bare strings."""
+    result = runner.invoke(
+        app,
+        [
+            "task",
+            "queue",
+            "t1",
+            "--no-interactive",
+            "--param",
+            "score=0.42",
+            "--param",
+            "enabled=true",
+            "--param",
+            "tag=baseline",
+        ],
+    )
+    assert result.exit_code == 0, result.output
+    _path, body = mock_post.call_args.args
+    cfg = body["config"]
+    assert cfg["score"] == 0.42
+    assert isinstance(cfg["score"], float)
+    assert cfg["enabled"] is True
+    assert cfg["tag"] == "baseline"
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_short_flag_overrides_default(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """`-p k=v` (short alias) overrides the default value from task.yaml."""
+    result = runner.invoke(
+        app,
+        ["task", "queue", "t1", "--no-interactive", "-p", "score=9.9"],
+    )
+    assert result.exit_code == 0, result.output
+    _path, body = mock_post.call_args.args
+    assert body["config"]["score"] == 9.9
+    # Other params still get their defaults
+    assert body["config"]["tag"] == "baseline"
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_value_with_equals_sign(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """Value containing '=' is preserved (split on first '=' only)."""
+    result = runner.invoke(
+        app,
+        ["task", "queue", "t1", "--no-interactive", "--param", "description=key=value pairs"],
+    )
+    assert result.exit_code == 0, result.output
+    _path, body = mock_post.call_args.args
+    assert body["config"]["description"] == "key=value pairs"
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_missing_equals_errors(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """`--param foo` (no '=') fails with a clear error and does not call the API."""
+    result = runner.invoke(app, ["task", "queue", "t1", "--no-interactive", "--param", "foo"])
+    assert result.exit_code != 0
+    assert "key=value" in strip_ansi(result.output).lower()
+    mock_post.assert_not_called()
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_empty_key_errors(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """`--param =5` fails with a clear error."""
+    result = runner.invoke(app, ["task", "queue", "t1", "--no-interactive", "--param", "=5"])
+    assert result.exit_code != 0
+    assert "empty key" in strip_ansi(result.output).lower()
+    mock_post.assert_not_called()
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK_WITH_PARAMS))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_unknown_key_errors(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """`--param notdeclared=1` fails hard when key isn't in the task's parameters block."""
+    result = runner.invoke(
+        app,
+        ["task", "queue", "t1", "--no-interactive", "--param", "notdeclared=1"],
+    )
+    assert result.exit_code != 0
+    out = strip_ansi(result.output)
+    assert "notdeclared" in out
+    # Error should list the valid keys so the user can spot a typo
+    assert "score" in out or "description" in out
+    mock_post.assert_not_called()
+
+
+@patch("transformerlab_cli.commands.task.api.post_json", return_value=_mock_resp({"job_id": "j1"}))
+@patch("transformerlab_cli.commands.task.fetch_providers", return_value=SAMPLE_PROVIDERS)
+@patch("transformerlab_cli.commands.task.api.get", return_value=_mock_resp(SAMPLE_TASK))
+@patch("transformerlab_cli.commands.task.require_current_experiment", return_value="exp1")
+def test_task_queue_param_when_task_has_no_parameters_errors(_mock_exp, _mock_get, _mock_providers, mock_post):
+    """Passing `--param` when the task declares no parameters fails clearly."""
+    result = runner.invoke(app, ["task", "queue", "t1", "--no-interactive", "--param", "x=1"])
+    assert result.exit_code != 0
+    assert "no parameters" in strip_ansi(result.output).lower()
+    mock_post.assert_not_called()
+
+
 @patch(
     "transformerlab_cli.commands.task.api.post_json",
     side_effect=[