Skip to content

Commit d30f836

Browse files
authored
Merge pull request #1974 from transformerlab/add/experiment-skill-docs-v2
Document `lab experiment` namespace in transformerlab-cli skill
2 parents ae7ed51 + 38eeed1 commit d30f836

1 file changed

Lines changed: 57 additions & 2 deletions

File tree

  • .agents/skills/transformerlab-cli

.agents/skills/transformerlab-cli/SKILL.md

Lines changed: 57 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,10 @@ lab config set server https://your-server-url
3737
# Step 2: Login with an API key
3838
lab login --api-key YOUR_API_KEY --server https://your-server-url
3939

40-
# Step 3: Set the current experiment
40+
# Step 3: Set the current experiment.
41+
# Use `lab experiment list` to see existing experiments.
42+
# If yours doesn't exist yet, run `lab experiment create your_experiment_name` first.
43+
# `lab experiment set-default <id>` is a convenience equivalent to this command.
4144
lab config set current_experiment your_experiment_name
4245

4346
# Step 4: Verify connectivity
@@ -242,6 +245,54 @@ lab.finish(message="Hello world complete!")
242245
lab task add ./hello-world-task --no-interactive
243246
```
244247

248+
## Managing Experiments
249+
250+
Use `lab experiment` commands to list, create, delete, and set the default experiment. **Experiments are the container for tasks and jobs** — most `lab task` / `lab job` commands operate against the *current* experiment (the one stored in `~/.lab/config.json` as `current_experiment`).
251+
252+
```bash
253+
# List all experiments. The current default is marked with `*`.
254+
lab experiment list
255+
lab --format json experiment list
256+
257+
# Create a new experiment
258+
lab experiment create my-experiment
259+
260+
# Create and immediately set as the default
261+
lab experiment create my-experiment --set-default
262+
263+
# Delete an experiment (`--no-interactive` to skip confirmation)
264+
lab experiment delete my-experiment --no-interactive
265+
266+
# Switch which experiment is the default. This writes to ~/.lab/config.json.
267+
lab experiment set-default my-experiment
268+
```
269+
270+
### `lab experiment set-default` vs `lab config set current_experiment`
271+
272+
Both write the same key (`current_experiment`) to `~/.lab/config.json`. Differences:
273+
274+
- `lab experiment set-default <id>` validates that the experiment exists on the server before writing. Prefer this when scripting — it fails fast on a typo.
275+
- `lab config set current_experiment <id>` is a raw config write and does not validate. Useful when bootstrapping a config (e.g. before the server is reachable) or when you've already confirmed the experiment exists.
276+
277+
### Finding an experiment ID by name
278+
279+
`lab experiment list` (and the JSON form) is the only sanctioned way to discover experiment IDs. **Do not fall back to `curl /experiment/`** — even when you only have a name and need the ID, this CLI surface covers it:
280+
281+
```bash
282+
# Get just the ID for a given name
283+
lab --format json experiment list | jq -r '.experiments[] | select(.name=="my-experiment") | .id'
284+
```
285+
286+
`lab --format json experiment list` returns:
287+
```json
288+
{
289+
"current_experiment": "my-experiment",
290+
"experiments": [
291+
{"id": "my-experiment", "name": "my-experiment", "config": {}}
292+
]
293+
}
294+
```
295+
245296
## Managing Models
246297

247298
Use `lab model` commands to list, inspect, create, edit, and delete model groups on the server. Models are organized as **groups** — each group can contain multiple versions (e.g. v1, v2, …).
@@ -571,6 +622,10 @@ This applies to launching jobs, fetching logs, checking cluster status, and ever
571622
| `lab logout` | Remove stored API key | No |
572623
| `lab whoami` | Show current user and team | No |
573624
| `lab version` | Show CLI version | No |
625+
| `lab experiment list` | List all experiments (current default marked with `*`) | No |
626+
| `lab experiment create <name>` | Create a new experiment (`--set-default` to also switch to it) | No |
627+
| `lab experiment delete <id>` | Delete an experiment (`--no-interactive` to skip prompt) | No |
628+
| `lab experiment set-default <id>` | Set the default experiment (validates server-side, then writes `current_experiment` to `~/.lab/config.json`) | No |
574629
| `lab task list` | List tasks in current experiment | Yes |
575630
| `lab task info <id>` | Get task details | Yes |
576631
| `lab task init` | Scaffold `task.yaml` + `main.py` in the current directory (`--interactive` to prompt) | No |
@@ -639,7 +694,7 @@ With non-zero exit code.
639694
- Commands exit with non-zero status on failure
640695
- With `--format json`, errors return `{"error": "<message>"}`
641696
- "config not set" errors → run `lab login` first
642-
- "current_experiment not set" → run `lab config set current_experiment <id>`
697+
- "current_experiment not set" → run `lab experiment list` to find an existing experiment, then `lab experiment set-default <id>` (or `lab experiment create <name> --set-default` if none exists)
643698
- Connection refused → check server URL with `lab config`, verify server is running
644699
- "No compute providers available" → add a provider in team settings first, or check `provider list`
645700

0 commit comments

Comments
 (0)