Merge branch 'main' into add/juicefs-mounting

Author: dadmobile

.agents/skills/lab-autoresearch/SKILL.md

@@ -0,0 +1,32 @@
+---
+name: lab-autoresearch
+description: Autonomous experiment loop on top of the Transformer Lab `lab` CLI — pick an idea, queue a job, score it, keep or discard, repeat. Use when the user types `/lab-autoresearch`, says "run autoresearch", "optimize X in a loop", "set up autoresearch for …", or asks to run an autonomous experimentation / optimization loop against Transformer Lab.
+allowed-tools: Bash(lab *), Bash(curl *lab.cloud*), Bash(curl *localhost:8338*)
+---
+
+# /lab-autoresearch
+
+This skill is the entry point for the autonomous experiment loop. The full spec lives in the `transformerlab-cli` skill — this skill exists so that typing `/lab-autoresearch` resolves to a real skill instead of falling through.
+
+## What to do
+
+1. **Read the spec first.** The authoritative workflow, subcommands (`init <goal>`, `run`, `finalize`), experiment-notes template, loop rules (parallelism, fire-and-advance, stale-job sweep, keep/discard policy, run-description discipline), and natural-language → `lab` mapping all live in:
+
+   `.claude/skills/transformerlab-cli/references/autoresearch.md`
+
+   (or, in the source repo, `.agents/skills/transformerlab-cli/references/autoresearch.md`)
+
+2. **Then use the `transformerlab-cli` skill** for every `lab` command the loop issues. That skill has the command reference, troubleshooting, and the rule that you must use `lab` (not raw `curl`) for normal operations.
+
+## Quick orientation
+
+The loop is layered on the `lab` CLI:
+
+- One **experiment** per session, one **job** per iteration.
+- A job's `-m/--description` is the iteration note; its `score` dict (set via `lab.finish(score=…)`) is the result; `lab job discard` is the keep/discard flag.
+- For hyperparameter fan-out, prefer the task's `sweeps:` block over manually queuing N jobs.
+- The session plan (objective, files in scope, constraints, backlog, what's been tried) is written to **experiment notes** via `lab notes` — there is no local `autoresearch.md` file.
+
+Everything beyond `init` / `run` / `finalize` (status, keep/discard, sweeps, ideas, stopping running jobs, exiting the loop) is just the agent running the right `lab` call in response to natural-language requests — no dedicated subcommand needed.
+
+**Do not improvise the loop without reading `references/autoresearch.md` first.** The rules around parallelism, fire-and-advance, and stale-job sweeps are load-bearing and not derivable from the `lab` CLI alone.

.agents/skills/react-doctor/SKILL.md

@@ -0,0 +1,32 @@
+---
+name: react-doctor
+description: Use when finishing a feature, fixing a bug, before committing React code, or when the user wants to improve code quality or clean up a codebase. Checks for score regression. Covers lint, dead code, accessibility, bundle size, architecture diagnostics.
+version: "1.0.0"
+---
+
+# React Doctor
+
+Scans React codebases for security, performance, correctness, and architecture issues. Outputs a 0–100 health score.
+
+## After making React code changes:
+
+Run `npx -y react-doctor@latest . --verbose --diff` and check the score did not regress.
+
+If the score dropped, fix the regressions before committing.
+
+## For general cleanup or code improvement:
+
+Run `npx -y react-doctor@latest . --verbose` (without `--diff`) to scan the full codebase. Fix issues by severity — errors first, then warnings.
+
+## Command
+
+```bash
+npx -y react-doctor@latest . --verbose --diff
+```
+
+| Flag        | Purpose                                       |
+| ----------- | --------------------------------------------- |
+| `.`         | Scan current directory                        |
+| `--verbose` | Show affected files and line numbers per rule |
+| `--diff`    | Only scan changed files vs base branch        |
+| `--score`   | Output only the numeric score                 |

.agents/skills/transformerlab-cli/SKILL.md

@@ -76,6 +76,20 @@ Show current user, team, and server.
 
 ---
 
+## Experiment Commands
+
+### `experiment list [--tag <tag>]...`
+
+List experiments. Pass `--tag` one or more times to filter to experiments that have **all** of the given tags (AND semantics). Filtering is client-side.
+
+```bash
+lab experiment list --tag fine-tuning            # all experiments tagged fine-tuning
+lab experiment list --tag fine-tuning --tag llama  # must have both tags
+lab --format json experiment list --tag llama    # JSON output includes `tags` field
+```
+
+---
+
 ## Task Commands
 
 **All task commands require `current_experiment` to be set.**
@@ -253,7 +267,7 @@ Stop a running job.
 
 Launch the interactive job monitor TUI (Textual app).
 
-**Warning:** This launches a full terminal UI. **Never use in automated or agent contexts.** Use `job list` + `job machine-logs` instead.
+**Warning:** This launches a full terminal UI. **Never use when operating as an AI agent or in non-interactive automation.** Use `job list`, `job info`, and `job task-logs` instead.
 
 ---
 
@@ -285,33 +299,71 @@ Add a new compute provider. Interactive prompts by default.
 | Option | Description |
 |---|---|
 | `--name <name>` | Provider name |
-| `--type <type>` | Provider type: `slurm`, `skypilot`, `runpod`, `local` |
+| `--type <type>` | Provider type: `slurm`, `skypilot`, `runpod`, `local`, `dstack`, `aws`, `gcp`, `azure` |
 | `--config <json>` | Config as JSON string |
 | `--interactive` / `--no-interactive` | Toggle prompts. Non-interactive requires `--name`, `--type`, AND `--config` (pass `'{}'` for `local`). |
+| `--credentials-file <path>` | Path to a JSON file containing provider secrets. Shape depends on `--type` (see below). Keeps secrets out of `argv` (shell history, `ps`). |
 
 **Always use `--no-interactive` with `--name`, `--type`, and `--config` in automated workflows.**
 
 #### Per-type config schema
 
 The shape of `--config` depends on `--type`:
 
-| Type | Config keys |
-|---|---|
-| `local` | (none — pass `{}`) |
-| `skypilot` | `server_url`, `api_token` |
-| `slurm` | `mode` (`ssh` or `rest`); for `ssh`: `ssh_host`, `ssh_user`, `ssh_key_path`, `ssh_port`; for `rest`: `rest_url`, `api_token` |
-| `runpod` | `api_key` (required), `api_base_url`, `default_gpu_type`, `default_region`, `default_template_id`, `default_network_volume_id` |
+| Type | Config keys | `--credentials-file` shape |
+|---|---|---|
+| `local` | (none — pass `{}`) | (n/a) |
+| `skypilot` | `server_url` | `{"api_token": "..."}` (or any other secret keys to merge into config) |
+| `slurm` | `mode` (`ssh` or `rest`); for `ssh`: `ssh_host`, `ssh_user`, `ssh_key_path`, `ssh_port`; for `rest`: `rest_url` | `{"api_token": "..."}` for REST mode |
+| `runpod` | `api_base_url`, `default_gpu_type`, `default_region`, `default_template_id`, `default_network_volume_id` | `{"api_key": "..."}` |
+| `dstack` | `server_url`, `dstack_project` | `{"api_token": "..."}` |
+| `aws` | `region`. | `{"aws_access_key_id": "...", "aws_secret_access_key": "..."}` — uploaded to `~/.aws/credentials` on the API host. |
+| `gcp` | `region`, optional `zone`. | The **raw service account JSON key file** (point `--credentials-file` directly at the file you'd pass to `gcloud auth activate-service-account --key-file=`). |
+| `azure` | `azure_subscription_id`, `azure_tenant_id`, `azure_client_id`, `azure_location` | `{"azure_client_secret": "..."}` |
+
+You can put any combination of secret fields in `--credentials-file` — for non-AWS/GCP types they merge on top of `--config` and take precedence on conflict. AWS access keys and GCP service account JSON are routed to their dedicated upload endpoints.
 
 ```bash
 lab provider add --no-interactive --name local --type local --config '{}'
+
+# Secrets via --credentials-file — preferred for scripted / CI flows
+# skypilot-creds.json: {"api_token": "TOKEN"}
 lab provider add --no-interactive --name sky1 --type skypilot \
-  --config '{"server_url": "https://sky.example.com", "api_token": "TOKEN"}'
+  --config '{"server_url": "https://sky.example.com"}' \
+  --credentials-file ./skypilot-creds.json
+
+# slurm-creds.json: {"api_token": "TOKEN"}
 lab provider add --no-interactive --name slurm-ssh --type slurm \
   --config '{"mode": "ssh", "ssh_host": "cluster.example.com", "ssh_user": "ali", "ssh_key_path": "~/.ssh/id_rsa", "ssh_port": "22"}'
+
+# runpod-creds.json: {"api_key": "KEY"}
 lab provider add --no-interactive --name rp1 --type runpod \
-  --config '{"api_key": "KEY", "default_gpu_type": "NVIDIA H100"}'
+  --config '{"default_gpu_type": "NVIDIA H100"}' \
+  --credentials-file ./runpod-creds.json
+
+# dstack-creds.json: {"api_token": "TOKEN"}
+lab provider add --no-interactive --name dstack1 --type dstack \
+  --config '{"server_url": "http://0.0.0.0:3000", "dstack_project": "main"}' \
+  --credentials-file ./dstack-creds.json
+
+# aws-creds.json: {"aws_access_key_id": "AKIA...", "aws_secret_access_key": "..."}
+lab provider add --no-interactive --name aws1 --type aws \
+  --config '{"region": "us-east-1"}' \
+  --credentials-file ./aws-creds.json
+
+# Pass the GCP service account JSON file directly
+lab provider add --no-interactive --name gcp1 --type gcp \
+  --config '{"region": "us-central1"}' \
+  --credentials-file ~/.config/gcloud/sa-key.json
+
+# azure-creds.json: {"azure_client_secret": "REDACTED"}
+lab provider add --no-interactive --name azure1 --type azure \
+  --config '{"azure_subscription_id": "sub", "azure_tenant_id": "tenant", "azure_client_id": "client", "azure_location": "eastus"}' \
+  --credentials-file ./azure-creds.json
 ```
 
+> **Security note:** Secrets embedded inside `--config` (`api_token`, `api_key`, `azure_client_secret`, etc.) appear in your shell history and in `ps`/proc listings while the command runs. **For scripted / CI use, prefer `--credentials-file PATH`** — values stay on disk, never on `argv`. `chmod 600` the file, source it from a secret manager / CI vault, and delete it after the `lab provider add` call. For one-off interactive use, the interactive `lab provider add` flow prompts for secrets without echoing them to argv.
+
 ### `provider update <provider_id>`
 
 Update a compute provider. Fields are merged with existing config.
@@ -320,6 +372,7 @@ Update a compute provider. Fields are merged with existing config.
 |---|---|
 | `--name <name>` | New provider name |
 | `--config <json>` | Config fields as JSON string (merged with existing) |
+| `--credentials-file <path>` | Path to a JSON file whose fields are merged into the config patch (file values win over `--config`). Use this for credential rotation — keeps secrets out of `argv`. |
 | `--disabled` / `--enabled` | Disable or enable the provider |
 
 ### `provider delete <provider_id>`

.agents/skills/transformerlab-cli/references/autoresearch.md

@@ -128,7 +128,7 @@ curl -s -H "Authorization: Bearer API_KEY" -H "X-Team-Id: TEAM_ID" \
 - `provider delete` → add `--yes`
 - `task gallery` → use `--import GALLERY_ID` instead of browsing
 
-**Never use these commands in automated contexts:**
+**Never use these commands when operating as an AI agent or in non-interactive automation:**
 - `job monitor` (launches TUI)
 - `task interactive` (blocks for interactive session)
 

.agents/skills/transformerlab-cli/references/commands.md

@@ -117,17 +117,32 @@ lab --format json provider list --include-disabled
 # 2. Add a new provider (non-interactive — see SKILL.md "Managing Providers"
 #    for the per-type config schema). Examples:
 
-# SkyPilot
+# For all examples below, prefer --credentials-file PATH for secrets so they
+# stay out of shell history and `ps`. `chmod 600` the file and delete it after.
+
+# SkyPilot — skypilot-creds.json: {"api_token": "TOKEN"}
 lab --format json provider add --no-interactive --name my-skypilot --type skypilot \
-  --config '{"server_url": "https://sky.example.com", "api_token": "TOKEN"}'
+  --config '{"server_url": "https://sky.example.com"}' \
+  --credentials-file ./skypilot-creds.json
 
-# Slurm over SSH
+# Slurm over SSH (SSH key path lives on the API host; no secret on argv)
 lab --format json provider add --no-interactive --name my-slurm --type slurm \
   --config '{"mode": "ssh", "ssh_host": "cluster.example.com", "ssh_user": "admin", "ssh_key_path": "~/.ssh/id_rsa", "ssh_port": "22"}'
 
-# RunPod
+# RunPod — runpod-creds.json: {"api_key": "RUNPOD_KEY"}
 lab --format json provider add --no-interactive --name my-runpod --type runpod \
-  --config '{"api_key": "RUNPOD_KEY", "default_gpu_type": "NVIDIA H100"}'
+  --config '{"default_gpu_type": "NVIDIA H100"}' \
+  --credentials-file ./runpod-creds.json
+
+# AWS — aws-creds.json: {"aws_access_key_id": "...", "aws_secret_access_key": "..."}
+lab --format json provider add --no-interactive --name my-aws --type aws \
+  --config '{"region": "us-east-1"}' \
+  --credentials-file ./aws-creds.json
+
+# GCP — point --credentials-file directly at the service account JSON key file
+lab --format json provider add --no-interactive --name my-gcp --type gcp \
+  --config '{"region": "us-central1"}' \
+  --credentials-file ~/.config/gcloud/sa-key.json
 
 # 3. Health-check immediately after creating
 lab --format json provider check PROVIDER_ID
@@ -136,7 +151,10 @@ lab --format json provider check PROVIDER_ID
 lab provider disable PROVIDER_ID
 lab provider enable PROVIDER_ID
 
-# 5. Update fields (config is MERGED — pass only changed keys)
+# 5. Update fields (config is MERGED — pass only changed keys).
+#    For credential rotation prefer --credentials-file over inline --config.
+lab --format json provider update PROVIDER_ID --credentials-file ./rotated.json
+# Or inline (secret will appear in shell history / ps):
 lab --format json provider update PROVIDER_ID --config '{"api_token": "NEW_TOKEN"}'
 
 # 6. Delete (note: --no-interactive, NOT --yes)