Skip to content

feat(cli): refactor help to give better hints for humans & ai#966

Open
jycouet wants to merge 10 commits intomainfrom
feat/better-ai-oneshot
Open

feat(cli): refactor help to give better hints for humans & ai#966
jycouet wants to merge 10 commits intomainfrom
feat/better-ai-oneshot

Conversation

@jycouet
Copy link
Contributor

@jycouet jycouet commented Mar 2, 2026

Closes: #964

Should be better in oneshoting things

@changeset-bot
Copy link

changeset-bot bot commented Mar 2, 2026

🦋 Changeset detected

Latest commit: 14aa8f7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
sv Patch
@sveltejs/sv-utils Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 2, 2026

Open in StackBlitz

npx https://pkg.pr.new/svelte-migrate@966
npx https://pkg.pr.new/sv@966
npx https://pkg.pr.new/@sveltejs/sv-utils@966

commit: 28eadc0

@jycouet

This comment was marked as outdated.

@shreyassanthu77
Copy link

oooo this is nice

it'd be even better if we detect any env vars and use those to just print the text directly instead of doing the hidden thing i think. Like completely disable the interactive mode when an agent is detected and have a force-interactive flag if the user really wants to do this

Opencode sets AGENT=1 and OPENCODE=1
Claude code sets CLAUDECODE=1
not really sure about others but they also probably have env vars that can be used

@jycouet
Copy link
Contributor Author

jycouet commented Mar 2, 2026

Interactive is not ideal, but it's asking not answerd question. And ai can answer them.

I would like to do some more bench with even better instructions

@manuel3108
Copy link
Member

Honestly, im not sure i agree with this. Now the default help screen is dramatically making sv create it's favorite, since it's the only command that print's it's full capabilities. Either we should do the same for every command or leave it, if you ask me. Or maybe I'm just totally misunderstanding the goal of this PR.

I do agree though, I wouldn't make any expectations on the currently executing agent by checking a bunch of env vars. Especially since a new agent type will popup all the time and someone will always be unhappy.

I do wonder though, why does this problem even exist? Would it be helpful for the llm to know about the docs on our website. I did try around with a few prompts locally, with the main idea to point the llm to the correct markdown docs, but seems like it did not like too much as well. https://svelte.dev/docs/cli/llms.txt

@jycouet
Copy link
Contributor Author

jycouet commented Mar 6, 2026

I think I didn't manage the "sell" well the feature here. I'll take your feedback into account and push some changes before a new review.

I think that we can do better. Let's see 😁

@svelte-docs-bot
Copy link

@jycouet
Copy link
Contributor Author

jycouet commented Mar 7, 2026

Closes: #964

Why

AI agents (Claude Code, Cursor, OpenCode, ...) struggle with sv because:

  • The CLI syntax for add-on options (vitest="usages:unit") is non-obvious
  • There's no way to know which flags are needed to skip interactive prompts
  • Agents end up in trial-and-error loops, wasting turns and money

The alternative — fetching llms.txt — requires internet, a tool call, and is ~28KB. The --help output is ~4KB (7x smaller), local, and already in the tool the agent is using.

What changed

1. sv (bare) — unchanged

Same behavior as before. Just the command list:

Usage: sv [options] [command]

Commands:
  create [options] [path]        Scaffold a new project (--add to include add-ons)
  add [options] [add-on...]      applies specified add-ons into a project
  migrate [options] [migration]  a CLI for migrating Svelte(Kit) codebases
  check [options]                a CLI for checking your Svelte code

2. sv --help — full reference in one call

Appends sv create help (with add-on reference) so agents get everything in a single --help call.

3. sv create --help — add-on reference + examples + non-interactive hints

Shows all add-ons with their options and defaults, syntax guide, and copy-paste examples:

Add-Ons:
  vitest                            usages: unit, component (default: unit, component)
  tailwindcss                       plugins: typography, forms (default: none)
  drizzle                           database: postgresql, mysql, sqlite (default: sqlite)
                                    client: postgres.js, neon, ... (default: libsql)
                                    docker: yes, no (default: none)
  ...

Add-On Syntax:
  <addon>                               add with defaults (may still prompt)
  <addon>=<opt>:<val>                   set a single option
  <addon>=<opt1>:<val1>+<opt2>:<val2>   set multiple options
  <addon>=<opt>:none                    explicitly set no value (for multiselect)
  To skip prompts, explicitly set ALL options (use defaults shown above).

Non-interactive usage:
  Provide --template, --types, --add, and --install (or --no-install) to skip prompts entirely.
  Note: --add and --no-add-ons cannot be used together.

Examples:
  sv create my-app --template minimal --types ts --add prettier eslint --install pnpm
  sv create my-app --template minimal --types ts --add vitest="usages:unit" tailwindcss="plugins:none" --install pnpm
  sv create my-app --template minimal --types ts --add drizzle="database:postgresql+client:postgres.js" --no-install

4. sv add --help — same add-on reference + its own examples

Examples:
  sv add prettier eslint
  sv add vitest="usages:unit" tailwindcss="plugins:none"
  sv add drizzle="database:postgresql+client:postgres.js+docker:yes"

Refactoring

  • Extracted getHelpSections() in common.ts — shared help boilerplate used by both add and create
  • Extracted formatAddonHelpSection() in add.ts — shared add-on list + syntax section
  • Added color.hidden to sv-utils for the invisible AI hint

Benchmark results

Prompt: "Create a sveltekit project with tailwind and vitest (unit testing), non-interactive, pnpm"

Using Claude Code (Sonnet) — measuring turns to successful sv create:

Version Turns Time Cost sv create result
Published sv 49 187s $0.41 ❌ Wrong (manual Tailwind v3 setup)
This PR (initial) 21 97s $0.18 ❌ 8 retries (usage vs usages, missing options)
This PR (with examples + defaults) 6 32s $0.07 One-shot

The key improvements that got us to one-shot:

  • Showing defaults (e.g. tailwindcss ... (default: none)) — agents know to pass plugins:none
  • Concrete examples with exact syntax — no more guessing vitest=unit vs vitest="usages:unit"
  • Non-interactive hint listing required flags — agents pass --template and --types immediately

The published version "looks fast" at 2 turns, but it actually failed — the agent got stuck on the tailwindcss plugin prompt and fell back to manually installing packages. This PR makes it succeed reliably.

Benefits for humans too

This isn't just for AI. The examples, syntax reference, and defaults are useful for anyone scripting sv or just wanting to skip prompts. The --help output is now a self-contained reference.

@jycouet jycouet requested a review from manuel3108 March 7, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Describe how to run non-interactively in --help

3 participants