|
| 1 | +--- |
| 2 | +title: 'chat' |
| 3 | +sidebar_label: 'chat' |
| 4 | +pagination_prev: null |
| 5 | +pagination_next: null |
| 6 | +description: 'spice chat CLI documentation' |
| 7 | +--- |
| 8 | + |
| 9 | +Start an interactive or one-shot chat with a [model](/docs/components/models/index.md) registered in the Spice runtime. |
| 10 | + |
| 11 | +## Requirements |
| 12 | + |
| 13 | +- Spice runtime must be running |
| 14 | +- At least one model defined in `spicepod.yaml` and the model is ready |
| 15 | + |
| 16 | +## Usage |
| 17 | + |
| 18 | +### Interative Chat: Invoke the command without arguments to open a REPL |
| 19 | + |
| 20 | +```shell |
| 21 | +spice chat [flags] |
| 22 | +``` |
| 23 | + |
| 24 | +### One-shot Chat: Pass a single message as the argument to send a one-shot chat request and print the response |
| 25 | + |
| 26 | +```shell |
| 27 | +spice chat [flags] [<message>] |
| 28 | +``` |
| 29 | + |
| 30 | +## Flags |
| 31 | + |
| 32 | +- `--cloud` Send requests to a Spice Cloud instance instead of the local instance. Default: `false`. |
| 33 | +- `--http-endpoint <string>` Runtime HTTP endpoint. Default: `http://localhost:8090`. |
| 34 | +- `--model <string>` Target model for the chat request. When omitted, the CLI uses the single ready model or prompts for a choice if several models are ready. |
| 35 | +- `--temperature <float32>` Model temperature used for chat request. Default: `1`. |
| 36 | +- `--user-agent <string>` Custom `User-Agent` header sent with every request. |
| 37 | + |
| 38 | +## Examples |
| 39 | + |
| 40 | +When exactly one model is **ready**, `spice chat` opens a REPL that uses that model automatically: |
| 41 | + |
| 42 | +```shell |
| 43 | +> spice chat |
| 44 | +Using model: openai |
| 45 | +chat> hello |
| 46 | +Hello! How can I assist you today? |
| 47 | + |
| 48 | +Time: 0.57s (first token 0.53s). Tokens: 18. Prompt: 8. Completion: 10 (325.04/s). |
| 49 | +``` |
| 50 | + |
| 51 | +When multiple models are **ready**, the command prompts for a selection before starting the REPL: |
| 52 | + |
| 53 | +```shell |
| 54 | +> spice chat |
| 55 | +Use the arrow keys to navigate: ↓ ↑ → ← |
| 56 | +? Select model: |
| 57 | + ▸ openai |
| 58 | + llama |
| 59 | +Using model: openai |
| 60 | +chat> hello |
| 61 | +Hello! How can I assist you today? |
| 62 | + |
| 63 | +Time: 0.55s (first token 0.43s). Tokens: 18. Prompt: 8. Completion: 10 (80.09/s). |
| 64 | +``` |
| 65 | + |
| 66 | +Passing `--model` skips the prompt and directs the request to the specified model. The flag works both in REPL mode and in one‑shot mode: |
| 67 | + |
| 68 | +```shell |
| 69 | +# REPL |
| 70 | +spice chat --model openai |
| 71 | +chat> hello |
| 72 | +Hello! How can I assist you today? |
| 73 | + |
| 74 | +Time: 0.61s (first token 0.58s). Tokens: 18. Prompt: 8. Completion: 10 (285.90/s). |
| 75 | +``` |
| 76 | + |
| 77 | +Single prompt: |
| 78 | + |
| 79 | +```shell |
| 80 | +# One‑shot |
| 81 | +spice chat --model openai "hello" |
| 82 | +Hello! How can I assist you today? |
| 83 | + |
| 84 | +Time: 1.10s (first token 0.80s). Tokens: 18. Prompt: 8. Completion: 10 (33.74/s). |
| 85 | +``` |
0 commit comments