Skip to content

Commit eabc6d6

Browse files
committed
stuff not in v2.0.0
1 parent 23feeb6 commit eabc6d6

5 files changed

Lines changed: 13 additions & 57 deletions

File tree

evals/README.md

Lines changed: 5 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Spice can be used to both run language models but also to evaluate their perform
66

77
This recipe demonstrates how to measure the performance of a language model, configured entirely in the spice runtime.
88

9+
910
## Prerequisites
1011

1112
- Ensure you have the Spice CLI installed. Follow the [Getting Started](https://docs.spiceai.org/getting-started) if you haven't done so.
@@ -23,27 +24,12 @@ spice run
2324
2. Run an evaluation against `my_model`. This will take a moment to complete.
2425

2526
```shell
26-
curl -XPOST "http://localhost:8090/v1/evals/tetris" \
27-
-H "Content-Type: application/json" \
28-
-d '{
29-
"model": "my_model"
30-
}'
27+
spice eval tetris --model my_model
3128
```
3229

33-
```json
34-
[
35-
{
36-
"id": "15b8c5351cff98d96db28b8c76ad19dc",
37-
"created_at": "2024-12-30T06:14:54",
38-
"dataset": "small_tetris",
39-
"model": "my_model",
40-
"status": "Completed",
41-
"scorers": ["match"],
42-
"metrics": {
43-
"match/mean": 0.375
44-
}
45-
}
46-
]
30+
```text
31+
ID CREATEDAT DATASET MODEL STATUS SCORERS METRICS
32+
15b8c5351cff98d96db28b8c76ad19dc 2024-12-30T06:14:54 small_tetris my_model Completed [match] map[match/mean:0.375]
4733
```
4834

4935
3. Inspect the results

mcp/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ ai_chat ✅ 15417.63ms 8cda3b72ccc32496
126126

127127
## Connect to Spice over MCP
128128

129-
Spice is an MCP server. It can be connected to like any other MCP server running over HTTP SSE.
129+
Spice is an MCP server. It can be connected to like any other MCP server running over HTTP.
130130

131131
1. Clone the cookbook, and navigate to the MCP recipe.
132132

@@ -169,7 +169,7 @@ kind: Spicepod
169169

170170
tools:
171171
- name: spice_mcp
172-
from: mcp:http://localhost:8090/v1/mcp/sse
172+
from: mcp:http://localhost:8090/v1/mcp
173173
```
174174
175175
6. Run the second Spice instance on separate ports.

mcp/child/spicepod.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ kind: Spicepod
44

55
tools:
66
- name: spice_mcp
7-
from: mcp:http://localhost:8090/v1/mcp/sse
7+
from: mcp:http://localhost:8090/v1/mcp
88

99
models:
1010
- name: openai-with-spice

websearch/README.md

Lines changed: 5 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ This recipe demonstrates how to configure and use Perplexity web search within S
1111
- `SPICE_PERPLEXITY_AUTH_TOKEN`: A valid authentication token for the Perplexity API. Obtain it from [Perplexity's Getting Started guide](https://docs.perplexity.ai/guides/getting-started).
1212
- `SPICE_OPENAI_API_KEY`: A valid OpenAI API key (or equivalent).
1313

14-
## Using Perplexity for internet-informed conversations.
14+
## Using web search for internet-informed conversations
1515

1616
1. Start the Spice runtime:
1717

@@ -25,23 +25,9 @@ spice run
2525
spice chat
2626
```
2727

28-
3. Select `perp`
28+
3. Ask a question
2929

3030
```shell
31-
Use the arrow keys to navigate: ↓ ↑ → ←
32-
? Select model:
33-
openai-w-internet
34-
▸ perp
35-
```
36-
37-
4. Ask a question
38-
39-
```shell
40-
spice chat
41-
```
42-
43-
```shell
44-
Using model: perp
4531
chat> What's the weather in Korea this week?
4632
Here's a summary of the weather in Seoul, South Korea, for this week:
4733

@@ -62,7 +48,7 @@ The weather will transition from cold and snowy to clearer conditions by the wee
6248
Time: 5.02s (first token 1.50s). Tokens: 345. Prompt: 35. Completion: 310 (88.28/s).
6349
```
6450

65-
5. Check the citations used by the model
51+
4. Check the citations used by the model
6652

6753
```shell
6854
spice sql
@@ -117,16 +103,7 @@ spice run
117103
spice chat
118104
```
119105

120-
3. Select `openai-w-internet`
121-
122-
```shell
123-
Use the arrow keys to navigate: ↓ ↑ → ←
124-
? Select model:
125-
▸ openai-w-internet
126-
perp
127-
```
128-
129-
4. Ask a question
106+
3. Ask a question
130107

131108
```shell
132109
>>> spice chat
@@ -158,7 +135,7 @@ For more detailed local conditions, you can check:
158135
Time: 18.28s (first token 10.02s). Tokens: 183. Prompt: 160. Completion: 23 (2.78/s).
159136
```
160137

161-
5. Check that the LLM did use the internet
138+
4. Check that the LLM did use the internet
162139

163140
```shell
164141
spice trace ai_chat

websearch/spicepod.yaml

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,6 @@ models:
1414
tools: the_internet
1515
system_prompt: |
1616
You have access to the internet. You don't know what day it is today.
17-
18-
- name: perp
19-
from: perplexity:sonar
20-
params:
21-
perplexity_auth_token: ${ secrets:SPICE_PERPLEXITY_AUTH_TOKEN }
22-
system_prompt: |
23-
When asked a question with dates, locations or numbers, use an ASCII table when appropriate. Don't overuse it.
2417
tools:
2518
- name: the_internet
2619
from: websearch

0 commit comments

Comments
 (0)