Skip to content

Commit 748cfe1

Browse files
beaufortfrancoisbeaufortfrancois
committed
Add basic WebMCP zaMaker! Evals
1 parent 611a2b1 commit 748cfe1

File tree

2 files changed

+126
-0
lines changed

2 files changed

+126
-0
lines changed

‎evals-cli/examples/pizza-maker/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# WebMCP zaMaker! Evals
2+
3+
This directory contains evaluation test cases for the [WebMCP zaMaker!](../../../demos/pizza-maker/) demo.
4+
5+
Note that `schema.json` is not included here because these evaluations are designed to be run against the [live demo](https://googlechromelabs.github.io/webmcp-tools/demos/pizza-maker) directly in the evals-cli UI, which discovers the tool schemas dynamically from the page.

‎evals-cli/examples/pizza-maker/evals.json‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
[
2+
{
3+
"messages": [
4+
{
5+
"role": "user",
6+
"type": "message",
7+
"content": "I'd like a small Pesto pizza with extra sauce, and add bell peppers and onions."
8+
}
9+
],
10+
"expectedCall": [
11+
{
12+
"functionName": "set_pizza_size",
13+
"arguments": { "size": "Small" }
14+
},
15+
{
16+
"functionName": "set_pizza_style",
17+
"arguments": { "style": "Pesto" }
18+
},
19+
{
20+
"functionName": "toggle_layer",
21+
"arguments": { "layer": "sauce-layer", "action": "add" }
22+
},
23+
{
24+
"functionName": "add_topping",
25+
"arguments": { "topping": "🫑" }
26+
},
27+
{
28+
"functionName": "add_topping",
29+
"arguments": { "topping": "🧅" }
30+
}
31+
]
32+
},
33+
{
34+
"messages": [
35+
{
36+
"role": "user",
37+
"type": "message",
38+
"content": "Make it BBQ"
39+
}
40+
],
41+
"expectedCall": [
42+
{
43+
"functionName": "set_pizza_style",
44+
"arguments": { "style": "BBQ" }
45+
}
46+
]
47+
},
48+
{
49+
"messages": [
50+
{
51+
"role": "user",
52+
"type": "message",
53+
"content": "Add 10 mushrooms"
54+
}
55+
],
56+
"expectedCall": [
57+
{
58+
"functionName": "add_topping",
59+
"arguments": { "topping": "🍄", "count": 10 }
60+
}
61+
]
62+
},
63+
{
64+
"messages": [
65+
{
66+
"role": "user",
67+
"type": "message",
68+
"content": "We are 10 actually"
69+
}
70+
],
71+
"expectedCall": [
72+
{
73+
"functionName": "set_pizza_size",
74+
"arguments": { "size": "Extra Large" }
75+
}
76+
]
77+
},
78+
{
79+
"messages": [
80+
{
81+
"role": "user",
82+
"type": "message",
83+
"content": "Add all remaining toppings"
84+
}
85+
],
86+
"expectedCall": [
87+
{
88+
"functionName": "add_topping",
89+
"arguments": { "topping": "🍕" }
90+
},
91+
{
92+
"functionName": "add_topping",
93+
"arguments": { "topping": "🌿" }
94+
},
95+
{
96+
"functionName": "add_topping",
97+
"arguments": { "topping": "🍍" }
98+
},
99+
{
100+
"functionName": "add_topping",
101+
"arguments": { "topping": "🥓" }
102+
},
103+
{
104+
"functionName": "add_topping",
105+
"arguments": { "topping": "🫒" }
106+
},
107+
{
108+
"functionName": "add_topping",
109+
"arguments": { "topping": "🌽" }
110+
},
111+
{
112+
"functionName": "add_topping",
113+
"arguments": { "topping": "🌶️" }
114+
},
115+
{
116+
"functionName": "add_topping",
117+
"arguments": { "topping": "🐑" }
118+
}
119+
]
120+
}
121+
]

0 commit comments

Comments
 (0)