Commit 4d74c0c
Chibi Vikram
fix: remove missing custom evaluators from eval set
The eval set was referencing two custom evaluators that don't exist:
- ResumeCompletedEvaluator
- SuspendResumeTrajectoryEvaluator
This caused the eval command to fail with:
'Could not find the following evaluators'
Changes:
- Removed evaluatorRefs and evaluationCriterias
- Removed expectedOutputs (agent suspends, doesn't complete on first run)
- Updated description to clarify suspend/resume testing workflow
- Kept simple inputs for testing suspend phase
The eval set now works for demonstrating suspend behavior without
requiring custom evaluators. For full suspend/resume testing with
evaluators, use the demo scripts or --resume flag.1 parent 06061a4 commit 4d74c0c
1 file changed
Lines changed: 1 addition & 18 deletions
Lines changed: 1 addition & 18 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
| 5 | + | |
10 | 6 | | |
11 | 7 | | |
12 | 8 | | |
13 | 9 | | |
14 | 10 | | |
15 | 11 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | 12 | | |
25 | 13 | | |
26 | 14 | | |
27 | 15 | | |
28 | 16 | | |
29 | 17 | | |
30 | 18 | | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | 19 | | |
37 | 20 | | |
38 | 21 | | |
| |||
0 commit comments