Skip to content

Commit b1afa16

Browse files
Add --eval-provider option to CLI reference documentation
1 parent 7f05f64 commit b1afa16

1 file changed

Lines changed: 9 additions & 4 deletions

File tree

docs/projects/hedit/cli-reference.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ hedit annotate DESCRIPTION [OPTIONS]
5858
| `--api-url` | | TEXT | | API endpoint URL |
5959
| `--model` | `-m` | TEXT | | Model to use for annotation |
6060
| `--eval-model` | | TEXT | | Model for evaluation/assessment agents (see below) |
61+
| `--eval-provider` | | TEXT | | Provider for evaluation model (e.g., Cerebras) |
6162
| `--provider` | | TEXT | | Provider preference (e.g., Cerebras) |
6263
| `--temperature` | `-t` | FLOAT | | LLM temperature |
6364
| `--schema` | `-s` | TEXT | 8.3.0 | HED schema version |
@@ -68,13 +69,14 @@ hedit annotate DESCRIPTION [OPTIONS]
6869
| `--api` | | BOOL | False | Use API backend (default) |
6970
| `--verbose` | `-v` | BOOL | False | Show detailed output |
7071

71-
!!! info "About `--eval-model`"
72-
The `--eval-model` option specifies a separate model for the evaluation, assessment, and feedback summarization agents. This is useful for:
72+
!!! info "About `--eval-model` and `--eval-provider`"
73+
The `--eval-model` option specifies a separate model for the evaluation, assessment, and feedback summarization agents. The `--eval-provider` option allows specifying a different provider for the evaluation model. This is useful for:
7374

74-
- **Model benchmarking**: Use a consistent evaluator (e.g., `qwen/qwen3-235b-a22b`) across different annotation models for fair comparison
75+
- **Model benchmarking**: Use a consistent evaluator (e.g., `qwen/qwen3-235b-a22b-2507` via Cerebras) across different annotation models for fair comparison
7576
- **Cost optimization**: Use a cheaper model for annotation and a more capable model for quality assessment
77+
- **Speed optimization**: Use a fast provider like Cerebras for the evaluation model
7678

77-
When not specified, all agents use the same model as `--model`.
79+
When not specified, all agents use the same model and provider as `--model` and `--provider`.
7880

7981
**Examples:**
8082

@@ -101,6 +103,7 @@ hedit annotate "..." --standalone
101103
hedit annotate "A monkey reaches for a reward" \
102104
--model openai/gpt-4o-mini \
103105
--eval-model qwen/qwen3-235b-a22b-2507 \
106+
--eval-provider Cerebras \
104107
--standalone
105108
```
106109

@@ -128,6 +131,7 @@ hedit annotate-image IMAGE [OPTIONS]
128131
| `--api-key` | `-k` | TEXT | | OpenRouter API key |
129132
| `--model` | `-m` | TEXT | | Model to use for annotation |
130133
| `--eval-model` | | TEXT | | Model for evaluation/assessment agents |
134+
| `--eval-provider` | | TEXT | | Provider for evaluation model (e.g., Cerebras) |
131135
| `--provider` | | TEXT | | Provider preference (e.g., Cerebras) |
132136
| `--temperature` | `-t` | FLOAT | | LLM temperature |
133137
| `--schema` | `-s` | TEXT | 8.4.0 | HED schema version |
@@ -154,6 +158,7 @@ hedit annotate-image screen.png -o json > result.json
154158
hedit annotate-image nsd_image.png \
155159
--model openai/gpt-4o-mini \
156160
--eval-model qwen/qwen3-235b-a22b-2507 \
161+
--eval-provider Cerebras \
157162
--standalone
158163
```
159164

0 commit comments

Comments
 (0)