You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/openmathreasoning1/evaluation.md
+8-2Lines changed: 8 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -231,16 +231,22 @@ ns generate \
231
231
--cluster=local \
232
232
--server_type=trtllm \
233
233
--server_gpus=1 \
234
-
--num_random_seeds=8
234
+
--num_random_seeds=64
235
235
```
236
236
237
237
The output folder will have three folders (apart from log folders):
238
+
238
239
1.`comparison_instances`: This is the folder where input instances for genselect are kept.
240
+
239
241
2.`comparison_judgment`: Output of GenSelect judgments.
242
+
240
243
3.`hle` / `math`: Folder with outputs based on GenSelect's judgments. If `dataset` is not specified in the command, we create a folder with the name `math`
241
244
242
245
To print the metrics run:
243
246
244
247
```bash
245
-
ns summarize_results /workspace/openmath-nemotron-1.5b-eval-cot/self_genselect_hle/hle --metric_type math --cluster local
0 commit comments