Skip to content

Commit 7779e48

Browse files
jackfranklinDevtools-frontend LUCI CQ
authored andcommitted
AutoRun: store autorun example ID in eval output
If you run the auto-run tool with more than one example, it is not clear in the resulting eval.json file which example any given conversation maps to. Bug: None Change-Id: I9b88fa3ecfdc20ceb4392e34701e2ae0bc2342d1 Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/7456257 Auto-Submit: Jack Franklin <jacktfranklin@chromium.org> Reviewed-by: Kim-Anh Tran <kimanh@chromium.org> Commit-Queue: Kim-Anh Tran <kimanh@chromium.org>
1 parent 7be0f8a commit 7779e48

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

scripts/ai_assistance/suite/to_eval_output.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ export function convertRawOutputToEval(opts: RawToEvalOptions): EvalFileOutput {
8080
assert.ok(modelData, 'No inferenceOptionMetadata');
8181
const processed: Conversation = {
8282
id,
83+
autoRunExampleId: exampleIdFromInput,
8384
chromeVersion,
8485
explanation: exampleMetadata?.explanation ?? '',
8586
model: {

scripts/ai_assistance/suite/types.d.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ export interface EvalFileOutput {
3131
export interface Conversation {
3232
id: string;
3333
chromeVersion: string;
34+
autoRunExampleId: string;
3435
// These are explanations found in the input example HTML that can be used to
3536
// judge the AI's output.
3637
explanation: string;

0 commit comments

Comments
 (0)