Skip to content

Commit edbad61

Browse files
committed
chore: fix top comments and improve the summaries (EGWALKER)
1 parent ed45a98 commit edbad61

File tree

5 files changed

+9
-18
lines changed

5 files changed

+9
-18
lines changed

benchmarks/arteval_bench/data/benchmark/eurosys25_egwalker/_agent_eval/main.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/usr/bin/env python3
2-
"""Runs environment setup, build, benchmark prep, and experiment runs checks for EGWALKER."""
2+
"""Runs environment setup, build, benchmark prep, and experiment runs checks for EGWALKER (EuroSys'25)."""
33

44
from __future__ import annotations
55

benchmarks/arteval_bench/data/benchmark/eurosys25_egwalker/_agent_eval/oracle_artifact_build.py

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,8 @@
1-
"""Artifact build oracle for the Eurosys'25 EGWALKER artifact.
1+
"""Artifact build oracle for EGWALKER (EuroSys'25).
22
33
Validates:
44
- Repository working directory exists.
55
- Build commands execute successfully (captures stdout/stderr/return code).
6-
7-
This implementation maximizes reuse of the base primitives:
8-
- Uses OracleArtifactBuildBase orchestration unchanged.
9-
- Expresses all checks as BuildCommandRequirement instances.
10-
- Avoids re-implementing command execution, output truncation, or cwd-escape logic.
116
"""
127

138
from __future__ import annotations

benchmarks/arteval_bench/data/benchmark/eurosys25_egwalker/_agent_eval/oracle_benchmark_prep.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
#!/usr/bin/env python3
2-
"""Benchmark preparation oracle for _agent_eval bundles.
1+
"""Benchmark preparation oracle for EGWALKER (EuroSys'25).
32
43
Validates:
54
- Dataset manifest JSON is readable and well-formed.

benchmarks/arteval_bench/data/benchmark/eurosys25_egwalker/_agent_eval/oracle_env_setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Environment setup oracle for the Eurosys'25 EGWALKER bundle.
1+
"""Environment setup oracle for EGWALKER (EuroSys'25).
22
33
Validates:
44
- Required tools and minimum versions where applicable.

benchmarks/arteval_bench/data/benchmark/eurosys25_egwalker/_agent_eval/oracle_experiment_runs.py

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,9 @@
1-
#!/usr/bin/env python3
2-
"""Experiment runs oracle for the EuroSys'25 EGWALKER artifact.
1+
"""Experiment runs oracle for EGWALKER (EuroSys'25).
32
4-
This oracle compares experiment-produced timings against reference timings.
5-
6-
Design goal:
7-
- Keep artifact-specific parsing/alignment here.
8-
- Delegate *all numeric comparison logic* (similarity scoring + mismatch reporting)
9-
to evaluator.oracle_experiment_runs_primitives base requirements.
3+
Validates:
4+
- Timing results file can be read and parsed.
5+
- Ground-truth reference timings file exists and can be read.
6+
- Observed timings meet the configured similarity threshold against reference timings.
107
"""
118

129
from __future__ import annotations

0 commit comments

Comments
 (0)