-
Notifications
You must be signed in to change notification settings - Fork 79
Open
Labels
Description
Parent Issue
Part of #880 — Introduce Batch-Level Learning and Strategy Memory
Context
CliConsoleManager currently prints per-task headers and process output but no run-level summary. SpecTaskRunnerListener.onRunFinished() receives only completed, skipped, total, and finalState counts. There is no telemetry breakdown at the end of a batch run.
Task
Add a comprehensive batch run summary to the console and listener interface.
Key Changes
-
Create
BatchRunSummarymodel:@Data public class BatchRunSummary { String batchId; // UUID for this run Instant startedAt; Instant finishedAt; long totalExecutionTimeMs; int totalTasks; int completedTasks; int skippedTasks; int failedTasks; long totalInputTokens; long totalOutputTokens; double totalCost; RunnerState finalState; List<TaskExecutionMetrics> taskMetrics; // per-task breakdown Map<TaskResultStatus, Integer> statusBreakdown; }
-
Build summary in
SpecTaskRunnerService.finish():- Aggregate all
TaskExecutionMetricscollected during run - Calculate totals and breakdowns
- Pass to listener via extended
onRunFinished()
- Aggregate all
-
Print to
CliConsoleManager:═══════════════════════════════════════════════ Batch Run Summary ═══════════════════════════════════════════════ Total tasks: 8 (6 completed, 1 skipped, 1 failed) Total time: 4m 32s Total tokens: 145,230 in / 67,890 out Total cost: $0.47 Per-task breakdown: ✓ TASK-1 Set up schema 45s $0.05 ✓ TASK-2 Create User model 32s $0.04 ✗ TASK-3 Email validation — (timeout) ✓ TASK-4 REST endpoints 68s $0.12 ... ═══════════════════════════════════════════════ -
Extend
SpecTaskRunnerListener:default void onRunFinished(BatchRunSummary summary) { onRunFinished(summary.getCompletedTasks(), summary.getSkippedTasks(), summary.getTotalTasks(), summary.getFinalState()); }
Files to Create / Modify
src/main/java/com/devoxx/genie/model/spec/BatchRunSummary.java(new)src/main/java/com/devoxx/genie/service/spec/SpecTaskRunnerService.javasrc/main/java/com/devoxx/genie/service/spec/SpecTaskRunnerListener.javasrc/main/java/com/devoxx/genie/service/cli/CliConsoleManager.java
Dependencies
- Requires: TaskExecutionMetrics ([FEATURE] #880 - Create TaskExecutionMetrics model for per-task telemetry #910)
Acceptance Criteria
-
BatchRunSummaryaggregates all per-task metrics - Console prints formatted summary at end of every batch run
- Per-task breakdown shows status, timing, and cost
- Listener receives full summary (backward compatible default method)
- Summary printed for both CLI and LLM mode runs
Reactions are currently unavailable