Feature request: add an option to summarize and analyze agent trace with LLM after each run

[KaminariOS]

#101

I have developed some scripts to do this, but it may be valuable to integrate this into the eval pipeline.

Currently, we lack observability in agent behaviors:

• Traces are long, and nobody bothers to read them
• Traces alone are not enough. We also need the evaluation and fault-injection logic to understand the run better.
• Many times, the agent did the right thing, but there are bugs in the benchmark itself.

For summary and analysis, we can feed the context to the most advanced LLM with a better reasoning capability and a larger context length at a low cost(because it is one-step).

[gaganso]

Hi @KaminariOS, thanks for raising this! Observability in agents is very important. I agree with all your points. For (3), can you please make a list of such bugs so that they can be fixed?

For (2), what do you think of a ground truth trajectory/path based on fault-injection logic and doing some approximate match of the agent's trajectory with this? Just a rough idea and this could revamp how we do evaluation - not just on outcome based but also based on trajectory. What do you think?

[KaminariOS]

Hi @KaminariOS, thanks for raising this! Observability in agents is very important. I agree with all your points. For (3), can you please make a list of such bugs so that they can be fixed?

I have created other issues for the bugs I found, and some of them have been fixed.

For (2), what do you think of a ground truth trajectory/path based on fault-injection logic and doing some approximate match of the agent's trajectory with this? Just a rough idea and this could revamp how we do evaluation - not just on outcome based but also based on trajectory. What do you think?

What if there is no ground truth? What if there are multiple different paths to the same solution?

LLM-as-judge may be a more generablizable solution.

[gaganso]

@KaminariOS Ground truth should exist for effective evaluation. We will only include problems that have a solution/ground truth.

I agree on multiple different paths for the same solution and that's why I was talking about an approximate match. It is an interesting problem.

I think we already have LLM-as-judge and we could add more context like trajectory path and the ground truth trajectory. But yeah, this also involves writing down the ground truth trajectory. Would you be willing to work on this? May be ground truth trajectory for 1 or 2 simple problems.