Increase mitigation eval success rate by providing localization info in the init prompts

[KaminariOS]

AIOpsLab/aiopslab/orchestrator/tasks/mitigation.py

Line 27 in 957c550

self.task_desc = """\

Currently, the mitigation tasks have the lowest success rate(18%) when evaluated with GPT-4o-mini, ReAct.

Failed traces revealed that sometimes the agent could not even locate the faulty service correctly.

To isolate and evaluate mitigation itself, the agent should be given the localization info at the beginning.

Even if the agent manages to locate the faulty service, the localization process(reading long logs etc) takes up the context window and degrades agent performance.

[gaganso]

Hi @KaminariOS, good point! Considering that we evaluate agents for each task separately, this makes sense. Would you be willing to work on this?

Also, do you think we can have an end-to-end task (including detection, localization, rca, and mitigation) for problems that can support this?

[KaminariOS]

Hi @KaminariOS, good point! Considering that we evaluate agents for each task separately, this makes sense. Would you be willing to work on this?

Sure, I can work on it when I have time.

Also, do you think we can have an end-to-end task (including detection, localization, rca, and mitigation) for problems that can support this?

For evaluation purposes, a full evaluation is not needed because all subtasks combined are effectively "end-to-end".

And for a mitigation agent, does it even care about the reasoning process of localization or detection? It just needs the result: the faulty location. Redundant context is not just useless but harmful.

It is like refactoring a large function into multiple small functions.