Add Triage agent testsuite #367
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a test suite for the triage agent and a mechanism for gathering metrics from agent runs. The changes are well-structured, with a new MetricsMiddleware for collecting performance data and a comprehensive e2e test suite using pytest. The refactoring of the triage agent to extract the workflow logic into a separate run_workflow function is a great improvement for testability. My review includes suggestions to improve the robustness and completeness of the new tests, along with a minor cleanup in the metrics middleware.
TomasKorbar
force-pushed
the
triage_testsuite
branch
from
0cd910d to
03b8d89
Compare
TomasKorbar
force-pushed
the
triage_testsuite
branch
3 times, most recently
from
cc2660b to
b0c835f
Compare
TomasKorbar
force-pushed
the
triage_testsuite
branch
from
b0c835f to
65199d8
Compare
nforro
left a comment
nforro
left a comment
There was a problem hiding this comment.
I haven't been able to run the tests successfully, still getting Error while writing mock up Jira issue [Errno 13] Permission denied: 'mcp_server/tests/data/RHEL-XXXXX' despite running with DRY_RUN=true.
Can be executed by `make run-triage-agent-e2e-tests` Currently none of the tests reliably passes, but each of them provides insight into portion of process that is not working.
TomasKorbar
force-pushed
the
triage_testsuite
branch
from
65199d8 to
c74d0f1
Compare
Very nice catch, i changed the permissions and then did not test the suite without them in place. Should work now. I also fixed dry run application in add_jira_comment tool. |
2 participants
This pull request is the initial proposal for test suite of triage agent and mechanism for gathering of metrics from agent runs.
Related: https://github.com/packit/jotnar/issues/233