Running instruction?

[jacklxc]

Hi, is there any instruction to run this refactor benchmark?

[paul-gauthier]

Thanks for your interest in aider and the refactor benchmark.

You can run it like the main aider benchmark:

https://github.com/Aider-AI/aider/tree/main/benchmark

You just need to use --exercises-dir to point it at the refactor exercises.

[jacklxc]

Thanks for your quick reply. I was diving into the source code but still confused. Can you explain what is the outputs of the LLM and how is the output applied to the original file? Is the LLM output the full updated file content or only the difference between the original file content and the updated file content?

I am trying to use my own LLM for this benchmark, thank you.