Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(autofix): Use an LLM to apply edits to full files in the coding step #1895

Merged
merged 5 commits into from
Feb 10, 2025

Conversation

roaga
Copy link
Member

@roaga roaga commented Feb 9, 2025

Inspired by Cursor's apply model:

  • remove strict diff requirements from the coding prompts
  • apply unstructured code suggestions to full files with an LLM (gpt-4o-mini) using speculative decoding (300-500 tokens per second from my observations)
  • automatically create the diff using unifieddifflib
  • process code changes in parallel

Pros:

  • improved fix evals by 4 points up to 41% (after all these changes, we're up to 88% correct RC and 41% correct fix on the new evals)
  • removes unreliability of incorrect diffs, missing changes, and poorly applied changes (which should solve some of the issues that have been plaguing us forever, including recent reports)
  • usually insignificant difference in overall latency. It takes a few seconds to apply, but reduces tokens the main agent needs to write and removes incorrect diff fixer.

Cons:

  • adds more latency if editing a very big file (in the future, we could consider using a smaller model on Fireworks with predicted output for even more speed; that's what Cursor does)

@roaga roaga changed the base branch from main to autofix/semantic-file-search February 9, 2025 20:07
current_diff_contents.append(line)
continue

# Skip empty lines that are immediately before or after a header as they are not part of the chunk
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this fixes the bug we've had for a while where it adds random empty lines in the diff

@roaga roaga marked this pull request as ready for review February 10, 2025 16:32
@roaga roaga requested a review from a team as a code owner February 10, 2025 16:32
@roaga roaga requested a review from jennmueng February 10, 2025 16:32
Base automatically changed from autofix/semantic-file-search to autofix/autocorrect-file-paths February 10, 2025 18:15
@roaga roaga merged commit 6bea5b9 into autofix/autocorrect-file-paths Feb 10, 2025
11 checks passed
@roaga roaga deleted the autofix/apply-model branch February 10, 2025 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants