[EXPERIMENT!] Use AI to migrate Solr tests from SolrJettyTestBase to using SolrJettyTestRule #3947
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rTestCaseJ4 I used the prompt that I previously checked in, and pretty much just let it go to town. I had to help it a bit on the BasicHttpSolrClientTest. tests_not_migrated.md represents the ones that didn't go on the first pass.
…ious batch They all look pretty straightforward however.
Contributor
Author
|
Some lessons learned:
|
Contributor
|
Big changes like this should be a multi-step journey. Anticipate that and ask the AI where to start and just stay focused there. For example look at SolrJettyTestBase and focus on removing that. Perhaps even that might be too much for one PR, so narrow further on RestTestBae, for example. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a bit of an experiment. I wanted to see if I could reduce the amount of effort to get our tests updated. Over time we've been accumulating tech debt in our tests, and addressing that manually appears pretty difficult!
In this PR I've checked in my prompt
prompt_to_use.mdthat I used in VS Code. My thought is that the most important aspect of this PR is crafting a really great prompt. Instead of reviewing and commenting on the changes made per individual test file, what I really want is review of the overall change, and feed that review BACK into my prompt. Then I will re-run the improved prompt and that hopefully will output the final set of changes. Maybe in this PR, maybe in another fresh PR with improved prompt.As an example of feeding back input, after running everything I noticed in one test it hardcoded a
127.0.0.1url, and so I addedPlease do not create hard coded urls like 127.0.0.1 in the tests.to prevent that in the future.There are three files that start with
MIGRATION_*.mdand they are somewhat messy AI generated status files. I wanted it to maintaintests_not_migrated.mdas the but it kind of forgot that file....