Adding project-wide type-aware search for ocaml code patterns (ocamlgrep)

[mjambon]

Dear project maintainers,

LexiFi created a tool called ocamlgrep (formerly known as grep_cmt) and has been using it internally for a while. For example, it allows a query such as (__ : int list) to locate all expressions of type int list in a project. We would like to make this tool more widely available and easy to use by the OCaml community. I'm looking into the possibility of integrating it into merlin and eventually in IDEs. I would try to do most of the work and would make it easy to maintain.

Ocamlgrep's development would benefit from being part of the merlin code base in the following ways:

• locating the project's cmt files reliably (ocamlgrep currently assumes a dune build using the default context)
• managing compiler-libs variations across OCaml versions (ocamlgrep currently works with a single OCaml version)

One approach would be to move ocamlgrep's matching code to the merlin project. It would be part of merlin-lib and a new merlin command would function like ocamlgrep (takes a user-specified pattern and returns matches in various files). A standalone ocamlgrep command could still be provided but it would be a thin wrapper around the merlin command.

Please let me know if this looks like a good idea and if you see any challenges.

[xvw]

Yes, when it was first presented, it seemed really interesting. Is it already possible to perform some kind of search with Sherlodoc? By using (for ocaml-eglot) the various commands: https://github.com/tarides/ocaml-eglot#search-for-values

However, ocamlgrep offers undeniable advantages (for searching beyond public APIs). Would @nojb agree to integrating ocamlgrep directly into Merlin?

[nojb]

Would @nojb agree to integrating ocamlgrep directly into Merlin?

Yes! To be clear: this proposal has full support from LexiFi. Indeed, @mjambon is collaborating with LexiFi to drive this forward. And we would all be more than happy to explore any venues or ideas proposed by the Merlin team on this subject.

[mjambon]

Here's a pull request that integrates ocamlgrep into merlin: #2064
It's big but most of the code was simply copied from the original repo.

Please let me know what I can do to facilitate the code review.

[voodoos]

Ah, I had a message I forgot to send yesterday 🤦‍♂️

To follow-up on this, I want to confirm the we would be happy to collaborate on having ocamlgrep-like features added  to Merlin. We can help with the implementation and review and also contribute to the client integrations, which we expect to be very similar to the existing UI for search-by-type. Does that sounds good to you ?

I was also going to propose to have a short meeting to discuss the technical aspects before committing to an implementation... but that didn't age well 🙂

When I first read the proposition I imagined that the matching algorithm would be added as a new analysis in Merlin, not as a separate binary. It doesn't feel that relevant to me to merge the two projects if it remains a separate binary. It looks like the currently proposed implementation does not use any of Merlin's internal helpers for artifact discovery and caching. There is also a re-implementation of the search for a Dune project root that Merlin already knows about by configuration. So the only benefits are the integration to Merlin's command line and the offloading of the work of supporting new versions of OCaml to the Merlin maintainers, which does not seem worth it compared to a separately packaged tool.

What I think could be worth it is to really dissolve cmt-grep into Merlin, which would allow you to share the artifacts caches with Merlin's, preventing loading multiple times the same cmi / cmt files. (The risk being that the server doesn't free the memory fast enough after loading all these cmis, but that should be tweakable.) You should also benefit from merlin configuration in your discovery of the root of the project, but I guess you still need to recursively traverse the build directory since cmt-grep searches in the entire project, not only the dependencies.

Also I think we need to think a bit about the relationship with search-by-type. Two (three with polarity search) slightly different query languages with different scopes is a bit awkward, what should we do about it ?

Finally, we don't have AI guidelines in the repository yet, but we have been discussing the subject a lot. I expect that they will say things along these lines: we accept PRs made with the help of AI, but, as for any other PR, the authors must have complete ownership and understanding of the changes they propose. That means these changes should be thoroughly reviewed by the authors before asking for maintainers reviews. We are also inclined to require that all PR descriptions and comments be written by the authors. I am not saying that it is not the case for this PR, it's just a heads'up, until we have official guidelines.

[nojb]

To follow-up on this, I want to confirm the we would be happy to collaborate on having ocamlgrep-like features added  to Merlin. We can help with the implementation and review and also contribute to the client integrations, which we expect to be very similar to the existing UI for search-by-type. Does that sounds good to you ?

I was also going to propose to have a short meeting to discuss the technical aspects before committing to an implementation... but that didn't age well 🙂

This sounds like an excellent idea, and we should still do it, I think. @mjambon: do you agree? If yes, I can make a poll to find a suitable time (I am not myself driving the implementation, but I would be interested in understanding the various technical choices for this project).

When I first read the proposition I imagined that the matching algorithm would be added as a new analysis in Merlin, not as a separate binary. It doesn't feel that relevant to me to merge the two projects if it remains a separate binary. It looks like the currently proposed implementation does not use any of Merlin's internal helpers for artifact discovery and caching. There is also a re-implementation of the search for a Dune project root that Merlin already knows about by configuration. So the only benefits are the integration to Merlin's command line and the offloading of the work of supporting new versions of OCaml to the Merlin maintainers, which does not seem worth it compared to a separately packaged tool.

I think this is spot on. I believe #2064 is a first attempt not really intended for direct inclusion. In offline discussion with @mjambon, I believe we agreed that the interesting part of integrating with Merlin was precisely to share much of the existing infrastructure that already exists in Merlin, as you mention above. So I definitely think the above is the right direction to take. Let's have a chat to synchronize next steps and spread knowledge around, shall we?

[mjambon]

@voodoos I apologize -- I'm just discovering your message right now after @nobj pinged me in private.

1. There's of course nothing shocking about the technical points you're bringing up.
2. A meeting would be great. My email address is martin@mjambon.com.

[mjambon]

I was also going to propose to have a short meeting to discuss the technical aspects before committing to an implementation... but that didn't age well 🙂

On the positive side, this code was cheap to create and it's easy to rearrange as needed.

[mjambon]

I imagined that the matching algorithm would be added as a new analysis in Merlin, not as a separate binary

In the current state of the PR, there are both. The standalone ocamlgrep command is a lightweight wrapper around merlin-lib. It doesn't have to be part of the merlin repo.

[mjambon]

Finally, we don't have AI guidelines in the repository yet, but we have been discussing the subject a lot. I expect that they will say things along these lines: we accept PRs made with the help of AI, but, as for any other PR, the authors must have complete ownership and understanding of the changes they propose. That means these changes should be thoroughly reviewed by the authors before asking for maintainers reviews. We are also inclined to require that all PR descriptions and comments be written by the authors. I am not saying that it is not the case for this PR, it's just a heads'up, until we have official guidelines.

This is all very reasonable.

Now consider something more extreme:

If the whole Merlin team is already actively using coding agents, you could consider adopting a more radical stance such as "we don't accept pull requests from outsiders but only detailed design proposals". The advantage is that it trades a back-and-forth between author and reviewer for a back-and-forth between the maintainer and their coding agent, the latter being faster and friction-free in my experience. Maybe this is best for a new AI-native project, though.

i.e. I don't mind deleting my whole PR and turning it into a pure GitHub Issue. I would more or less copy-paste the prompts I gave to Claude Code. They are much shorter than the resulting code diffs, so presumably they would be easier to review for someone already familiar with the project.