Hey guys, love the project and find the performance and functionality awesome!
The biggest issue I've encountered is that the opencode agent harness pretty much never chooses to call the osgrep tool on its own.
Problem with adding explicit prompting on top with something like AGENTS.md is that you are adding more, and confusing context every request.
Critiques with the prompt from me reading it, that could be used as a starting point / ideas for improving tool usage rate:
- There is too much specificity about when to/not call this tool.
"grep for exact strings, osgrep for concepts" is quite abstract. LLMs have a huge bias towards both the "grep" tokens but also against semantic code search (so new) and what that means.
-> My hypothesis is that not including any of these things or even the word grep for a simpler description would do a lot of heavy lifting. Something like: "Finds files/code quickly by asking a question, for example "where is auth handled?".
- Dead and confusing tokens in prompt.
Since the tool description prompt for opencode seems to be auto generated or copy pasted over from a Claude skill, just don't include --- name: description: allowed-tools: ---, and instead start the prompt right away. The `` string template used also contains "\n" and "\n\n" (new line) tokens at both the start and end of the prompt, assuming opencode later on does not perform a strip. In general a lot of trimming could be done to convey correct osgrep usage in a lot fewer tokens, unlike a skill, tool description prompts get bundled in to every LLM request, so making them smaller goes a long way.
-> My hypothesis is that because the tool description pattern for osgrep will look so different compared to any other tool due to this, it could be heavily contributing to this tool being ignored as noise. Markdown titles can also be quite confusing in tool descriptions. For reference, here is what all the other tool description prompts opencode uses look like, you can see how much the osgrep one would stick out.
Anyway, hope this does not come off too strongly, you guys did awesome and I'm really grateful to be using osgrep and would like to see it improve! Let me know if you'd be interest in a PR along these lines focused on improving the opencode tool prompt!
Hey guys, love the project and find the performance and functionality awesome!
The biggest issue I've encountered is that the opencode agent harness pretty much never chooses to call the osgrep tool on its own.
Problem with adding explicit prompting on top with something like AGENTS.md is that you are adding more, and confusing context every request.
Critiques with the prompt from me reading it, that could be used as a starting point / ideas for improving tool usage rate:
"grep for exact strings, osgrep for concepts" is quite abstract. LLMs have a huge bias towards both the "grep" tokens but also against semantic code search (so new) and what that means.
-> My hypothesis is that not including any of these things or even the word grep for a simpler description would do a lot of heavy lifting. Something like: "Finds files/code quickly by asking a question, for example "where is auth handled?".
Since the tool description prompt for opencode seems to be auto generated or copy pasted over from a Claude skill, just don't include --- name: description: allowed-tools: ---, and instead start the prompt right away. The `` string template used also contains "\n" and "\n\n" (new line) tokens at both the start and end of the prompt, assuming opencode later on does not perform a strip. In general a lot of trimming could be done to convey correct osgrep usage in a lot fewer tokens, unlike a skill, tool description prompts get bundled in to every LLM request, so making them smaller goes a long way.
-> My hypothesis is that because the tool description pattern for osgrep will look so different compared to any other tool due to this, it could be heavily contributing to this tool being ignored as noise. Markdown titles can also be quite confusing in tool descriptions. For reference, here is what all the other tool description prompts opencode uses look like, you can see how much the osgrep one would stick out.
Anyway, hope this does not come off too strongly, you guys did awesome and I'm really grateful to be using osgrep and would like to see it improve! Let me know if you'd be interest in a PR along these lines focused on improving the opencode tool prompt!