Remove nvim-treesitter dependency#390
Conversation
|
Thank you for contributing 🙏! Looks like are some merits to both this and #330, I will take bits of both. Since they are extensive I will likely merge them in bits and pieces where possible. Both PRs have mentioned performance issues (which likely do exist in head already) and which are not yet solved by either.
I am curious which approach do we think is more viable? Can you give any more detail on what changes would be required for async parsing (no need for a full PR for that yet)? |
Awesome c:.
I actually was talking about performance in an abstract way (because I haven't had any performance issues using
To make the parsing async, we would need to modify To take a callback (for when the parsing has finished) as a second argument. This would mean that
That same line also has a comment mentioning that instead of parsing the whole buffer, we could use I should also mention that async parsing wouldn't make the plugin faster, it would simply avoid freezing Neovim while treesitter is parsing the buffer. Something that could actually make the plugin faster is parsing a given range instead of the whole buffer (like I mentioned above). |
If I could hug you right now, I would. For this and for cleanup around old attach/detach routines - I totally overlooked them. Slotos#1 - this is my test branch/PR that I use for CI runs. I have a change that parses on a cursor there, which resolves some performance issues, but there's still a failure cascade in a "no match" handling, which can take a few minutes on a file I'm verifying against ( It also works correctly with injections, whereas The problem with memoizing results when parsing on a cursor position is that it introduces memory bloat. Most of memoized values will be used in one user action (there are multiple calls associated with it) never to be touched again.
Anyways, if you see good bits of my PR and want to merge them into yours - please do so. Ping me if you need help or just a rubber duck - I'll try to contribute as much as time allows. |
|
@Slotos any concerns if I go ahead and merge this PR? Based on your comments regarding "cleanup around old attach/detach routines" it seems this will be a bit cleaner to build on top of than your original. I would like to take your CI/CD updates, though. Just want to make sure any performance improvements you are planning could be built on top of this CR and not make it difficult for you. @TheLeoP Any guidance for users to migrate? It seems they just need to replace a call from nvim-treesitter.setup to matchup.setup, I will have to add some readme section on this migration. Doing nothing it seems the user's config would break right? |
Users simply need to add vim.g.matchup_treesitter_enabled = trueor g:matchup_treesitter_enabled = v:trueto their configs. They could also use the lua interface if they want to require("match-up").setup {
treesitter = {
enabled = true,
},
}It's worth noting that I'm using either Lua's
It shouldn't break, unless I'm missing something. It'll stop using treesitter by default and rely on the regex engine unless they enable treesitter as mentioned above. I don't know how |
|
There are three issues that pop out to me:
As a personal opinion, I believe treesitter functionality should be enabled by default. While treesitter is said to be experimental in neovim docs, its functionality has been a stable presence over the years, and all changes to |
Oh, I missed that, you are right. I first though that parsing the whole buffer including injections would be too costly. As mentioned previously, parsing only a certain range around the cursor is also feasible. On the case of parsing a range, iterating over all language trees would still be the right thigh to do.
It was already present in the plugin. It was just not documented and had some edge cases that I cleaned up. It's by no means necessary and I'm not against removing it. Sane defaults are there whether
Agree, but I though the sanest default would be to disable it and let users opt-in (just like it worked before). |
I agree- the treesitter capability should be enabled by default now and controlled just by g: variables (like in @TheLeoP's PR) as this plugin has for every other functionality. However, I think we should still offer an optional setup function which simply updates the corresponding g: variables because many neovim users expect this convenience. This would mean setting
Thanks, got it, we need @Slotos's version of this part. |
|
Hey! Stumbled across this: local function f(a) end
f(function()
local x = "("
end)The open bracket in the string This is with This is not the case on Seems to have been introduced in 829df75. Unfortunately I'm not familiar enough with treesitter or this project to dig much deeper :( Thanks for the great work! |
|
@peter-bread that's not because of this PR (as far as I can tell). That happens simply because there is no treesitter capture for a function call and the regex engine fails to correctly find the closing parenthesis. If you add the following capture to (function_call
(arguments
"(" @open.call
")" @close.call)) @scope.call |
|
Thanks! Also found these: C: int main() {
printf("}"); // } matches with { opening the block
}int main() {
printf(")"); // ) matches with ( opening the function call
}Go: func main() {
fmt.Println("}") // } matches with { opening the block
}func main() {
fmt.Println(")") // ) matches with ( opening the function call
}Bash: foo() {
echo "}" # } matches with { opening the block
}These seem to be fixed with the following queries, based off the one in this comment: C: (compound_statement
"{" @open.block
"}" @close.block) @scope.block(argument_list
"(" @open.call
")" @close.call) @scope.callGo: (block
"{" @open.block
"}" @close.block) @scope.block(argument_list
"(" @open.call
")" @close.call) @scope.callBash: (compound_statement
"{" @open.block
"}" @close.block) @scope.blockI'm assuming this is the case for other languages but I haven't tested any more. In all cases (with and without the extra queries), putting the cursor over the bracket inside a string highlights the incorrect match. However (with the queries), after pressing
These images show what happens as you press After some quick testing I found that these queries aren't needed on I realise this is a bit of a weird edge case but I'm wondering if more of the supported languages might need extra queries or if there's been some other fundamental change. |
|
Thanks for the example @peter-bread , I now understand the issue properly. The issue was an oversight on my side when modifying the vimscript side of the codebase. It should be working in the same was as |
|
Hello again! Strange one this time. There seems to be a bug triggered by the builtin I've tested this on Neovim 0.11.2 on both MacOS and Ubuntu via WSL2. With a minimal configuration, this happens on the second use of Minimal repro config-- Bootstrap lazy.nvim
local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not (vim.uv or vim.loop).fs_stat(lazypath) then
local lazyrepo = "https://github.com/folke/lazy.nvim.git"
local out = vim.fn.system({ "git", "clone", "--filter=blob:none", "--branch=stable", lazyrepo, lazypath })
if vim.v.shell_error ~= 0 then
vim.api.nvim_echo({
{ "Failed to clone lazy.nvim:\n", "ErrorMsg" },
{ out, "WarningMsg" },
{ "\nPress any key to exit..." },
}, true, {})
vim.fn.getchar()
os.exit(1)
end
end
vim.opt.rtp:prepend(lazypath)
-- Setup lazy.nvim
require("lazy").setup({
spec = {
{
url = "https://github.com/TheLeoP/vim-matchup",
branch = "update-treesitter",
},
{
"nvim-treesitter/nvim-treesitter",
branch = "main",
},
},
})
I traced the issue back to this line in I added some logging to Logging codefunction M.is_enabled(bufnr)
bufnr = bufnr or api.nvim_get_current_buf()
local lang = ts.language.get_lang(vim.bo[bufnr].filetype)
if not lang then
return false
end
+ local info = {
+ bufnr = bufnr,
+ name = vim.api.nvim_buf_get_name(bufnr),
+ filetype = vim.bo[bufnr].filetype,
+ buftype = vim.bo[bufnr].buftype,
+ bufhidden = vim.bo[bufnr].bufhidden,
+ swapfile = vim.bo[bufnr].swapfile,
+ modifiable = vim.bo[bufnr].modifiable,
+ readonly = vim.bo[bufnr].readonly,
+ loaded = vim.api.nvim_buf_is_loaded(bufnr),
+ listed = vim.bo[bufnr].buflisted,
+ lines = vim.api.nvim_buf_get_lines(bufnr, 0, 5, false),
+ changedtick = vim.api.nvim_buf_get_changedtick(bufnr),
+ }
+
+ print(vim.inspect(info))
local _, err = ts.get_parser(bufnr, nil, { error = false })
if err then
return false
end
return is_enabled(lang, bufnr)
endI was also careful to use Below are two videos, one from a Lua file (which works fine) and one from a Go file (which throws an error): Lua: lua.movGo: go.movIn both cases, the main file is buffer 1, as seen in the first logging output. After pressing A quick fix is just to put the following check somewhere in if not vim.api.nvim_buf_is_loaded(bufnr) then
return false
endSetting Questions:
Thanks! |
In my opinion, it looks good enough. I also don't know why the builtin
I lack context of the bigger It's strange that the |
|
btw, thanks for the detailed research and report. It helps a lot, @peter-bread :D |
|
The parsing issue should be fixed now. I also added documentation for the new I chose I also rebased the branch on top of master and force-pushed it |
|
I tried testing things on a fresh As an aside, I noticed something while trying to figure out what I was doing wrong. You're using Brief explanation of my previous statement - function That said, I've been throwing various scenarios at the problem for an hour or so and I don't see a reasonable one where parsing on a range would create a realistic breakage. Maybe there's something possible with disjointed injections (disconnected chunks of code in different language that are joined into a single tree), but I'm not even sure whether they are still a thing, so it's best to leave it until and if a real breakage happens. I also remember that range parsing is used to optimize syntax highlighting, which I took to mean that parse can yield a partial tree, but that's a big assumption on my part that I never took time to properly verify. Finally, I apologize for stalling things. This branch is in a great state, you've meticulously cleaned obsolete code pathways, and in my opinion it can be merged as is. I won't say no to a fix for nightly breakage, but I took a bleeding edge path with full knowledge of its pitfalls and I can fix things at my own pace with no complaints. |
certainly, no concern here |
Don't worry. nvim nightly has had a lot of regressions around treesitter lately. One that affected me recently was neovim/neovim#34631
That's code that I haven't touch, so I don't feel confident doing that change. Thanks for the suggestion anyway c: .
There's no need to apologize. Thanks for your help and I appreciate the effort of making sure the PR is working as expected. |
|
I think this PR is ready to be merged/further comments @andymass |
|
@TheLeoP Thanks! I'm good to merge this soon. I notice it is failing tests, not sure if they are concerning or blocking. |
|
A bit of self-promotion, docker test runner commit from my obsolete PR can help a lot with local development. No need to include it in the PR, just plop it on top of your branch and remove when you’re done. I can’t write code without tests, it was an absolute must-have for me. |
|
@Slotos yes, totally agreed with that idea |
|
I've rebased my branch on top of @Slotos' docker commit and ran btw thanks for your help @Slotos |
|
@TheLeoP are there likely to be users of neovim 0.10.1 or does everyone use 0.11? Note that head also seems to have failed. |
|
The current stable is 0.11
Lucas
…On Mon, 7 Jul 2025, 21:51 Andy Massimino, ***@***.***> wrote:
*andymass* left a comment (andymass/vim-matchup#390)
<#390 (comment)>
@TheLeoP <https://github.com/TheLeoP> are there likely to be users of
neovim 0.10.1 or does everyone use 0.11? Note that head also seems to have
failed.
—
Reply to this email directly, view it on GitHub
<#390 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJGX4HBJHLBLNJ2OQYDRL33HLMWVAVCNFSM6AAAAAB6G6VYVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTANBWGQ4DQNJZGU>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
I found why I couldn't reproduce the issues locally. The env var |
Co-authored-by: Dmytro Soltys <soap@slotos.net>
Co-authored-by: Dmytro Soltys <soap@slotos.net>
Not everyone will ever be in the same version. I guess that most non-novices will be using either Ultimately, the decisions is yours, @andymass . For what is worth, the plugin mentions in both the README and matchup.txt that only the latest stable version of Neovim is supported for the treesitter integration. |
|
Fantastic, merging! Thanks again all for the extensive work.
That's true, though I am concerned someone will update the plugin who was not using treesitter, which is now default on, and may get errors. We'll see if anyone has such issues. |
|
@TheLeoP I decided to guard the enablement behind 0.11.2, for with it, it seems anyone not having used nvim-treesitter would find an error. 0aa22bc#diff-8a95a1a9bad487ba155641945b311ad079b2ce106d1ba45a502cb751aa4e181fR80 |
|
@TheLeoP Ah, I just noticed a regression Putting cursor in if, before the merge shows the "end arrow" at the return 3. |
|
@andymass like this right? I think the problem is simply that I disabled vim-matchup/autoload/matchup.vim Line 85 in 6193440 Adding either vim.g.matchup_treesitter_disable_virtual_text = falseor g:matchup_treesitter_disable_virtual_text = v:falseto your configuration will make it work as before. I can open a PR to make it |




I started working on this without realizing that #330 existed :p
This PR:
mainbranch).setupwrapper (that simply definesg:vimscript variables).g:vimscript variables).syntax.luamodule). Now this plugin parses the buffer if necesary (and could even use theg:matchup_delim_stoplineif we could know which window is being used to get the cursor location. An option would be to always use the current window, but I don't know enough of the codebase to say if that would work).make-rangedirective. Instead, it relies on the fact that treesitter matches can capture multiple nodes under the same name. So, a range can be obtained from the first and last nodes with the same name on a given match%between the start and end of Lua tables because I was getting mad at an edge case that the regex based engine couldn't catch on a file in my personal config (https://github.com/TheLeoP/nvim-config/blob/1ba78ab0a268fd5f8e3dc371a82e1af44c829fee/lua/personal/emmet.lua#L27-L51)Just
memoizewas copied fromnvim-treesitter, the wholequery.luadependency was removed in favor of a smaller (and non-nvim-treesitter dependant) implementation of the functions used by this plugin.This PR does not
This PR lacks documentation for the changes introduced in it, but I wanted to see what everyone thinks of it before commit further to it.
Additionally, since treesitter parsing can now be async, I could make use of it in follow up PR if deemed necessary.