Handle the pattern field in ctags/readtags output

[dramforever]

Some ctags files have a search pattern in the "tagaddress" field for robustness against file changes. This allows us to check if the file has changed and has caused the line to move, and possibly still find it based on the pattern. Implement this feature.

Since this feature is a performance hit, add the
ctags-companion.usePatternField option to allow toggling it off.

---

This closes #59. There could still be minor problems, but I have been using this to browse Linux source code and it has worked reasonably well for me. Please take a look.

[gediminasz]

Hi, thank you for the contribution. I will try to have a look as soon as I can.

[gediminasz]

Oh, CI is failing due to npm audit. Feel free to run make update and commit the updated lock file. Or I can fix it later and you can then rebase on master.

[dramforever]

Hi, I've included an "Update dependencies" commit. Hope that works for you.

By the way, sorry for the many updates after opening the PR. As mentioned earlier I only noticed I forgot to do the tests after opening this PR so there was more extra stuff than I expected to have submitted.

[gediminasz]

By the way, sorry for the many updates after opening the PR. As mentioned earlier I only noticed I forgot to do the tests after opening this PR so there was more extra stuff than I expected to have submitted.

No worries, been there myself :)

[dramforever]

There, that should be enough coverage...

[gediminasz]

Sorry I'm taking my time with this PR but it's kinda big for a repo as small as this one and I try to be careful when adding new functionality...

In docs it says:

Client tools should use the -E option, which keeps the escape sequences in the tags file, so the only field that could contain tabs is the pattern field.

    -E | --escape-output
        Escape characters like tabs in output as described in tags(5).

Should we update the default readtags command to include -E? (settings readtagsGoToDefinitionCommand and readtagsGoToSymbolInWorkspaceCommand)

[masatake]

The following command line illustrates the effect of -E option of readtags.

$ printf 'the tile including \t tab\n----------------------------\n' 
the tile including 	 tab
----------------------------
$ printf 'the tile including \t tab\n----------------------------\n'  > foo.rst
$ ./ctags -o -  foo.rst | ./readtags -t - -l
the tile including 	 tab	foo.rst	/^the tile including 	 tab$/
$ ./ctags -o -  foo.rst | ./readtags -t - -l -E
the tile including \t tab	foo.rst	/^the tile including 	 tab$/

[dramforever]

Sorry I'm taking my time with this PR but it's kinda big for a repo as small as this one and I try to be careful when adding new functionality...

That's very understandable. Thank you :) for your review as well.

Client tools should use the -E option, which keeps the escape sequences in the tags file, so the only field that could contain tabs is the pattern field.

I hadn't changed it to preserve the current behavior, but it seems reasonable to me. But maybe it's out of scope for this PR?

Notably, the pattern field is still unescaped, so the handling I wrote would still be needed even with -E

[gediminasz]

I hadn't changed it to preserve the current behavior, but it seems reasonable to me. But maybe it's out of scope for this PR?

Yes, let's keep it out of scope. I just checked and it is broken on master, so I'll have to look into that separately.

[gediminasz]

I apologise @dramforever, you must have spent a lot of your precious time on this, but after a couple of weeks of marinating I decided to not take in this feature.

When creating the original issue #59 I did not know about all of the intricacies. Turns out it's a non trivial change, and I don't know if I'll have the capacity to maintain it in the future.

If you still have interest, I would suggest exploring alternative approaches:

• Instead of using patterns, perhaps we can simply look for the closest occurrence of the literal tag name to the originally indexed position. Less robust but the implementation should be simpler. I would also consider showing a warning notification informing that tags file is out of date.
• Or perhaps even better: during resolveWorkspaceSymbol run ctags on the fly on the file in question (similarly to provideDocumentSymbols). Should be pretty robust. Need to check if it would introduce any delays noticeable to the user.

[dramforever]

Hi, this is understandable. I'm happy enough running a fork as well - your extension, by virtue of using readtags is the only one I've found that handles huge ctags files. Thank you for the care as well.