Quitting in the middle of gtags generating raises an error

[rockyzhang24]

Describe the bug
In a very large repo like https://github.com/torvalds/linux, when I open a file, gtags will be generating automatically but it takes a while. In the middle of the gtags generating process, I run :q to quit Neovim. Then I open this file again, it shows gutentags: gtags-cscope job failed, returned: 1 at the bottom. When I try to run :GutentagsUpdate! to re-generate the gtags, this error shows again.

To make it work, I need go to the directory storing the tags and delete the folder for gtags of this repo. Then I open a file of the repo, gtags will be generated again without that error.

This means that, in a very large repo, I cannot quit and have to stay there until the gtags generating process finishes.

Steps to reproduce
See above.

Share your setup

• What OS and version of Vim are you using?
  macOS 12.2
  nvim 0.6
• What version of ctags, gtags, or whatever do you have installed?
  Universal Ctags 5.9.0,
  gtags (Global) 6.6.8
• Are you using g:gutentags_cache_dir?
  Yes. I set it to ~/.cache/tags/. However, if I don't set it, this issue still exists.

Post the logs

gutentags: [job stdout]: ['Locking tags file...', 'Running ctags on whole project', 'ctags -f "/Users/rockyzhang/.cache/tags/Users-rockyzhang-Downloads-linux-.tags.temp" "--options=/Users/rockyzha
ng/.config/nvim/pack/minpac/start/vim-gutentags/res/ctags_recursive.options"  --fields=+niazS --extras=+q --exclude=@/Users/rockyzhang/.cache/tags/_wildignore.options "/Users/rockyzhang/Downloads/
linux"', '']                                                                                                                                                                                        
gutentags: [job stdout]: ['']                                                                                                                                                                       
gutentags: [job stderr]: ['gtags: /Users/rockyzhang/.cache/tags/Users-rockyzhang-Downloads-linux/GTAGS seems corrupted.', '']                                                                       
gutentags: [job stderr]: ['']                                                                                                                                                                       
gutentags: Finished gtags_cscope job.                                                                                                                                                               
gutentags: gtags-cscope job failed, returned: 1                                                                                                                                                     
Press ENTER or type command to continue 

Additional context
Nothing.

Thank you very much.

[ludovicchabant]

In theory, Neovim is supposed to kill child processes on exit, and the child process' script has a trap command in there to clean up files... I wonder why that doesn't work. I don't use neovim so you might have to troubleshoot if it has changed somehow, or if a bug got fixed with a later version.

Incidentally, I was testing some stuff on Windows and because cmd.exe doesn't have trap, I have to add a Vim-only way of cleaning up temporary files when the update process gets interrupted. I'm going to limit it to Windows platforms, but it might be easy to enable it elsewhere too if Vim doesn't behave.

[rockyzhang24]

Okay, thank you.

I'm going to limit it to Windows platforms, but it might be easy to enable it elsewhere too if Vim doesn't behave.

I have no idea how to fix it in Neovim. Could you please adapt it to Neovim? When we exit Neovim before the finish of gtags generation, delete the gtags files before exit so that they will be regenerated next time.

Another idea is to expose two APIs. One is to output the gtags folder of the project. The other is to output whether gtags generation is finished or not. Then we can use autocmd to delete the folder if the generation is not finished. Any thoughts?

Thank you very much @ludovicchabant

[resoliwan]

In my case, I installed https://github.com/universal-ctags/homebrew-universal-ctags at my mac.
After that the error message is gone.

[iLeeWell]

I've discovered the problem and submitted a PR(#359), maybe it helps you @rockyzhang24 .

[smemsh]

@iLeeWell this is indeed the issue (for me, with vim-9.1.418 and gtags-6.6.12). Your fix does stifle the error message (thanks), but @rockyzhang24 is right that it means we have to start all over again next time we start. There is not much choice but to let the generation run until completion, or else the database will never be generated. For a large source tree this can take a long time. I'm not sure if it's possible, but there should be some way to have partial results saved so it could be closer and closer each time the editor was started, and eventually finish.

The error also serves as a warning for when there is some problem generating the files, for example I'm here because my gtags was hanging in a certain directory and would never finish, turns out to be a Pygments issue. I had to disable Pygments as a GNU global method. If the GTAGS, GRTAGS, GPATH files got deleted every time, there would have been no way to know a problem existed and the db would never have been generated.

So I think maybe the delete should occur as in the patch, with the following caveats:

1. some warning should still happen upon delete of the zero-byte files
2. there should be some sort of [very long] timeout on update script completion, with warning issued, and
3. if possible, partial db gen results should be saved on quit and resume on next start (maybe this is impossible).

the child process' script has a trap command in there to clean up files... I wonder why that doesn't work

 $ grep -lsr trap plat/unix
plat/unix/update_scopedb.sh
plat/unix/update_tags.sh
plat/unix/update_pyscopedb.sh

@ludovicchabant no trap in update_gtags.sh, only the others.

[smemsh]

I added file-delete traps to update_gtags.sh, like ctags has already, and made locking work correctly (previously, locking was unimplemented or broken), in #361. it probably means #359 would not be needed.