Debouncing file removal

[chreekat]

What happened?

I am watching an individual file:

ghciwatch --command "cabal repl day14" --watch day14.hs --error-file errors.err --clear --test-ghci main

After making edits in vim, I get this behavior from ghciwatch:

• Removing modules from ghci:
  • day14.hs

Ok, 8 modules loaded.
• All good! Finished reloading in 70.28ms
• Running test command command=main

<interactive>:7:1: error: [GHC-88464]
    Variable not in scope: main
    Suggested fix: Perhaps use ‘min’ (imported from Prelude)
• Finished running tests in 55.47ms

This kills the ghciwatch workflow, naturally. :)

What did you expect to happen?

ghciwatch should see the file change (not get deleted) and reload it.

Hypothesis

Here's what I think is happening. For atomicity, vim does a little dance when writing a file: Write to a temp file, delete the real file, move the temp file to the real file. I assume ghciwatch sees the deletion and stops tracking the file. I've seen this problem with similar tool in the past.

If that is the case (and I haven't looked, sorry), then it might be better to debounce the deletion: Keep tracking the file until the debounce period is over.

Workaround

Watch the directory instead of the file.

ghciwatch --command 'ghci main.hs' --watch . --test-ghci main

Steps to reproduce the issue

$ echo 'main = putStrLn "hello world!"' > main.hs
$ ghciwatch main.hs --test-ghci main             

(in a separate shell)

$ vim +wq main.hs

The version of ghciwatch with the bug

ghciwatch 1.0.1

[9999years]

The default debouncing interval is 500ms; try --debounce 1s or similar to increase the value, which should provide better reliability at the cost of latency (debouncing to one second delays reloads by one second).

You might try --poll 500ms or similar to use the polling-based file watching, which tends to be slower but more reliable (especially on macOS).

Unfortunately file-watching libraries are very unreliable and full of bugs, even notify which is pretty much best-in-class (see the list of known & unsolvable problems in those docs!). My understanding is that this is a consequence of filesystems themselves being fairly unreliable.

[9999years]

BTW it's always helpful to hear about the performance characteristics of ghciwatch so I appreciate this ticket; I'm closing it because I don't think there's too much I can do right now at least. Feel free to comment if you have more information.

[chreekat]

Ok yeah it's exactly as described in notify-rs/notify#247 (comment) . I just hadn't realized that vim's behavior was still unsupported. :P

[9999years]

Ahhh that's frustrating. FWIW I use ghciwatch with Neovim and it works well, but I'm also on macOS so I'm using the --poll file watching system. Let me know if that's more reliable for you.