Race condition? How do I stop Obsidian from overwriting changes to files caused by a shell command?

[wealthychef1]

Hello, I have a shell script I've been using for a while now, called "smaller-images.sh", which I use to shrink the images in a file. I have it "hooked up" to a Shell Command called "Shrink Images in File" and have used it successfully for many files to resize images.

#!/usr/bin/env bash

img_size="$1"
md_file="$2"

echovals img_size md_file # for debugging

e1='s=\[(!\[[^\]*]\([^\)]+\))]\(.+\)=\1='
e2='s/!\[[^]]*]([^\)]*)(jpeg|png|jpg|gif)\)/!['"${img_size}"']\1\2)/'
gsed -Ei -e "${e1}" -e "${e2}" "${md_file}" # modify the file in place

# for debugging, show what happened
grep -E 'jpeg|png|jpg|gif' "${md_file}" 

Here is the "Shrink Images in File" command:

It prompts for a new image size, calls my little script and voila! Life is a little better afterwards.
A couple days ago this was working fine.
Today it started mysteriously failing, and what I'm seeing is that while the script works properly, Obsidian does not notice the changes caused by the script, and instead overwrites them. Obsidian seems to flush the file to disk as soon as I move to another file in Obsidian's file browser, but since it doesn't know the file has changed, this is a reversion. It's like Obsidian has a cache of the file in memory and writes it out to disk whenever I move away from viewing the file in Obsidian. The grep statement at the end of the script allows me to confirm this is what happens. Also, when I quit Obsidian and run the script by hand, then launch Obsidian, the changes are there and they stick.

1. Is this new behavior in Obsidian or perhaps something changed in Shell Commands?
2. How can I modify the existing file that's being viewed without the changes getting overwritten by Obsidian?
3. Could my Shell Command or script somehow signal Obsidian that the file has changed so it notices and updates its cache?

Thanks for any help!

[Taitava]

Hi! Thank you for you question 🙂.

1. Is this new behavior in Obsidian or perhaps something changed in Shell Commands?

I don't know Obsidian's updates well enough to say if they may affect this or not. There has been no significant changes to SC in the past few months. When was the last time you have updated the SC plugin? What version of SC and Obsidian are you using? And if you have updated SC recently, do you remember what was the version you had before? (The earlier version can be seen if you open the plugin's installation folder and look for files named like data-backup-version-0.7.0-before-upgrading-to-0.8.0.json, where 0.7.0 in this example is the from version, and 0.8.0 is the to version. You'll have different version numbers, though.)

2. How can I modify the existing file that's being viewed without the changes getting overwritten by Obsidian?

Hmm, modifying files with shell commands / scripts is a very common task and should be perfectly doable. I don't know why it has stopped working in your use case. But you can check the following:

• What is you shell command's stdout output channel? As your script is modifying the file directly, I think you should have output channel Ignore selected as the output channel for stdout in the shell command's settings. If you happen to have output channel Current file selected instead, then the SC plugin might write something to the currently open file after your shell command has finished executing. That might overwrite the content that your script file writes.
• Do you have any events turned on that might execute shell commands that write to files?

3. Could my Shell Command or script somehow signal Obsidian that the file has changed so it notices and updates its cache?

The Shell commands plugin does not use this kind of signaling. I don't even know if Obsidian's API has any method so that a plugin could tell Obsidian that files have changed and should be reloaded from disk. However, there should not be any need for that kind of signaling, because Obsidian has been very good in noticing all external changes to files automatically, at least according to my personal experience.

If you need further help, may I ask you to include the SC plugin's data.json file in this discussion, please? 🙂 It contains all the plugin's settings, and might help me to better troubleshoot the problem. You can read it through in a normal text editor before sending it, to make sure it doesn't contain any private information.

Also, to rule out some possible issues:

• Do you have your Obsidian vault stored on a network drive?
• Do you use any synchronising service for your Obsidian vault (e.g. Dropbox or Obsidian Sync)

[wealthychef1]

OK this morning I woke up intending to try to search for a plugin conflict. I disabled half my community plugins and the problem went away. But when I enabled them, the problem did not come back!

So now I'm in crazy land. I suppose maybe it was in the middle of a File Vault backup or something. Honestly, I can't imagine what could possibly be interfering with the "file changed" notification that every MacOS app seems to be capable of seeing. Seem filesystem related...

So it's "intermittent," the bane of all programmers. I will hunt more but right now I can't even reproduce the problem. 😖 🤷
Sorry to waste a bit of your time, thanks for the fast response. If I figure it out I will post the answer.

[Taitava]

No problem! 🙂 Just send a message if you encounter the problem again. 👍

[wealthychef1]

Thanks for your feedback, it was very helpful.