Realtime updates - Update fails sometimes when queuing multiple sites

[luckman212]

Platform

Docker container v0.52.1

Issue

Since updating to 0.52.1 I noticed that queuing multiple sites to be checked (with the "Recheck all in xxx" button) usually results in some of them spinning forever, stuck on "Checking now"

If I manually refresh the page, usually I find that the check actually did complete, but the UI just never updated to reflect that.

Once in a while, an error like this appears

Exception: Brotli compression subprocess failed for /datastore/xxx/last-fetched.br

I ran docker logs -f changedetection-io and saw this error

The red underlined UUID does correspond to one of the sites that was spinning endlessly, so it seems related.

The URL for that site is: https://pkgs.tailscale.com/stable/#macos and it is using the default Fetcher (system default) and no filters/triggers.

Let me know if you need any more info!

[dgtlmoon]

And if you reload the page?

I need your container log files attached here as . txt

[dgtlmoon]

Any proxies etc?

[dgtlmoon]

What platform exactly? Windows?

[dgtlmoon]

Can you paste the log lines around that brotli exception?

[dgtlmoon]

there should be more log info around "Brotli compression subprocess failed for...."... please add the sourrounding 30 lines or so

[luckman212]

Sorry, yes I will post the full log shortly

Platform is Linux/docker (Synology)

[luckman212]

I access my cdio instance via Tailscale (no proxy). For some reason I can't reproduce the Brotli error now. I made a screen recording showing the CDIO UI side by side with the docker log tail, and am attaching 2 log files here in case it helps.

cdio2.mp4

• log1.txt
• log2.txt

[dgtlmoon]

Thanks! That lines up roughly with the fixes at #3732 , It looks like the brotli compression of the snapshots on storing might have been causing a problem, and it turns out each 'worker' was not truely running independant.

[dgtlmoon]

@luckman212 try 0.52.2 in a few minutes when it's out, it should fix that, and speed up the checks for multiple workers, let me know if that bug comes back!

[luckman212]

Got it! I will check shortly and thank you!

[luckman212]

@dgtlmoon I updated to 0.52.2 but sorry to report that I am still seeing this issue. Here is a fresh log and a screenshot of how the page looks after clicking "Recheck all" and waiting about a minute...

• log3.txt

[dgtlmoon]

so 7 or so are stuck at 'checking now' even after page reload?

[luckman212]

no, after page reload, they have all successfully completed. It's just that the inital action never seems to refresh the page, something is timing out or disconnecting, which forces me to manually refresh the page.