High CPU load when checking new data

[spitz66]

Description

First of all, thanks for your great work however I have an issue as follows:

Is it expected that CPU Load goes to 100% during checking of new data?
I have default Observer Interval set to 30000 ms and it corresponds well with CPU Load, then back to normal:

What Platform / OS are you running?

Windows

What version are you running?

19043.1110

Anything in the logs or a references that might be useful?

No response

Additional information

No response

[timmo001]

Does the CPU go straight back down afterwards? (You also have windows defender scanning so it might be throwing things off)

I'm thinking we may want to split these intervals so the observer checks for changes for each endpoint independently instead of running all checks at once.

I've not seen a spike show up in the data frontend however:

This may work differently though

[spitz66]

Yes, it goes down again afterwards until next 30 seconds have passed..

I also noticed windows defender scanning but it actually peaks only at the same time as the other processes peak, then also goes down.

[timmo001]

#713 Should help. I've also have opened a few more issues which should decrease overall load by reducing the amount of tasks the observer does

[SilentException]

Hey @timmo001, any chance you can do beta release soon to test improvements so far? 2.5.0 makes complete system unusable here, it spawns those processes over and over and CPU is working 200% all the time...

[timmo001]

I plan on releasing soon. Currently #739 is blocking a release as you can only run the app for a few before it silently crashes.

If you are feeling daring though, you can download one of the artifacts from the CI pipeline which the releases source from.

https://github.com/timmo001/system-bridge/actions/workflows/ci.yml

[SilentException]

Hey, thanks. I got the CI artifact and installed that.
Here is the log, there is the first startup and somewhere in the middle I restart the server because I adjusted data acquisition interval from 30000ms to 120000ms. Some observations follow.
During the first run I noticed that collection of data on my system takes more than default 30s actually. The data collection starts even though last one didn't finish yet. So that would cause processes to pile up and take more and more of the resources!
After adjusting the interval and restarting you can clearly tell that data collection takes about 45s here.
During data collection the CPU would still spike, a lot but it is down (after adjustment and restart) until next data collection starts.

Here are some things you might consider improving.
Instead of piling up data collection threads and processes, wait for previous one to finish instead and then start next iteration, resetting the interval time for the third run and so on...
Set data collection processes and threads priority to lower priority on Windows so that the CPU spikes are not that high. https://docs.microsoft.com/en-us/windows/win32/procthread/scheduling-priorities, see base priority, something like BELOW_NORMAL_PRIORITY_CLASS / THREAD_PRIORITY_BELOW_NORMAL or base priority 6 would make sense I think. Or make it configurable. I for example am not on the newest and greatest of the CPU front, only using 5th gen Intetl NUC...

system-bridge.log

[romanskij]

Hello guys,
my question does not quite belong here, but this is the only post where i found something a update interval.
I have recently discovered System Bridge and am trying to transfer information to Home Assistant in order to calculate the power consumption from some values. Unfortunately, the current update rate of the values is 30 seconds. I would like to reduce it to e.g. 5 seconds.
Unfortunately, I can't find a setting for the Observe Interval.

Has this setting been removed or am I overseeing something?

I use the current version 3.4.4.

Thank you for your support.