[Performance] Raw Graph Performance Improvement

[ahmedtarek-]

A. Plot Re-rendering

Using react-scan, I could see that the component was not re-rendering after using the subscription logic as per the change in this PR.

B. Time Profiling

After time profiling the before and after, the results are actually pretty disappointing. I made a console log for when 1000 raw_data messages have been received through the socket (in SocketStore.js) and logged the time Plotly spent in .restyle() (this update) vs .react() (existing code). The results are as following

• After using Subscription

1107 Restyles

255123 ms
255 seconds (4.25 minutes)

• Before using Subscription

460 Reacts 

41298 ms
41 seconds

C. Next Steps

Currently I am trying to understand why this could be the case. I am splitting this into two different questions

1. Why the number of restyles/reacts are different?
2. Why is one restyle slower than one react?
3. Is there a particular bug in the new code that could be causing this?

Still WIP

[ahmedtarek-]

Update

Debugging performance for 500 messages
- .Restyle() yTraces only -> 611 update, 121.921 s
- .React() with subscription only -> 500 updates, 114.072 s
- .React() without subscription (before this PR) -> 254 updates, 22.481 s

This basically means that the subscription direction isn't performing well. The only idea I have in mind now is to batch the data/messages and update the graph every x amount of messages. I am currently testing this out, and it might make sense to do that in general.

Apart from this, I ran out of ideas how to improve the subscription logic.

[timonmerk]

Thanks a lot @ahmedtarek. @toni-neurosc Can you briefly check the changes? It seems the subscription pattern slowed down the redrawing quite a lot.
I think it would be great to integrate ECharts now. This will be quite quick changes @ahmedtarek. I would just rewrite it with AI and then see for performance improvements. Similar to plotly, disabling hovering and disabling zoom function will probably speed up things also.

[ahmedtarek-]

Okk, will do the ECharts direction then ✌🏽