Gloomberb runs suddenly extremely laggy (100% CPU use of one CPU core)

[ristomatti]

On #422 I commented about the app freezing. It's not actually freezing completely but is just extremely laggy. I tried restoring a backup from earlier today when the app was working fine. The backup included both ~/.gloomberb and the project source before pulling new changes (at commit dbf43b5). The backed up version did not work either though.

This makes me suspect either network connectivity issues or maybe changes done in Gloomberb Cloud (or Yahoo API's). Has something changed?

With some patience I was able to export a debug log. I'm not pasting the whole log here, but there's a lot of lines line this:

[18:54:35.563] INFO  [startup] tickers loaded  {"count":1934}
[18:54:35.564] INFO  [startup] refresh plan built  {"count":17,"financials":17,"quotes":0,"priority0":5,"priority1":0,"priority2":12,"symbols":["financials:BNOR.OL","financials:IXUA","financials:ES6Y","financials:ALV","financials:3529","financials:CIBR","financials:MRVL","financials:ASML","financials:000660","financials:SXR8","financials:INTC","financials:SIE","financials:AAPL","financials:SXR8.DE","financials:IXUA.DE","financials:SEC0","financials:QCOM"]}
[18:54:35.566] WARN  [quote-resolution] Rejected incoming stale quote contribution for the current session  {"incomingProviderId":"gloomberb-cloud","incomingPrice":23.235,"lastUpdated":1780414200000,"marketState":"CLOSED","exchange":"XETRA","symbol":"ES6Y"}
[18:54:35.567] WARN  [quote-resolution] Rejected incoming stale quote contribution for the current session  {"incomingProviderId":"yahoo","incomingPrice":5.941,"lastUpdated":1779317773197,"marketState":"CLOSED","exchange":"GER","symbol":"IXUA.DE"}
[18:54:35.567] WARN  [quote-resolution] Rejected incoming stale quote contribution for the current session  {"incomingProviderId":"yahoo","incomingPrice":385.6,"lastUpdated":1779486914038,"marketState":"CLOSED","exchange":"GER","symbol":"ALV.DE"}
[18:54:35.571] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"gloomberb-cloud","price":2333000,"marketState":"CLOSED","lastUpdated":1780034400000,"exchange":"KRX","symbol":"000660"}
[18:54:35.576] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"yahoo","price":203.545,"marketState":"REGULAR","lastUpdated":1778605721154,"exchange":"NMS","symbol":"QCOM"}
[18:54:35.578] WARN  [quote-resolution] Rejected incoming stale quote contribution for the current session  {"incomingProviderId":"gloomberb-cloud","incomingPrice":392.765,"lastUpdated":1781038680000,"marketState":" POST ","exchange":"NASDAQ","symbol":"AVGO"}
[18:54:35.587] WARN  [quote-resolution] Rejected incoming stale quote contribution for the current session  {"incomingProviderId":"yahoo","incomingPrice":119.24,"lastUpdated":1779397735017,"marketState":"CLOSED","exchange":"EBS","symbol":"NOVN.SW"}
[18:54:35.588] WARN  [quote-resolution] Rejected incoming stale quote contribution for the current session  {"incomingProviderId":"yahoo","incomingPrice":294.305,"lastUpdated":1780419894658,"marketState":"REGULAR","exchange":"NMS","symbol":"PANW"}
[18:54:35.589] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"gloomberb-cloud","price":27.5,"marketState":"CLOSED","lastUpdated":1780639956000,"exchange":"ASX","symbol":"SGM"}

Error level logs with a bit of surrounding context included:

[18:57:03.864] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"gloomberb-cloud","price":27.5,"marketState":"CLOSED","lastUpdated":1780639956000,"exchange":"ASX","symbol":"SGM"}
[18:57:03.866] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":12010.4,"elapsedMs":12060.4,"sampleIntervalMs":50,"sampleCount":195,"maxLagMs":13118.2}
[18:57:03.891] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"yahoo","price":203.545,"marketState":"REGULAR","lastUpdated":1778605721154,"exchange":"NMS","symbol":"QCOM"}
--
[18:57:15.941] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"gloomberb-cloud","price":27.5,"marketState":"CLOSED","lastUpdated":1780639956000,"exchange":"ASX","symbol":"SGM"}
[18:57:15.953] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":12036.9,"elapsedMs":12086.9,"sampleIntervalMs":50,"sampleCount":196,"maxLagMs":13118.2}
[18:57:27.865] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"yahoo","price":88,"marketState":" POST ","lastUpdated":1780419989499,"exchange":"GER","symbol":"IFX.DE"}
--
[18:57:38.862] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"gloomberb-cloud","price":27.5,"marketState":"CLOSED","lastUpdated":1780639956000,"exchange":"ASX","symbol":"SGM"}
[18:57:38.864] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":10812.7,"elapsedMs":10862.7,"sampleIntervalMs":50,"sampleCount":200,"maxLagMs":13118.2}
[18:57:38.871] ERROR [perf] persist.config.save  {"durationMs":10932.3}
[18:57:49.485] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"yahoo","price":88,"marketState":" POST ","lastUpdated":1780419989499,"exchange":"GER","symbol":"IFX.DE"}
--
[18:58:22.600] WARN  [cloud-api] websocket closed  {"quoteTargets":23,"channelTargets":1,"code":1006,"reason":"Connection ended","tokenSource":"session"}
[18:58:22.606] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":10713.2,"elapsedMs":10763.2,"sampleIntervalMs":50,"sampleCount":204,"maxLagMs":13118.2}
[18:58:33.335] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":10612.8,"elapsedMs":10662.8,"sampleIntervalMs":50,"sampleCount":206,"maxLagMs":13118.2}
[18:58:33.353] INFO  [cloud-api] open websocket  {"hasToken":true,"tokenSource":"session","quoteTargets":23,"channelTargets":1}
[18:58:44.155] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":10696.5,"elapsedMs":10746.5,"sampleIntervalMs":50,"sampleCount":208,"maxLagMs":13118.2}
[18:58:44.167] ERROR [perf] persist.config.save  {"durationMs":10831.3}
[18:58:54.992] INFO  [cloud-api] websocket open
[18:58:54.993] INFO  [cloud-api] send websocket message  {"type":"chat.subscribe","channelId":"everyone"}
[18:58:54.993] INFO  [cloud-api] send websocket message
[18:58:54.993] INFO  [cloud-api] websocket ready  {"emailVerified":true}
[18:58:54.993] ERROR [main-thread] event loop blocked  {"scope":"opentui","lagMs":10787.8,"elapsedMs":10837.8,"sampleIntervalMs":50,"sampleCount":209,"maxLagMs":13118.2}
[18:59:05.846] WARN  [quote-resolution] Rejected stale quote contribution for the current session  {"providerId":"yahoo","price":88,"marketState":" POST ","lastUpdated":1780419989499,"exchange":"GER","symbol":"IFX.DE"}

[ristomatti]

Note that I cannot be reached through the built-in chat. Every input event takes ~10 seconds to register so I'm basically locked out from the app. 🙂

[ristomatti]

I tried again today with both the backed up older version and one up-to-date on main branch. The UI appears responsive for around 3-5 seconds, then becomes unusable.

I'd be happy to share my config and database for testing if you can come up with a way to do it w/o posting it here publicly.

[vincelwt]

hey sorry for the delay! was off a few days.
there was a window management issue that should be fixed, wonder if that was related? Would you mind giving a shot with the latest main

[ristomatti]

Sorry I've been busy with work, will check ASAP I can.

[amawnkey]

Hitting a similar issue. New user so I apologize if I get some of the terminology wrong. It doesn't appear to be related to window management to me. It appears to be a memory leak triggered by accessing either the Graphs tab on any detail window for a stock or merely opening the Fear and Greed window. If I access either of those functions my instance goes from a reasonable sub-1GB of RAM to immediate lag and rapid constantly growing upwards of 20-30GB as I continue using the instance. Restarting fixes the issue so long as I don't start on either of those views.

Running on Linux kernel 6.17.9 in a Debian Trixie container but got the same results running uncontainerized as well. Container was built with a fresh pull of gloomberb today, so I think I'm on latest revision of 0.9.3 unless I need to do something to pull a different tag.

[vincelwt]

@amawnkey that's pretty bad, just to clarify that was from a git pull or from the install script (latest release) ?

[ristomatti]

@vincelwt Sorry it took so long. I tried again now but the problem persists. My update/run steps for the test (everything shows already updated as I just updated):

$ git remote -v                
origin	git@github.com:ristomatti/gloomberb.git (fetch)
origin	git@github.com:ristomatti/gloomberb.git (push)
upstream	https://github.com/vincelwt/gloomberb (fetch)
upstream	https://github.com/vincelwt/gloomberb (push)

$ git pull upstream main       
From https://github.com/vincelwt/gloomberb
 * branch            main       -> FETCH_HEAD
Already up-to-date.

$ bun install
bun install v1.3.13 (bf2e2cec)

Checked 152 installs across 158 packages (no changes) [3.00ms]

$ bun run dev

But what I noticed was that the app does not get stuck on one of my other layouts. I then repeated the cycle of cloning my main layout, then removing panes one by one. It turned out to be the "Fear and Greed" pane that causes the issue. App works as normal with it removed while I still have all these on my layout:

• Portfolio list
• 2x watchlist with 10 tickers
• X Feed with 8 tabs
• 3x Graph Price cahrts
• Sector Performance with Sectors selected
• Sector Performance with Industries selected

If I then open a new Fear & Greed pane (this time floating) - the app gets sluggish. I hit control+w to close the F&G pane and the result was this:

[vincelwt]

@ristomatti thank you, that's very helpful! pushing a fix asap

[amawnkey]

@vincelwt Just pulled it in via bun install -g gloomberb during the container build process

Also I get similar results to @ristomatti if I try to kill pane after loading FnG or Graphs.

Issue persists with 0.9.4

[ristomatti]

@amawnkey AFAIK no fix has been published yet so it is to be expected.

[ristomatti]

Like @amawnkey already mentioned, the issue triggers also when I open "Graphs" tab from stock ticker details.

[vincelwt]

@ristomatti @amawnkey sorry for the delay, latest version/main branch should be fixed!