Reenabling current load button

[B0r1sD]

Trying to get our Current load page back up, as Graphana has been tracking the stats behind usegalaxy.be again (https://usegalaxy.be/grafana/d/000000023/node-detail-infrastructure1?orgId=1&from=now-3h&to=now&timezone=browser&var-host=usegalaxy.be).

Only the previous dashboards are still broken, see:

usegalaxy-be.github.io/current-load.md

Line 8 in 69ed5a1

<iframe src="https://usegalaxy.be/grafana/d-solo/000000005/galaxy?orgId=1&from=1715669252892&to=1715680052892&panelId=54" width="1100" height="400" frameborder="0" class="rounded-top"></iframe>

Checking the latest Grafana log at /var/log/grafana/, which dates from April 12th, includes errors such as:

logger=login_attempt t=2025-04-12T14:15:30.304174538+02:00 level=error msg="Failed to lock and execute cleanup of old login attempts" error="unable to open database file: no such file or directory"
logger=cleanup t=2025-04-12T14:15:30.316689486+02:00 level=info msg="Completed cleanup jobs" duration=14.42819ms
logger=plugins.update.checker t=2025-04-12T14:15:30.372674032+02:00 level=info msg="Update check succeeded" duration=22.226762ms
logger=login_attempt t=2025-04-12T14:25:30.303666068+02:00 level=error msg="Failed to lock and execute cleanup of old login attempts" error="unable to open database file: no such file or directory"
logger=cleanup t=2025-04-12T14:25:30.318327376+02:00 level=info msg="Completed cllogger=login_attempt t=2025-04-12T16:35:30.31685762+02:00 level=error msg="Failed to lock and execute cleanup of old login attempts" error="unable to open database file: no such file or directory"

which is odd as the database file seems to be in the correct location:/var/lib/grafana/grafana.db as indicated in /etc/grafana/grafana.ini:

...
# Directories
[paths]
data = /var/lib/grafana
logs = /var/log/grafana
...

Status of the grafana-server also seems fine. command sudo systemctl status grafana-server:

grafana-server.service - Grafana instance
     Loaded: loaded (/usr/lib/systemd/system/grafana-server.service; enabled; preset: disabled)
     Active: active (running) since Fri 2025-03-14 15:35:30 CET; 3 months 18 days ago
...

So there is still something disabled or not working, even though the main Grafan dashboard is working: https://usegalaxy.be/grafana/d/000000023/node-detail-infrastructure1?orgId=1&from=now-3h&to=now&timezone=browser&var-host=usegalaxy.be.

Maybe it has something to do with the inactive influxDB and influxd services?

sudo systemctl status influxdb.service
○ influxdb.service - InfluxDB is an open-source, distributed, time series database
     Loaded: loaded (/usr/lib/systemd/system/influxdb.service; enabled; preset: disabled)
     Active: **inactive (dead)** since Thu 2025-03-13 15:16:46 CET; 3 months 19 days ago
   Duration: 2month 2w 6d 23h 36min 3.163s
       Docs: https://docs.influxdata.com/influxdb/
   Main PID: 2927134 (code=exited, status=0/SUCCESS)
        CPU: 13h 34min 26.456s

Notice: journal has been rotated since unit was started, output may be incomplete.
galaxy_master@vgcn-galaxy-head-0:~$ sudo systemctl status influxd.service
○ influxdb.service - InfluxDB is an open-source, distributed, time series database
     Loaded: loaded (/usr/lib/systemd/system/influxdb.service; enabled; preset: disabled)
     Active: **inactive (dead)** since Thu 2025-03-13 15:16:46 CET; 3 months 19 days ago
   Duration: 2month 2w 6d 23h 36min 3.163s
       Docs: https://docs.influxdata.com/influxdb/
   Main PID: 2927134 (code=exited, status=0/SUCCESS)
        CPU: 13h 34min 26.456s

Config changes to get button back already In branch reenable_load: a13b20e.

[B0r1sD]

Flagging that the 'current load' Grafana dashboard (https://usegalaxy.be/grafana/d/000000023/node-detail-infrastructure1?orgId=1&from=now-90d&to=now&timezone=browser&var-host=usegalaxy.be) still does not seem to consistently keep track of the data since previous fix:

Data points now only start at 03/06/2026. Previous data points (that were showing up again when this issue was created) do not show up:

Do you know what change caused this @pauldg? Or do you think when we decide to put the effort in to reenable this, it will be a matter of changing the TSDB (" table the grafana graphs are sourcing their data from?