Migration from venus-docker-grafana to venus-grafana

[jjx-]

It would be great to give users a migration path from venus-docker-grafana to venus-grafana which includes keeping the data in the influx db and the grafana dasboards.

I am using venus-docker-grafana for about a year and would like to migrate to venus-grafana to be able to benefit from the latest updates. I do want to keep my data of the last year and the grafana dasboards I built.

Is there a way to upgrade/migrate and if so which are the steps I need?
If there is no easy way of migration then a guide on how to export the data on the old one and import it on the new one would be appreciated.

[mman]

@jjx- Great idea, I should document that in the future.

For the moment:

The underlying database, structure, and config file format have not changed. Also with Influx DB we are stuck at version 1.8 which is the last one running officially on 32bit arm systems, so no change there as well and all data that you are storing will still be readable and stored in the same way after any update.

The only changes that happened are actually internal, fixing bugs, making javascript code compile after many component updates, etc.

If you take your docker-compose.yaml file and simply replace the

• venus-docker-grafana-server:<version> with venus-influx-loader:1.0
• venus-docker-grafana:<version> with venus-grafana:1.0

all should be well. But of course YMMV, please post your docker-compose.yaml here and I can test the procedure for you on my test data and possibly write up what has happened :)

[jjx-]

Thank you for your quick response!

I tried it on a test VM and got some errors because the version numbers need to be 1.0.0 instead of 1.0. Other than that it worked flawlessly.

I do attach my docker-compose file for reference in case anybody needs it.

Thanks again for your help!

[mman]

@jjx- Good catch with the version numbers, I need to add some SemVer magic to automatically tag the build properly...

[BipedalPrimate]

Great idea, I should document that in the future.

@mman Wondering if there has been any progress the migration path?

[mman]

Great idea, I should document that in the future.

@mman Wondering if there has been any progress the migration path?

Nope, haven't gotten around to it yet, but here is the essence of it. The docker images were renamed. UPNP image was merged with the loader image. And hardcoded pointers were turned into environment variables so that they can be easily overridden. Let me please know which of these need better explanation and I will merge it into the main README.

--- docker-compose-old.yaml
+++ docker-compose-new.yaml

1. Replace docker image name venus-docker-server:latest with versioned venus-influx-loader:1.3 and add appropriate command

@@ -1,31 +1,40 @@
 version: '3.4'
 services:
-  server:
-    image: "victronenergy/venus-docker-server:latest"
+  loader:
+    image: "victronenergy/venus-influx-loader:1.3"
+    command: [ "bin/venus-influx-loader", "--enable-discovery-api" ]
     ports:
      - "8088:8088"
     volumes:
      - "config-storage:/config"

2. Replace docker image name venus-docker-upnp:latest with venus-influx-loader:1.3 and add appropriate command pointing towards loader.

-  upnp:
-    image: "victronenergy/venus-docker-upnp:latest"
+  browser:
+    image: "victronenergy/venus-influx-loader:1.3"
+    command: [ "bin/venus-upnp-browser", "--discovery-api", "http://localhost:8088/discovery-api/" ]
+    # `network_mode: host` is required for the `venus-upnp-browser` to access your host network
+    # and audo discover Venus devices.
     network_mode: host

3. Use latest version of InfluxDB 1.x

   influxdb:
-    image: "influxdb:1.7"
+    image: "influxdb:1.8"

4. Replace use of venus-docker-grafana:latest with venus-grafana:1.3 and add VIL_* environment variables to configure Grafana to point correctly towards loader.

-  graphing:
-    image: "victronenergy/venus-docker-grafana:latest"
+  grafana:
+    image: "victronenergy/venus-grafana:1.3"
+    environment:
+     - VIL_INFLUXDB_URL=http://influxdb:8086
+     - VIL_GRAFANA_API_URL=http://loader:8088/grafana-api
+     # specify URL that should be listed on Welcome dashboard
+     # to acess Venus Influx Loader Admin UI
+     - VIL_PUBLIC_URL=http://localhost:8088/ 

[sebix]

Thank you for the migration guide. With it, I was able to start the setup, but I still have some troubles.

In :8088/admin/#/troubleshooting I see

Failed saving config to: /config/config.json, error: Error: EACCES: permission denied, open '/config/config.json'.

which is true, as that file is owned by root:root. Entering this container with a shell ends up with the unprivileged user node, unable to change the permissions.

For the measurements and stats, they show (if at all) apparent random data, definitely not the measurements.
For example, this should be the AC frequency and voltage (with two different ranges, only one is shown now):

The shown data is garbage. No idea where that comes from.

[sebix]

I took a JSON backup and changed all

        "type": "datasource",
        "uid": "grafana"

with

        "type": "influxdb",
        "uid": "datasource-influxdb"

and imported it. Then it worked.