Empty dashboard with undefined value from Grafana heatMap.tsx

[alf0ns3]

Hi, first of all thank you for this wonderful tool 😀

I wanted to deploy PiRogue VPN on a machine without public IP address, and redirect everything through a VPN. Because it is not built with this niche behaviour in mind, I had to glue the project with custom network configurations (that I share below if my problem could be related to a wrong configuration or if anyone is interested to reproduce such a system).

From the clients, everything works fine in the sense they open the generated WG configuration and all their traffic goes through PiRogue and the VPN without any leaks reported from the VPN servers.

Problem I'm facing

However, when I connect to the PiRogue dashboard, even if I generated a lot of traffic, I see:

Then after few seconds:

And on the console:

Uncaught (in promise) TypeError: can't access property "length", c is undefined
    update heatMap.tsx:88
    dataChanged GeomapPanel.tsx:134
    initLayers GeomapPanel.tsx:270
[heatMap.tsx:88:29](webpack:///public/app/plugins/panel/geomap/layers/data/heatMap.tsx)

TypeError: can't access property "length", c is undefined
    update heatMap.tsx:88
    dataChanged GeomapPanel.tsx:134
    shouldComponentUpdate GeomapPanel.tsx:90
    React 8
    unstable_runWithPriority scheduler.production.min.js:18
    React 6
    onDataUpdate PanelChrome.tsx:228
    next PanelChrome.tsx:134
    RxJS 19
    next PanelQueryRunner.ts:268
    RxJS 15

It is due to heatMap.tsx:70:

// data.series == [], so frame is 'undefined' and the error occurs
const frame = data.series[0];
for (let i = 0; i < frame.length; i++) {
  // something
}

What I tried

Except of a 101 response for ws, I only have 200 status code among all URL fetched.

When I visit from the client https://whatismyipaddress .com, the Suricata detection is working so PiRogue truely analyze the traffic:

# journalctl -xeu pirogue-eve-collector
oct. 20 16:27:33 pirogue pirogue-eve-collector[9694]: [{'measurement': 'alert', 'tags': {'event_type': 'alert', 'proto': 'UDP', 'src_ip': '10.0.0.100', 'dest_ip': '10.0.0.10', 'community_id': '<redacted>', 'tx_id': 0, 'alert_action': 'allowed', 'alert_gid': 1, 'alert_signature_id': 2054170, 'alert_rev': 1, 'alert_signature': 'ET INFO External IP Lookup Domain in DNS Lookup (whatismyipaddress .com)', 'alert_category': 'Device Retrieving External IP Address Detected', 'alert_severity': 2, 'app_proto': 'dns'}}]

I changed my webserver configurations to drop all the middlewares, I also removed the forwarding through the remote VPN without any change.

All those configurations are automatised with Ansible and I built everything from scratch again before opening this issue, so I'm sure there is not an old odd change I made somewhere.

Once (and with all the wireguard configurations working), I called pirogue-admin-client external-network enable-public-access --domain <URL> --email <email> just in case, applied my CA files, touched /etc/gai.conf (because wrong localhost resolved as [::1] instead of 127.0.0.1, I don't manage IPv6), rebooted and 🥁 the dashboard worked! but from scratch again by doing the same steps I never reproduced this working state.

Root causes?

Maybe there is an issue because I don't use IPv6; somewhere in the API Grafana cannot read the correct data series from PiRogue inspections; my network configurations broke something in PiRogue; or anything else.

I don't know enough the PiRogueToolSuite and never played with Grafana earlier so I'm totally stucked now 🙏

Context

I installed pirogue-base latest version on a Debian 12 server, self-hosted on a PVE instance.

Wireguard connection between clients and the PiRogue server

I have a public facing wireguard proxy server which makes the clients able to redirect all their traffic through PiRogue.

Proxy configuration:

# /etc/wireguard/wg0.conf
[Interface]
Address = 10.0.0.1/24
PrivateKey = <redacted>
ListenPort = 51820
# redirects all client traffic through PiRogue
Table = 200
PreUp = ip rule add iif wg0 table 200 priority 200
PreDown = ip rule del iif wg0 table 200 priority 200
[Peer]
# PiRogue, IP 10.0.0.10
PublicKey = <redacted>
AllowedIPs = 0.0.0.0/0
[Peer]
# Client 1
PublicKey = <redacted>
AllowedIPs = 10.0.0.100/32

The client gets this configuration, I wrote a script to generate them instead of relying on the default generation:

# /etc/wireguard/wg0.conf
[Interface]
# client_100
Address = 10.0.0.100/24
PrivateKey = <redacted>
DNS = 10.0.0.10 # PiRogue

[Peer]
# Name = public proxy
PublicKey = <redacted>
# Redirects everything through PiRogue
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 25
EndPoint = <proxy public IP>:51820

Finally, PiRogue has this configuration:

# /etc/wireguard/wg0.conf
[Interface]
# pirogue
Address = 10.0.0.10/24
PrivateKey = <redacted>
# redirection through VPN server
PostUp = nft 'add table inet wg_pirogue'
PostUp = nft 'add chain inet wg_pirogue mangle { type filter hook prerouting  priority mangle; policy accept; }'
PostUp = nft 'add chain inet wg_pirogue nat    { type nat    hook postrouting priority srcnat; policy accept; }'
PostUp = nft 'add rule  inet wg_pirogue mangle iifname    %i counter meta mark set 0x30'
PostUp = nft 'add rule  inet wg_pirogue nat    oifname != %i counter      mark     0x30 masquerade'
PostUp = ip route add default via 192.168.1.10 dev eth0 table mullvad
PostUp = ip rule  add iif %i table vpn priority 100
PreDown = nft 'delete table inet wg_pirogue'
PreDown = ip route del default via 192.168.1.10 dev eth0 table mullvad
PreDown = ip rule  del iif %i table vpn priority 100
[Peer]
# proxy
PublicKey = <redacted>
AllowedIPs = 10.0.0.0/24
PersistentKeepalive = 25
Endpoint = <proxy public IP>:51820

VPN redirection after PiRogue analyses

On PiRogue, I forward all wg0-incoming packets to the VPN using a local named route table, configured with WG PostUp actions:

# /etc/iproute2/rt_tables
200 vpn

DNS leaks

To prevent DNS leaks, I configured stubby to resolve queries through the DoT VPN's servers on PiRogue, listening on 10.0.0.10. So the clients' queries are resolved by the VPN's DNS resolvers.

I also stopped dnsmasq service.

PiRogue configuration overrides

The EXTERNAL_* values are automatically set to eth0.

# /var/lib/pirogue/admin/user.config.yaml
ENABLE_PUBLIC_ACCESS: 'False'

ISOLATED_ADDRESS: "10.0.0.10"
ISOLATED_INTERFACE: "wg0"
ISOLATED_NETWORK: "10.0.0.0/24"

PUBLIC_CONTACT_EMAIL: <redacted>
PUBLIC_DOMAIN_NAME: <URL>
PUBLIC_EXTERNAL_ADDRESS: <public proxy>

SYSTEM_HOSTNAME: pirogue.infra.lan
SYSTEM_OPERATING_MODE: wireguard
DASHBOARD_PASSWORD: <redacted>

Webserver

I set ENABLE_PUBLIC_ACCESS: 'False' because I use a webserver located on the same local network than PiRogue to manage SSL certificates, with Traefik:

http:

  middlewares:
    # cannot use it, 'wss upgrade fails' error instead
    add-dashboard:
      addPrefix:
        prefix: "/dashboard"

    compress:
      compress: {}

    redirect-to-https:
      redirectScheme:
        scheme: https
        permanent: true

    security-headers:
      headers:
        isDevelopment: false

        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-XSS-Protection -> DEPRACATED
        browserXssFilter: false
        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Security-Policy
        contentSecurityPolicy: "frame-ancestors 'self'"
        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Content-Type-Options -> 'nosniff'
        contentTypeNosniff: true
        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Frame-Options
        customFrameOptionsValue: "SAMEORIGIN"
        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Referrer-Policy
        referrerPolicy: "strict-origin-when-cross-origin"
        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Strict-Transport-Security
        forceSTSHeader: true
        stsSeconds: 31536000 # 1 year
        stsIncludeSubdomains: true
        stsPreload: true
        customRequestHeaders:
          Sec-Fetch-Site: "cross-site"
          X-Forwarded-Proto: "https"

  routers:
    pirogue:
      rule: "Host(`<URL>`)"
      service: pirogue@file
      entrypoints:
        - websecure

      middlewares: redirect-to-https , security-headers , compress

  serversTransports:
    local-root-ca:
      insecureSkipVerify: false
      rootCAs:
        - "/usr/local/share/ca-certificates/own_ca_server.crt"

  services:
    pirogue:
      loadBalancer:
        serversTransport: local-root-ca
        servers:
          - url: "https://pirogue.infra.lan"