Skip to content

Add 2500 nodes with non-standard (Gluon) version numbers#119

Open
T-X wants to merge 9 commits intofreifunk-gluon:mainfrom
T-X:pr-more-versions
Open

Add 2500 nodes with non-standard (Gluon) version numbers#119
T-X wants to merge 9 commits intofreifunk-gluon:mainfrom
T-X:pr-more-versions

Conversation

@T-X
Copy link
Contributor

@T-X T-X commented Jan 13, 2026 •
edited
Loading

This adds around 1250 Gluon nodes which have some slightly "unconventional" version strings. Which, even though their Gluon base cannot be determined, are still clearly Gluon nodes.

For a bit more control on graph creation with these nodes included the extra label "vtype" is added. So if one would like to keep a graph as is, with these new nodes excluded, one can filter for a vtype="gluon-base" label. The new nodes are using a vtype="gluon-unknown", vtype="gluon-commitid" and vtype="gluon-custom" label. See the respective Git commits for details.

Furthermore, this adds around 1250 nodes which do not run Gluon, but which still announce themselves to the Gluon census exporter. These are mainly Debian/Ubuntu/NixOS systems. But also OpenWrt routers from Freifunk Dresden. To ignore these nodes in graphs, one can filter them out via the new vtype="foreign" label. And some announce no version at all, these can be filtered out via the new vtype="undefined".

@T-X T-X force-pushed the pr-more-versions branch 9 times, most recently from 2264ee6 to 27ee671 Compare January 13, 2026 06:25
@T-X T-X changed the title Add 2450 nodes with non-standard (Gluon) version numbers Add 2500 nodes with non-standard (Gluon) version numbers Jan 13, 2026
@T-X T-X mentioned this pull request Jan 13, 2026
Open
@T-X
Copy link
Contributor Author

T-X commented Jan 13, 2026

This PR was mainly motivated by slightly misleading values in this graph: https://gluon-census.ffrn.de/d/CA5PRFmMz/gluon-census3a-dashboard?orgId=1&from=now-10d&to=now&timezone=browser&var-community=$__all&var-version_major=$__all&var-version=v2025.1&var-version=v2023.2.5&var-version=v2023.2.4&var-version=v2023.2.3&var-version=v2023.2.2&var-version=v2023.2.1&var-version=v2023.2&var-version=v2023.1.2&var-version=v2023.1.1&var-version=v2023.1&var-models=$__all&refresh=15m&viewPanel=panel-28.
For instance Südholstein is not actually at 100% at a Gluon version >=2023.1. Their old devices show a gluon-<commitid> version instead though, which so far the Gluon Census would simply ignore. For an update status percentage it would be useful to take these devices into consideration, too.

@AiyionPrime
Copy link
Member

AiyionPrime commented Jan 14, 2026

Thanks for sharing the motivation :)

Is there a better way to filter non gluon devices out, even if they provide respondd data?

@AiyionPrime
Copy link
Member

AiyionPrime commented Jan 14, 2026

(it feels like they are out of scope, but I might be wrong)

@T-X
Copy link
Contributor Author

T-X commented Jan 14, 2026 •
edited
Loading

@AiyionPrime to always leave out non-gluon devices one could just omit the last two patches in this PR. But I thought these aren't that many and I still found it interesting to know.

Of course, with the last two patches (nearly) all graphs probably would need an update/addition to filter out outputs for vtype=foreign and vtype=undefined, if a graph is supposed to only show counts for devices running Gluon. My preference would be to do this. And then I would be interested to create one new graph showing the number of Gluon nodes vs. "foreign" vs. "undefined".

@T-X
Copy link
Contributor Author

T-X commented Jan 14, 2026

@AiyionPrime hm, or another option could be to use a separate table for the "foreign" (e.g. "version=Debian/NixOs/Ubuntu/..." and "undefined" (no version info) ones. So instead of putting these nodes into metric_gluon_version_total, one could introduce a new metric_gluon_foreign_total, for instance? Advantage: No changes needed to current graphs only showing Gluon nodes?

@T-X T-X force-pushed the pr-more-versions branch 2 times, most recently from e061026 to a9485e1 Compare January 21, 2026 05:03
@T-X
Copy link
Contributor Author

T-X commented Jan 21, 2026

Rebased onto:

Also migrated the foreign/unknown node counting to a separate prometheus collection.

T-X added 9 commits January 23, 2026 05:06
@T-X
exporter: remove unnecessary "global" declarations
2a961d5 
The global "seen" and "duplicates" variables are used in a sub function.
Therefore it is not necessary to declare them global in the parser
functions.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: unify ParseResult() initialization
4437b6d 
As we already use classes just use an appropriate initializer to avoid
initializing per parser.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: add per collection debug output sums
86a7b78 
Adding a "Collections summaries" debug output line as follows:

```
[...]
2026-01-20 02:56:04 [info     ] Collections summaries          domain_sum=31569 model_sum=37589 version_sum=34872
2026-01-20 02:56:04 [info     ] Summary                        duplicate=82 unique=39152
```

This allows to more easily and early on check that a change
likely/hopefully has no regression. Without needing to manually sum
things up from the gluon-census.prom file, without needing to wait for
rendered Grafana graphs.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: refactor versions to allow more pattern matches, add vtype
b358cfc 
Currently we only consider nodes which have a base version string which
has which a typical Gluon release version string prefix.

However there are several nodes / communities which have a different
format, for instance because they didn't import the Gluon tags or
because they copied the repository without Git metadata.

This restructuring allows to add more pattern matches later. This also
adds a new label "vtype", currently defaulting to a value of
"gluon-base", intended for the current behaviour of matching nodes with
an expected Gluon version tag prefix.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: add nodes with a "gluon-unknown" version
69fdaec 
This adds the following ~1132 nodes:

gluon_base_total{base="gluon-unknown",community="Erfurt",version="gluon-unknown",vtype="gluon-unknown"} 25.0
gluon_base_total{base="gluon-unknown",community="Harz",version="gluon-unknown",vtype="gluon-unknown"} 803.0
gluon_base_total{base="gluon-unknown",community="Ingolstadt",version="gluon-unknown",vtype="gluon-unknown"} 1.0
gluon_base_total{base="gluon-unknown",community="Rhein-Sieg",version="gluon-unknown",vtype="gluon-unknown"} 303.0

"(gluon-)unknown" is output and baked into a firmware by Gluon's
./scripts/getversion.sh if no .git repository is present.
Track these nodes with a 'vtype="gluon-unknown"' label.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: add nodes with a "gluon-$commitid" version
d130c27 
This adds the following ~114 nodes:

gluon_base_total{base="gluon-ff0c72f",community="Erfurt",version="gluon-ff0c72f",vtype="gluon-commitid"} 55.0
gluon_base_total{base="gluon-d2a9cc3",community="Erfurt",version="gluon-d2a9cc3",vtype="gluon-commitid"} 3.0
gluon_base_total{base="gluon-031a835",community="Erfurt",version="gluon-031a835",vtype="gluon-commitid"} 7.0
gluon_base_total{base="gluon-b9a621f",community="Erfurt",version="gluon-b9a621f",vtype="gluon-commitid"} 3.0
gluon_base_total{base="gluon-2ea83c4",community="Erfurt",version="gluon-2ea83c4",vtype="gluon-commitid"} 6.0
gluon_base_total{base="gluon-d0f43ae",community="Erfurt",version="gluon-d0f43ae",vtype="gluon-commitid"} 10.0
gluon_base_total{base="gluon-f1f7f61",community="Südholstein",version="gluon-f1f7f61",vtype="gluon-commitid"} 30.0

"(gluon-)$commitid" is output and baked into a firmware by Gluon's
./scripts/getversion.sh if a .git repository is present, but without any
annotated Git tags. Track these nodes with a 'vtype="gluon-commitid"' label.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: add nodes with a "gluon-.*" version
8913f46 
This adds the following ~29 nodes:

gluon_base_total{base="gluon-2019-09-27",community="Halle",version="gluon-2019-09-27",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-2022.1+mwu1",community="Mainz/Wiesbaden",version="gluon-2022.1+mwu1",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-15-05-2018",community="Münsterland",version="gluon-15-05-2018",vtype="gluon-custom"} 2.0
gluon_base_total{base="gluon-experimental-2025-05-27",community="Stuttgart",version="gluon-experimental-2025-05-27",vtype="gluon-custom"} 13.0
gluon_base_total{base="gluon-experimental-2025-05-18",community="Stuttgart",version="gluon-experimental-2025-05-18",vtype="gluon-custom"} 5.0
gluon_base_total{base="gluon-experimental-2025-01-06",community="Stuttgart",version="gluon-experimental-2025-01-06",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-experimental-2025-10-02",community="Stuttgart",version="gluon-experimental-2025-10-02",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-experimental-2025-09-19",community="Stuttgart",version="gluon-experimental-2025-09-19",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-experimental-2025-10-15",community="Stuttgart",version="gluon-experimental-2025-10-15",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-experimental-2025-05-08",community="Stuttgart",version="gluon-experimental-2025-05-08",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-experimental-2025-10-23-base-28-g423258a+",community="Stuttgart",version="gluon-experimental-2025-10-23-base-28-g423258a+",vtype="gluon-custom"} 1.0
gluon_base_total{base="gluon-experimental-2024-11-24-4-g4b02790",community="Stuttgart",version="gluon-experimental-2024-11-24-4-g4b02790",vtype="gluon-custom"} 1.0

These were likely created via a custom .scmversion in the Gluon
repository / via a GLUON_VERSION environment variable.
Track these nodes with a 'vtype="gluon-custom"' label.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: add nodes with a non-gluon version
c6f958b 
This adds the following ~1211 nodes:

gluon_base_total{base="Debian",community="Bremen",version="Debian",vtype="foreign"} 3.0
gluon_base_total{base="NixOS",community="Darmstadt",version="NixOS",vtype="foreign"} 149.0
gluon_base_total{base="Debian",community="Chemnitz",version="Debian",vtype="foreign"} 2.0
gluon_base_total{base="Debian",community="Altdorf",version="Debian",vtype="foreign"} 11.0
gluon_base_total{base="Ubuntu",community="Aachen",version="Ubuntu",vtype="foreign"} 20.0
gluon_base_total{base="Debian",community="Aachen",version="Debian",vtype="foreign"} 20.0
gluon_base_total{base="Debian",community="Einbeck",version="Debian",vtype="foreign"} 2.0
gluon_base_total{base="Ubuntu",community="Emskirchen",version="Ubuntu",vtype="foreign"} 3.0
gluon_base_total{base="Debian",community="Essen",version="Debian",vtype="foreign"} 1.0
gluon_base_total{base="Ubuntu",community="Essen",version="Ubuntu",vtype="foreign"} 1.0
gluon_base_total{base="Debian",community="Dreiländereck",version="Debian",vtype="foreign"} 95.0
gluon_base_total{base="OpenWrt 18.06.2 r7676-cddd7b4c77",community="Dreiländereck",version="OpenWrt 18.06.2 r7676-cddd7b4c77" vtype="foreign"} 1.0
gluon_base_total{base="Ubuntu 20.04",community="Dresden",version="Ubuntu 20.04",vtype="foreign"} 3.0
gluon_base_total{base="Debian 12",community="Dresden",version="Debian 12",vtype="foreign"} 18.0
gluon_base_total{base="Debian 11",community="Dresden",version="Debian 11",vtype="foreign"} 7.0
gluon_base_total{base="OpenWrt 22.03.7",community="Dresden",version="OpenWrt 22.03.7",vtype="foreign"} 437.0
gluon_base_total{base="OpenWrt 18.06-SNAPSHOT r0+8073-5625f5bc36",community="Dresden",version="OpenWrt 18.06-SNAPSHOT r0+8073-5625f5bc36",vtype="foreign"} 223.0
gluon_base_total{base="OpenWrt 21.02.3",community="Dresden",version="OpenWrt 21.02.3",vtype="foreign"} 151.0
gluon_base_total{base="OpenWrt 21.02.1",community="Dresden",version="OpenWrt 21.02.1",vtype="foreign"} 1.0
gluon_base_total{base="OpenWrt 22.03.6",community="Dresden",version="OpenWrt 22.03.6",vtype="foreign"} 14.0
gluon_base_total{base="OpenWrt 22.03.5",community="Dresden",version="OpenWrt 22.03.5",vtype="foreign"} 31.0
gluon_base_total{base="OpenWrt 21.02.2",community="Dresden",version="OpenWrt 21.02.2",vtype="foreign"} 3.0
gluon_base_total{base="Freifunk Dresden 24.10.2",community="Dresden",version="Freifunk Dresden 24.10.2",vtype="foreign"} 12.0
gluon_base_total{base="OpenWrt 19.07.6",community="Dresden",version="OpenWrt 19.07.6",vtype="foreign"} 1.0
gluon_base_total{base="Ubuntu 22.04",community="Dresden",version="Ubuntu 22.04",vtype="foreign"} 1.0
gluon_base_total{base="Freifunk Dresden 24.10.0-rc5",community="Dresden",version="Freifunk Dresden 24.10.0-rc5",vtype="foreign"} 1.0

These are nodes which provide respondd data to meshviewer etc., but
which are not Gluon nodes. Track these nodes with a "vtype=foreign"
label. Track these nodes with a 'vtype="foreign"' label.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X
exporter: add nodes with an undefined version
1c7670c 
This adds the following ~30 nodes:

gluon_base_total{base="undefined",community="Frankfurt",version="undefined",vtype="undefined"} 1.0
gluon_base_total{base="undefined",community="Mainz/Wiesbaden",version="undefined",vtype="undefined"} 1.0
gluon_base_total{base="undefined",community="München",version="undefined",vtype="undefined"} 15.0
gluon_base_total{base="undefined",community="Nordhessen",version="undefined",vtype="undefined"} 14.0

These nodes provide a node_id but don't provide a .firmware.base
or .nodeinfo.software.firmware.base version.
Track these nodes with a 'vtype="undefined"' label.

Signed-off-by: Linus Lüssing <linus.luessing@c0d3.blue>
@T-X T-X force-pushed the pr-more-versions branch from a9485e1 to 1c7670c Compare January 23, 2026 04:21
@T-X
Copy link
Contributor Author

T-X commented Jan 23, 2026 •
edited
Loading

Changelog v3:

  • rebased to current main branch (no changes)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@T-X @AiyionPrime