Distinguish between multiple Gateways

[tobru]

Currently, there is no way to distinguish between multiple gateways, rather than running multiple instances of RuuviBridge (what I currently do and works). I'd like to suggest teaching RuuviBridge to distinguish between multiple Gateways.

An example could be:

influxdb_publisher:
  [...]
  additional_tags:
    location: $gateway_name

gateway_names:
  XXXXXXXXXXX: LocationA
  YYYYYYYYYYY: LocationB

(That's just a partial config to show the idea)

This approach would make it mandatory to configure the Gateways to publish to a predefined format:

mytopic/$GW_MAC/$TAG_MAC

We could also find a way to make the topic format configurable, if desired.

What do you think about this idea? I'm open to refine it if needed.

[Scrin]

This is actually part of something I've been thinking about having as a feature where certain configuration options would support go templates which could use certain values, such as user-configurable parsing a part of the topic to use in the additional_tags like you described, and some "built-in" ones such as the RuuviBridge version. I was thinking about using regexp with capturing groups for the "parsing configuration".

Basically the idea I had in mind was to parse and extract arbitrary portions of the topic with a config something like this (this example having the idea that you'd have multiple "locations" and each "location" would have one or more gateways):

mqtt_listener:
  [...]
  topic_prefix: ruuvi/+/+ # matches ruuvi/<location>/<gateway_mac>/<tag_mac>
  topic_vars_parse: "ruuvi/(?P<Location>.*)/(?P<GwMac>.*)/.*"

And then the additional_tags can be configured as go templates like this:

influxdb_publisher:
  [...]
  additional_tags:
    location: "{{.MqttTopic.Location}}"
    gw_mac: "{{.MqttTopic.GwMac}}"
    ruuvibridge_version: "{{.RuuviBridge.Version}}"

where the named capturing groups from the topic_vars_parse would be in the MqttTopic fed to the templates. Perhaps even parsing the tag_mac could be (optionally) configurable at some point in the future allowing even more exotic topics.

In the future the parsed variables could also be used in other future features, such as supporting similar templating in the mqtt_publisher topic and conditional filtering and/or rate limiting based on an extracted variable.

I'd like to "get this right" on the first time in order to avoid breaking changes in the future, so this should be carefully planned and thought out before implementing. Thoughts?

[tobru]

I like your suggestion!

While thinking about this idea, I had a similar solution in my mind: Define the topic pattern and let it match parts of it into variables. I didn't suggest this idea because I thought we could also define a pre-defined topic pattern to make it easier in the first place. But your suggestion makes it quite dynamic and opens it up for future ideas not having thought of right now. I'd definitively use this pattern to only run one RuuviBridge for the multiple gateways I have (in various forms).

[Scrin]

Hmm, a "more simple" topic pattern would be much more trivial to configure, especially for users with less regexp knowledge. I like that idea. I wonder if there's any realistic case where a simple pattern match wouldn't be sufficient.

Perhaps something like:

mqtt_listener:
  [...]
  topic_pattern: ruuvi/$Location/$GwMac/$TagMac

influxdb_publisher:
  [...]
  additional_tags:
    location: "{{.MqttTopic.Location}}"
    gw_mac: "{{.MqttTopic.GwMac}}"
    ruuvibridge_version: "{{.RuuviBridge.Version}}"

which would be internally compiled into a regexp that creates named matching groups for those $variables as well as creating the mqtt subscription, ruuvi/+/+/+ for the above example. The special TagMac would be used for the MAC address in the actual measurement data.

Then in case topic_pattern is not defined, the "existing" topic_prefix configuration would be used instead in order to maintain backwards compatibility and to offer a simpler config for users who don't need any of these advanced features.

[tobru]

That would make it even easier to use. While I would be able to write a regexp, I could imagine that just seeing "regexp" could make people think of it to be too complicated. This new proposed pattern is very readable and probably also easily understandable.

[Scrin]

Yeah, exactly, and on top of that using a pattern like this reduces the amount of config compared to the regexp version since the topic pattern would simply replace the topic prefix. Although technically you could create a mqtt subscription based on the regexp, but that'd be much more work than it's worth. And also if the underlying implementation already uses regexp, adding a regexp-config later on would be relatively easy if someone comes up with a weird use-case where a simple pattern match wouldn't be advanced enough.

I guess we should wait for a bit now in case someone else happens to be reading around here and has some ideas. Or in case one of us comes up with even better ideas

[tobru]

I think it would be cool to get that one implemented, doesn't seem that anyone else had a better idea.

[Scrin]

I suppose so. Would you be willing to implement this?

[tobru]

I must admit that I lack the needed Go know how and time. If I find time to learn more, I'm willing to do it. I'll get back to you with a PR when I have one.