real-time flow rates and pressure

[dulitz]

I just noticed this gem in const.py:

# FIXME: for real time access to flow rates and
#https://firestore.googleapis.com/google.firestore.v1.Firestore/Listen/channel?database=projects%2Fflotechnologies-1b111%2Fdatabases%2F(default)&gsessionid={GSESSION_ID}&VER=8&RID=rpc&SID={SESSION_ID}&CI=0&AID=113&TYPE=xmlhttp&zx=lsvpgbg6aoar&t=2

That looks like what we would need to dig into to get real time access. Has anyone looked into this yet?

Here are the docs for realtime listeners in Cloud Firebase.

[electric-monk]

I think I've got this working. I'll see if I can get a PR for it sorted.

[dulitz]

Awesome, thank you @electric-monk . I pull flo (and a whole lot of other info) into Prometheus using https://github.com/dulitz/porter . Firestore reads are under 4 cents per 100,000 documents so I had been assuming negligible costs to stream all the changes. I was thinking that it might report 100,000 changes over the course of a month and I would get them all.

Now I'm thinking that maybe I will start a listener for every Prometheus poll (every 2 minutes or so), get one callback, and then stop it. This surely is a strange thing they're doing.

[dulitz]

So it turns out that if we hit them with the /presence/me endpoint to get realtime updates, they do an update every second for the next two minutes. This is just about the worst case scenario as far as I'm concerned. I want data every 30 seconds or a minute at most, every 2 minutes would be fine. I don't care about every-second updates. But if I tickle /presence/me every 2 minutes they will be sending updates every second.

Do any of y'all have any contacts at Flo? Because they should do something like "send updates whenever telemetry changes."

[electric-monk]

Yes, I only picked 30 seconds as that's what the meetflo.com web application does, so that's what they're expecting.

I think the real fix would be for them to do something sensible locally, like a UDP broadcast or HTTP endpoint containing that JSON (the WeatherFlow weather station is an example of such a cloud-connected device with a similar local UDP API), as that'd cost them nothing at all.

Sadly I have no contact at Flo personally, but we should probably try and send a message.

Note that there's another bug, if you leave the code running, after an hour the token changes and needs updated, otherwise our code loses access to firestore (and the device will continue to write to it, pointlessly). The fix is to use the provided expiry and pass it into the Credentials, and fix up the Credentials.refresh method to get a new token from Flo's API.

[dulitz]

I agree with you re: UDP broadcast or HTTP endpoint with JSON. The Generac PWRview, formerly Neurio is an example of a device with an unauthenticated HTTP endpoint serving a JSON blob.

Interesting re: token expiry. Either we should clean that up, or we should make sure to stop the heartbeat thread after an hour and notify the listener by passing it an empty dict.

[electric-monk]

I've got a fix for the token expiry in my fork but I'm not super happy with it (especially since we perhaps shouldn't be encouraging this usage, and so just having it shut down would be preferable).

[dulitz]

Yeah I'm so torn between "wow, you did it and it's fantastic!" and "benefit to me versus cost to them is really out of whack." I started this morning intending to find a way to get realtime stats into Porter, but everything I can think of just doesn't make sense.

120 updates from one ping! Couldn't it have been 30 seconds at one update per 5 seconds or something y'all??? :)