Cache Open-Meteo JSON reponse locally to reduce the number of API calls and add resilience#503
Conversation
|
Hi, there is a pending revision on this PR, can you please check it? |
|
Hi David, sorry I am not sure I understand what the pending revision is. I did ask a question about if you would like a configuration parameter for max_age (via REST and/or config.json). If so then I will change this PR back to draft and keep working on it. |
If you scroll up this PR you can see that I left this message: @paulhomes Could you please keep this commented code snippet? |
|
Hi David, that's really strange as when I look at this PR, the first message I see from you is the one about the pending revision. I don't see any prior messages other than my initial opening comment and the commits (see screenshot below). Could this be a github bug? I can see the commented code block I removed though. Sorry about that. I'll add it back in today.
|
… JSON reponse caching
|
Oh I see, that's very strange indeed you cannot see any review comment. Seems like a github bug |
|
|
I also added a configuration option for the max age of the cached open-meteo json so it can be set differently to the default of 30 minutes if required. |
| + quote(str(self.time_zone), safe="") | ||
| ) | ||
| response = get(url, headers=headers) | ||
| """import bz2 # Uncomment to save a serialized data for tests |
There was a problem hiding this comment.
@paulhomes Could you please keep this commented code snippet?
I know it's ugly but it is used from time to time to save the needed file to mock a request to be used in unit tests
2 participants
This change caches the default 3-day Open-Meteo JSON response locally with a configurable time limit (30 minutes by default). It is intended to resolve issue #492 where the post-subset data stored in the pickle weather_forecast_cache becomes stale quickly and, with regular scheduled 5 min MPCs, it results in frequent Open-Meteo API calls for weather data that does not need to be updated that frequently. I am currently running this in my test emhass docker instance where the Open-Meteo JSON response is cached using this change and I have turned off the pickle weather_forecast_cache.
It is only caching the JSON response from Open-Meteo so there are no changes to the existing pickle weather_forecast_cache code and it should not impact caching for any of the other weather methods.
By keeping the full 3 days in the JSON cache, and continuing to use it if there are network errors, it also means that emhass will be more resilient to any transient connectivity issues with Open-Meteo.
If you like this suggested change then I can try adding a configuration parameter for max_age for those that might want to change the default (via REST and/or config.json)
What are your preferences for a default? I picked 30 minutes to start with but it could be longer like an hour or maybe 3 or 4 hours.