-
-
Notifications
You must be signed in to change notification settings - Fork 210
Description
Hi,
I've been testing some setups with REDIS_CHECKPOINT=true and noticed that my keys in redis contain txid_current but not checkpoint info most of the time. Sometimes the situation reverts and I only have checkpoint there.
After checking some code my theory is that status thread overwrites the complete document at regular interval in
Line 2032 in 33eae21
| self.redis.set_meta({"txid_current": self.txid_current}) |
As set_meta just runs redis SET command:
Lines 114 to 116 in 33eae21
| def set_meta(self, value: t.Any) -> None: | |
| """Store an arbitrary JSON-serialisable value in a dedicated key.""" | |
| self.__db.set(self._meta_key, json.dumps(value)) |
Checkpointer's setter does the same:
Line 1614 in 33eae21
| self.redis.set_meta({"checkpoint": value}) |
This is not a problem when REDIS_CHECKPOINT is disabled as far as I can tell, because both commands operate on 2 different files.
If the above is correct, I believe the code either needs to use 2 different redis keys, or use HSET / HGET for field level updates to prevent interference