Skip to content

status thread overwrites checkpoint information when REDIS_CHECKPOINT is enabled #625

@ikavalio

Description

@ikavalio

Hi,

I've been testing some setups with REDIS_CHECKPOINT=true and noticed that my keys in redis contain txid_current but not checkpoint info most of the time. Sometimes the situation reverts and I only have checkpoint there.

After checking some code my theory is that status thread overwrites the complete document at regular interval in

self.redis.set_meta({"txid_current": self.txid_current})

As set_meta just runs redis SET command:

pgsync/pgsync/redisqueue.py

Lines 114 to 116 in 33eae21

def set_meta(self, value: t.Any) -> None:
"""Store an arbitrary JSON-serialisable value in a dedicated key."""
self.__db.set(self._meta_key, json.dumps(value))

Checkpointer's setter does the same:

self.redis.set_meta({"checkpoint": value})

This is not a problem when REDIS_CHECKPOINT is disabled as far as I can tell, because both commands operate on 2 different files.

If the above is correct, I believe the code either needs to use 2 different redis keys, or use HSET / HGET for field level updates to prevent interference

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions