Hi there!
When you have a very large number stored as a postgres numeric type, the cdc will automatically try to put this as a Decimal type in clickhouse, which will fail. See disscussion:
There has been good progress with #2049 etc which is massively appreciated, but from what I understand the actual data type conversion code still needs to be implemented so that syncing is successful. Mapping it instead from the Numeric in Postgres to Int256 type in clickhouse would work in our case. I'm sure it would also be very valuable for other teams to have control over types in various cases.
This issue is blocking us from using PeerDB as our CDC keeps failing with this essential data that we would need.
Would hugely appreciate any thoughts on a possible timeline or even hacky hard coded case we could use just for our use case. Knowing you thoughts on this issue and any timeline would be extremly appreciated!
Thanks so much in advance for your time in reviewing this.
Hi there!
When you have a very large number stored as a postgres numeric type, the cdc will automatically try to put this as a Decimal type in clickhouse, which will fail. See disscussion:
There has been good progress with #2049 etc which is massively appreciated, but from what I understand the actual data type conversion code still needs to be implemented so that syncing is successful. Mapping it instead from the Numeric in Postgres to Int256 type in clickhouse would work in our case. I'm sure it would also be very valuable for other teams to have control over types in various cases.
This issue is blocking us from using PeerDB as our CDC keeps failing with this essential data that we would need.
Would hugely appreciate any thoughts on a possible timeline or even hacky hard coded case we could use just for our use case. Knowing you thoughts on this issue and any timeline would be extremly appreciated!
Thanks so much in advance for your time in reviewing this.