Feat: Auto select a cursor value incase of default Incremental sync mode

[vaibhav-datazip]

Problem

Currently when a source doesn't support CDC sync the default sync mode is set as incremental in streams.json . And the cursor field which is used to track new records is set to blank in that case, so that user can select the best suited field for it. But if the user forgets to add any cursor field in it, this will give an error as shown below

"message":"error occurred while reading records: failed to run incremental sync: backfill failed: failed to fetch max cursor values: failed to scan primary cursor value: ERROR: zero-length delimited identifier at or near \"\"\"\" (SQLSTATE 42601)"}

This can lead to problem in olake ui as well. When someone dosen't click the stream even once while creating a job the cursor field won't get populated from ui side, as the streams.json sends it empty.

Solution

Select the available cursor fields 0th value and set it as cursor field while generating the streams.json

[vasubansal1033]

Hi @vaibhav-datazip can you assign this to me?

[Saquib1973]

@vaibhav-datazip is this assigned to someone, or can we work on it in parallel?

[nayanj98]

@Saquib1973 Assigning the issue to you. Please make sure to attach a validation video along with your PR. To know more about the video you can check it out here