schedule large connection jobs over the weekend

[Ishankoradia]

For connections syncing large data, certain jobs that might break the machine can be pushed off to the weekend.
This will only be for select connection_ids which can be called large

• Schema change

• When user opens the schema changes modal, he will see all the changes along with a message Accepting these changes, will schedule a schema job on {Date}

• As soon as user accepts the schema change updates, we will schedule the schema change job (update catalog + reset + sync ) on the nearest Sunday.

• The schema changes will still appear in the Pending Action tab along with the datetime of when it is scheduled.

• If before the next scheduled time, there is a new schema change and the user wants to accept those changes. He can do it and we will update the flow_run params & make sure it again scheduled on the nearest Sunday.

• Store the scheduled_at and flow_run_id in the backend.

• Connection Reset

• When the user presses on the Reset connection button on the ingest page. A modal appears that asks for confirmation.

• The confirmation modal will also have the message On clicking confirm, this will schedule a reset connection job on {Date}

• Store the scheduled_at and the flow_run_id for the upcoming reset job somewhere in the backend

@fatchat

[nairabhishek73]

Schema Change: https://jam.dev/c/3dbdf287-ea6d-4316-8128-9bac2c8d9bb0

1. After I approve I still see it under pending actions, I can open it and then approve it again, confusing loop.

• Once I approve it should indicate to me that its approved and scheduled, no pending action from the users end, only from the system's end.
• If there is another schema change top the same connection it should show up separately as a pending action, if approved it should let me know that the other approval is overridden by the latest one and ask for confirmation

2. Relatedly, but perhaps a separate issue, when I click on view I should see the connection name on the pop up box and then the details of the schema changes within that connection. Currently it doesn't show me connection name in the pop up box.

Tried connection reset, I don;t think that's done yet.