**Describe the solution you'd like** Run the data pipeline update job automatically and on a schedule. **Describe alternatives you've considered** Manually triggering job runs - this is what is happening right now. **Additional context** Would have to persist the LAST_METADATA_UPDATE_FIELD value and update it after each successful job run.
Describe the solution you'd like
Run the data pipeline update job automatically and on a schedule.
Describe alternatives you've considered
Manually triggering job runs - this is what is happening right now.
Additional context
Would have to persist the LAST_METADATA_UPDATE_FIELD value and update it after each successful job run.